Run accelerate config once, hit Enter through the prompts, and forget the YAML it writes — the workload runs, the defaults hold, and nobody looks again. That works right up to the moment you add a second GPU or move to a different node. Then the interactive defaults quietly leave devices idle, or the run fails on launch, and it’s rarely obvious why. The core point of this article is simple: accelerate config does not “set up your GPUs.” It writes a portable description of an execution topology — how many processes to spawn, which device each one binds to, what mixed-precision policy applies, and which distributed backend coordinates them. Treat that YAML as a throwaway side effect and you lose the one artifact that decides whether restructured, parallel code actually reaches the hardware. Treat it as a version-controlled description of where and how the work runs, and moving between a single-GPU workstation and a multi-GPU cluster stops being a source of silent divergence. What does accelerate config actually do? Hugging Face Accelerate sits between your training or compute loop and the distributed machinery underneath it — torch.distributed, DeepSpeed, or Fully Sharded Data Parallel. The accelerate config command is the interactive step that records how you want that machinery wired up for a given machine. It asks a series of questions and writes the answers to a YAML file (by default ~/.cache/huggingface/accelerate/default_config.yaml), which accelerate launch then reads to spawn and place processes. The mental model that trips people up is thinking of the config as a one-time environment setup, like installing a driver. It isn’t. It’s closer to a deployment manifest: a declarative statement of the parallel layout the code expects to run under. The same Python script, launched against two different config files, runs as a single-process job on one and a four-way data-parallel job on the other — without changing a line of the model code. That decoupling is the whole point, and it’s also why the config deserves more attention than a hurried pass through the prompts. If you’re new to the surrounding tool, our explainer on what Hugging Face Accelerate does across multi-GPU training and inference sets the broader context; this article focuses narrowly on the config file itself and what each field controls. What does each setting in the generated YAML control? The interactive prompts map onto a handful of fields that together define the execution topology. Understanding what each one governs is what separates “the run works on my machine” from “the run works, reproducibly, on the cluster too.” YAML field What it controls Where it bites if wrong distributed_type The backend: NO, MULTI_GPU, DEEPSPEED, FSDP Wrong backend → sharding never happens, or a large model OOMs on a single device num_processes How many worker processes to spawn (one per GPU, typically) Left at 1 on a multi-GPU node → all but one GPU sit idle num_machines / machine_rank Node count and this node’s rank in a multi-node run Mismatch across nodes → processes never rendezvous; launch hangs or fails gpu_ids Which physical devices this run may use all on a shared node collides with other jobs; explicit IDs isolate them mixed_precision no, fp16, or bf16 numeric policy fp16 on hardware without stable support → NaNs; no on capable hardware → throughput left on the table main_process_ip / main_process_port The rendezvous endpoint for multi-node coordination Unreachable IP/port → the classic multi-node hang with no useful error The single most common under-utilisation cause we see is a num_processes: 1 config carried over from a single-GPU workstation onto a multi-GPU node. Nothing errors. The job runs. It just runs on one of the eight GPUs present, and the other seven idle at near-zero utilisation the whole time (observed pattern across GPU-porting engagements, not a benchmarked rate). The fix is trivial once you know to look; the cost is the days of wall-clock time burned before anyone checks nvidia-smi. The mixed_precision field deserves its own note because it interacts with hardware. bf16 needs Ampere-class silicon or newer (per NVIDIA’s published architecture specifications, bfloat16 tensor-core support arrives with the A100 generation); requesting it on older cards falls back or fails. For the precision-vs-throughput trade-off in depth, the 4-bit floating-point explainer on when reduced precision pays off covers the reasoning that applies here at a coarser granularity. When do the interactive defaults under-utilise a node, and how do you tell? The defaults are tuned for the machine you ran accelerate config on. That’s the trap. A config authored interactively on a single-GPU laptop encodes “one process, no distributed backend” — and it will happily carry those assumptions onto hardware that can do far more. Three diagnostic signals catch this before it wastes a run: Utilisation floor. If nvidia-smi shows one GPU busy and the rest near-idle during a job you expected to parallelise, num_processes is almost certainly stuck at 1. This is the fastest tell and the most common failure. Backend mismatch on model size. A model that OOMs on a single device but was supposed to shard is a sign distributed_type is MULTI_GPU (which replicates) rather than FSDP or DEEPSPEED (which shard). Our walkthrough of how Accelerate FSDP shards large models across GPUs explains that distinction and when replication simply can’t fit the weights. Silent precision fallback. Throughput far below what the hardware’s tensor cores should deliver, with no error, often means mixed_precision: no when bf16 was available. None of these throw. That’s exactly why they persist. Utilisation observability — treating idle GPUs as a monitored signal rather than something you notice by accident — turns these into caught-at-launch problems rather than discovered-a-week-later ones. Our piece on the three pillars of observability applied to GPU utilisation frames why the idle-GPU signal belongs in your dashboards, not just your memory. Moving a config between a workstation and a cluster without silent divergence The portability failure mode is subtle: the config that’s correct on your workstation is silently wrong on the cluster, because fields like num_processes, gpu_ids, num_machines, and the rendezvous endpoint are machine-specific by nature. Copy the file verbatim and you inherit workstation assumptions on cluster hardware. The discipline that avoids this is to keep environment-specific configs as named, version-controlled files rather than relying on the single default the interactive command writes. Concretely: Author a config_dev.yaml (single GPU, distributed_type: NO) and a config_cluster.yaml (multi-GPU, correct num_processes, appropriate backend), both checked into the repo alongside the code. Select at launch with accelerate launch --config_file config_cluster.yaml your_script.py — the same script, a different topology, chosen explicitly. For fields that genuinely vary per node in a multi-node run (machine_rank, main_process_ip), drive them from environment variables set by the scheduler rather than baking them into the file. This is where the config earns its keep as a reproducibility artifact. The same YAML moves the workload across dev, CI, and production hardware, and any divergence is visible in a diff rather than hidden in an interactive session nobody recorded. The mechanics of how accelerate launch consumes these files — and why the launch step is itself part of the algorithmic restructuring, not a wrapper around it — are covered in how accelerate launch works and treats launch config as algorithmic restructuring. How does the config fit a redesigned-for-parallelism workflow? There’s a sequence that matters here, and getting it out of order is a common way to waste effort. accelerate config is not where parallelism is created. It’s where parallelism that already exists in the code gets mapped onto real devices. If an algorithm hasn’t been restructured to expose parallel work — batches that can be split, gradients that can be reduced, model shards that can live on separate devices — then no config will help. The config expresses a topology; it can’t manufacture one. This is why the honest first question for any compute-heavy workload is whether it’s parallelisable at all, which is the job of a GPU audit long before anyone touches a YAML file. Once that redesign is done, the config becomes the concrete, portable place where “this work can run four ways in parallel” turns into “these four processes bind to these four devices under this backend.” For simulation and compute-heavy planning workflows — RF planning, physics scenarios, large scenario sweeps — that mapping is what moves the workload from evaluating scenarios per week to evaluating them per day. The config is small; its leverage is not. The GPU engineering practice at TechnoLynx treats this ordering — audit for parallelisability first, express the topology second — as the through-line, because a well-authored config on unparallelised code buys nothing, and a careless config on well-parallelised code throws the redesign away at the last step. It’s worth noting that these decisions differ sharply by target. A config tuned for server-side multi-GPU simulation has almost nothing in common with the constraints of browser or mobile inference, where there is no cluster to configure and the parallelism story is entirely different. That divergence is a topic in its own right, and one worth keeping in mind whenever “just reuse the config” seems tempting. FAQ What does working with accelerate config involve in practice? accelerate config runs an interactive questionnaire and writes the answers to a YAML file that describes your execution topology — process count, device placement, mixed-precision policy, and distributed backend. accelerate launch reads that file to spawn and place processes, so the same script runs single-GPU or multi-GPU depending purely on which config it points at. In practice it decouples the parallel layout from the model code. What does each setting in the generated accelerate config YAML actually control? distributed_type picks the backend (single, MULTI_GPU, DEEPSPEED, or FSDP); num_processes sets how many workers spawn, usually one per GPU; gpu_ids selects which physical devices the run may use; mixed_precision sets the numeric policy (no, fp16, bf16); and num_machines/machine_rank/main_process_ip coordinate multi-node runs. Each maps to a distinct failure mode when misconfigured, from idle GPUs to launch hangs. When do the interactive accelerate config defaults under-utilise a multi-GPU node, and how do I tell? The defaults encode the machine you ran the command on, so a config authored on a single-GPU workstation carries num_processes: 1 onto a multi-GPU node and leaves the extra devices idle. The clearest tell is nvidia-smi showing one GPU busy and the rest near-zero during a job you expected to parallelise. A model that OOMs when it should shard, or throughput far below the hardware’s tensor-core ceiling, are the other two signals — none of which throw an error. How do I move an accelerate configuration between a single GPU workstation and a multi-GPU cluster without silent divergence? Keep named, version-controlled config files per environment (for example config_dev.yaml and config_cluster.yaml) rather than relying on the single interactive default, and select one at launch with --config_file. Drive genuinely per-node fields like machine_rank and main_process_ip from scheduler-set environment variables. Any divergence then shows up in a diff instead of hiding in an unrecorded interactive session. How does accelerate config fit into a workflow where an algorithm has been redesigned to expose GPU parallelism? It comes after the redesign, not before it. accelerate config maps existing parallelism onto real devices — it cannot manufacture parallelism in code that hasn’t been restructured to expose it. The correct order is: audit the workload for parallelisability, restructure it, then use the config to express that structure as a concrete, portable device topology. Should the accelerate config YAML be version-controlled, and why does treating it as a portability artifact matter? Yes. The config is the reproducibility artifact that lets the same workload move across dev, CI, and production hardware without silent divergence, because the topology decisions live in a file you can diff and review rather than in an interactive session. Treating it as a throwaway side effect is how workstation assumptions leak onto cluster hardware unnoticed. When a multi-GPU run under-delivers, the config file is the first place to look — an idle-GPU failure is almost always an execution-topology description that no longer matches the hardware it landed on, and that mismatch is exactly what a GPU audit is meant to catch before the redesign ever reaches a device.
