Run accelerate config once, accept every default, and you get a wizard-blessed setup that looks correct and quietly wastes half your hardware. The prompts read like a form. They are actually a set of engineering decisions about how your CV model touches memory, numerics, and multiple devices — and the wizard cannot make those decisions for you because it does not know your model’s memory profile or how sensitive your detection head is to reduced precision. That gap is where beginners lose weeks. A multi-GPU training run that scales no better than a single card, or a mixed-precision setting that silently erodes small-object detection accuracy, both trace back to the same root cause: the config was filled in, not reasoned about. This article reads each prompt as the engineering choice it encodes, so you can stand up a correct Hugging Face Accelerate training setup in your first weeks with computer vision rather than debugging silent misconfiguration for months. How does accelerate config work in practice? accelerate config is an interactive wizard that writes a YAML file describing how your training script should be launched — how many machines, how many GPUs, which distributed backend, and what numerical precision to use. When you later run accelerate launch train.py, Accelerate reads that file and sets up the process group, device placement, and precision context for you. The same training loop then runs single-GPU, multi-GPU, or multi-node with no code change. The value proposition is real: Accelerate abstracts away the boilerplate of torch.distributed — process spawning, DistributedDataParallel wrapping, gradient synchronization — so your PyTorch training loop stays close to what you’d write for a single device. But the abstraction only holds if the config underneath it matches your workload. The wizard writes whatever you tell it. If you tell it the wrong thing, it will faithfully launch a wrong configuration and give you no error, because a misconfiguration is not a crash — it’s a slow, correct-looking run. The naive mental model is “the wizard sets things up correctly.” The accurate model is “the wizard records the decisions I’ve already made.” Everything below is about making those decisions well. What does each config prompt actually decide? The prompts arrive in a fixed order, and each one commits you to something concrete. Reading them as a sequence of engineering questions — rather than a survey to click through — is the whole discipline. Prompt What it looks like What it actually decides Compute environment “This machine” vs. cloud (SageMaker, etc.) Where the launcher spawns processes and how it discovers peers Distributed type No distributed / multi-GPU / multi-node / TPU Whether your loop runs one process or a synchronized process group Number of machines 1, 2, … Multi-node collective communication (NCCL over the network) Number of processes Usually = GPU count How many replicas of your model exist; effective batch = per-device × count Mixed precision no / fp16 / bf16 / fp8 The numerical format for forward/backward; memory footprint and accuracy risk Backend detail DDP / FSDP / DeepSpeed options How model state and gradients are sharded across devices The single most consequential answer for a CV workload is the pairing of distributed type and mixed precision. Those two together determine whether you get near-linear throughput scaling and whether your model still converges to the accuracy you validated at full precision. Get the count right but the precision wrong, and you’ll train fast to a worse model. Get the precision right but leave DistributedDataParallel unconfigured, and you’ll pay for eight GPUs to do the work of one. When should I choose bf16 versus fp16? This is the prompt that most often causes silent damage, so it deserves the most care. Both fp16 and bf16 are 16-bit floating-point formats that halve activation and gradient memory versus fp32, which is why they matter: the freed memory is what lets you train at production batch sizes on a fixed card. The difference is where they spend their 16 bits. fp16 gives more mantissa bits (higher precision) but a narrow exponent range, so it can overflow or underflow during training — which is why it needs dynamic loss scaling to stay numerically stable. bf16 keeps the full fp32 exponent range but sacrifices mantissa bits, so it rarely overflows and generally needs no loss scaling, at the cost of coarser precision per value. For most CV training in our experience, bf16 is the safer default on hardware that supports it (NVIDIA Ampere and later — A100, H100, RTX 30/40 series). The wide dynamic range tolerates the large gradient swings common in detection and segmentation training without the loss-scaling gymnastics fp16 requires (observed across training-setup engagements; not a published benchmark). Reach for fp16 when you’re on older hardware without bf16 support (V100, T4), and be ready to tune the loss scale if training diverges. Where numerical sensitivity bites hardest is small-object and dense detection, where a metric like [email protected] can quietly drop a point or two under aggressive precision reduction that looked fine on the training loss curve. The rule that holds up: validate the mixed-precision run against a short fp32 baseline before you commit a long run to it. If the two converge to comparable validation accuracy in the first few epochs, the precision choice is safe. If they diverge early, your model is more numerically sensitive than the default assumed — and no wizard would have caught that. For teams pushing precision further down toward 8- or 4-bit for inference, the same sensitivity reasoning applies; our note on when 4-bit floating point earns its place walks the accuracy trade-off in more detail. Which distributed backend fits which CV workload? The wizard’s distributed-type answer, plus its follow-up detail prompts, selects one of three backends. They are not interchangeable, and the right one depends on whether your bottleneck is throughput or memory. DDP (DistributedDataParallel) replicates the full model on every GPU and synchronizes gradients each step. It’s the default for good reason: for a detector or classifier that fits comfortably in one card’s memory, DDP gives near-linear throughput scaling with minimal overhead. Most CV training — YOLO-family detectors, ResNet/ViT backbones, segmentation heads — belongs here. FSDP (Fully Sharded Data Parallel) shards model parameters, gradients, and optimizer state across GPUs, so each device holds only a slice. You want this when a single model replica no longer fits — large ViT backbones, high-resolution segmentation, or multimodal models that pair a vision encoder with a language model. FSDP trades some communication overhead for the ability to train models that DDP cannot fit at all. DeepSpeed offers ZeRO-stage sharding similar in spirit to FSDP, plus CPU/NVMe offload for extreme memory pressure. It shines on the largest multimodal training runs where even sharded state strains GPU memory. We cover the config path in detail in scaling multimodal training with Accelerate and DeepSpeed. The decision rubric is short. Does one model replica fit on one GPU with room for your target batch? Use DDP. Does it not fit, but sharding it across your cards would? Use FSDP. Are you memory-bound even after sharding, or already invested in the DeepSpeed ecosystem? Use DeepSpeed. Picking FSDP or DeepSpeed for a model that DDP handles fine just adds communication overhead and slows you down — the more advanced backend is not the better one, it’s the one that matches your memory constraint. How do I verify the config before a full training run? Never launch a multi-day run against an unverified config. The whole point of reading the prompts as decisions is that you can check each decision cheaply before you pay for it. A pre-flight checklist that catches the common failures: Inspect the file. Run accelerate env or open the YAML directly. Confirm distributed_type, num_processes, and mixed_precision match what you intended. This 30-second check catches the most common error — a stale config from a different machine. Confirm device count. num_processes should equal the GPUs you expect to use. If it says 1 on an eight-GPU box, your “multi-GPU” run is single-GPU. Run a smoke test. Launch accelerate launch train.py for a handful of steps on a tiny data slice. Watch that all GPUs show utilization (via nvidia-smi) and that per-step time drops roughly proportionally to GPU count. Flat utilization on all but one card means DDP isn’t actually distributing. Sanity-check effective batch size. Effective batch = per-device batch × num_processes × any gradient accumulation. If your learning-rate schedule was tuned for a different effective batch, convergence will look wrong even with a perfect config. Precision A/B. Run a short mixed-precision segment against a short fp32 segment and compare early validation accuracy, per the check in the precision section above. Instrument the smoke test the way you’d instrument the real run — throughput, per-step time, and utilization — so the verification and the production run speak the same language. If you’re not already capturing these systematically, our guidance on what to track in CV pipelines with TensorBoard and on why CV teams need experiment tracking covers the metrics that make a misconfiguration visible instead of silent. Where does the config file live, and how do I manage several? By default the wizard writes to ~/.cache/huggingface/accelerate/default_config.yaml. That single default is fine on one machine, but it’s the source of a recurring pain: the config that’s correct on your four-GPU dev box is wrong on an eight-GPU production node, and the shared default silently carries the wrong num_processes between them. The maintainable pattern is to keep named config files under version control alongside your training code — configs/dev_4gpu.yaml, configs/prod_8gpu.yaml — and pass them explicitly: accelerate launch --config_file configs/prod_8gpu.yaml train.py. This makes the launch environment reproducible and reviewable, the same discipline you’d apply to shipping CV stages to production. A config in Git is a config a teammate can read, diff, and correct — a config in a home-directory cache is a mystery waiting to bite the next person who trains on that box. This kind of grounded training-infrastructure setup is exactly what a team building out its computer vision capability should get right in the first weeks, not the last. FAQ What should you know about accelerate config in practice? accelerate config is an interactive wizard that writes a YAML file recording how your training script should launch — machine count, GPU count, distributed backend, and mixed precision. accelerate launch then reads that file and sets up process spawning, device placement, and precision so the same PyTorch loop runs single-GPU, multi-GPU, or multi-node unchanged. In practice the wizard records decisions you’ve already made; it does not make them for you, so a wrong answer produces a slow, correct-looking run rather than an error. What does each accelerate config prompt actually decide for a CV training run? The prompts commit you to concrete engineering choices: compute environment sets where processes spawn; distributed type decides whether you run one process or a synchronized group; process count fixes how many model replicas exist and your effective batch size; mixed precision sets the numerical format and memory footprint; backend detail governs how model state is sharded. The most consequential pairing for CV is distributed type and mixed precision, which together determine throughput scaling and whether the model still converges to validated accuracy. When should I choose bf16 versus fp16 mixed precision, and how does that interact with model numerical sensitivity? bf16 keeps the full fp32 exponent range but coarser precision, so it rarely overflows and usually needs no loss scaling — the safer default on Ampere-and-later hardware for most CV training. fp16 gives finer precision but a narrow range, requiring dynamic loss scaling and best reserved for older cards without bf16 support. Small-object and dense detection are the most numerically sensitive; validate any mixed-precision run against a short fp32 baseline before committing a long run. Which distributed backend (DDP, FSDP, DeepSpeed) fits which CV workload, and how do I select it in the wizard? Use DDP when one model replica fits on one GPU with room for your batch — it gives near-linear scaling for most detectors, backbones, and segmentation heads. Use FSDP when a single replica no longer fits and sharding parameters and optimizer state across cards would make it fit, such as large ViT or multimodal models. Use DeepSpeed for extreme memory pressure needing CPU/NVMe offload. Selecting a more advanced backend than your memory constraint requires only adds communication overhead. How do I verify my accelerate config is correct before launching a full training run? Inspect the YAML (accelerate env) to confirm distributed type, process count, and precision match intent; confirm num_processes equals your GPU count; run a few-step smoke test watching that all GPUs show utilization and per-step time drops with GPU count; sanity-check effective batch size against your learning-rate schedule; and A/B a short mixed-precision segment against fp32. These cheap checks catch the silent failures a full run would only reveal days later. Where does the accelerate config file live, and how do I maintain multiple configs across dev and production hardware? By default the wizard writes to ~/.cache/huggingface/accelerate/default_config.yaml, but that single shared default silently carries the wrong process count between machines with different GPU layouts. Keep named config files under version control alongside your training code — for example configs/dev_4gpu.yaml and configs/prod_8gpu.yaml — and pass them explicitly with --config_file. A config in Git is reviewable and reproducible; a config in a home-directory cache is a mystery waiting to bite the next person. The question the wizard can’t answer for you Every prompt in accelerate config is really asking the same underlying question: what does your model need from your hardware? The wizard cannot answer that, because it has never seen your memory profile, your batch requirements, or how much accuracy your detection head loses when you drop a bit of mantissa. When a multi-GPU CV run doesn’t scale or a mixed-precision setting quietly costs you accuracy, the failure class is almost never Accelerate itself — it’s a config filled in as a form when it should have been reasoned through as a decision. Get that reasoning right in the first weeks, and the difference between a first-shippable training pipeline and a quarter lost to distributed-training debugging is often just this one file.
