Renting an A100 on Lambda Labs is easy. Treating it as a generic “fast GPU” is where a codebase quietly acquires vendor lock. The instance gives you very specific hardware — third-generation tensor cores, 40 or 80GB of HBM2e, and NVLink topology — and the numbers you get depend on code that assumes those exact characteristics. That is fine on a single A100 target. It becomes technical debt the moment you need to run the same workload on an AMD MI-series card, an Intel data-centre GPU, or a cheaper NVIDIA part with different memory. The question worth asking before you spin up an instance is not “how fast is the A100?” but “which of the speed-ups I’m about to rely on are properties of my algorithm and which are properties of this device?” The two look identical in a benchmark. They diverge sharply when you migrate. How does a Lambda Labs A100 work in practice? Lambda Labs rents NVIDIA A100 GPUs by the hour, on-demand or reserved, as bare instances or inside multi-GPU systems. When you SSH in, you get a machine with one or more A100s, a CUDA driver stack, and whatever container or image you bring. Practically, it behaves like a local A100 workstation you don’t own — the same CUDA, the same nvidia-smi, the same PyTorch or TensorRT you’d run on-prem. That familiarity is the trap: because the software stack is identical to a physical box, teams write code exactly as they would for owned hardware, including all the device-specific assumptions. Two A100 memory configurations exist and the difference is not cosmetic. Per NVIDIA’s published specifications, the A100 ships in 40GB and 80GB HBM2e variants, with the 80GB part offering roughly 2TB/s of memory bandwidth versus about 1.5TB/s on the 40GB SXM part. If your model’s working set spills past 40GB, the 80GB instance is not a nice-to-have — it changes whether the workload runs at all without sharding. Sizing the instance to your actual memory footprint, rather than defaulting to the largest option, is the single clearest lever on hourly spend (an observed pattern across cloud-GPU engagements, not a published rate). What does an NVIDIA A100 actually offer, and how does that map to workloads? Strip away the marketing and the A100 gives you three things that matter for AI work: high-bandwidth memory, tensor cores tuned for reduced-precision matrix math, and fast device-to-device links. Each maps to a workload class, and each is a potential lock-in surface. A100 capability What it accelerates Portability risk if hardcoded HBM2e (40/80GB, ~1.5–2TB/s) Large-batch inference, big activations, KV caches Assuming a fixed bandwidth-to-compute ratio when tiling 3rd-gen tensor cores (TF32, BF16, FP16, INT8) Transformer and CNN matmuls Precision + tile shapes tuned to tensor-core geometry NVLink / NVSwitch Multi-GPU sharding, all-reduce Topology-specific collective patterns MIG (Multi-Instance GPU) Partitioning one A100 into isolated slices Slice-count assumptions baked into the scheduler TF32 is the clearest example of the double edge. It lets the A100 run FP32 matmuls through tensor cores at much higher throughput with minimal code change — a genuine A100 win. But the numerical behaviour is NVIDIA-specific, and if you tune convergence or accuracy thresholds around TF32 output, you are relying on a device property. On an AMD or Intel target that precision path simply is not there. This is the same class of concern we unpack in what the choice between OpenCL and CUDA means when porting AI workloads across GPUs: the fast path and the portable path are rarely the same path. Which A100 performance patterns are safe to rely on — and which lock your code to NVIDIA? This is the core distinction, and it is worth being precise. A performance pattern is portable if it expresses an intent the compiler or runtime can re-satisfy on other hardware. It is locked if it encodes a number or geometry that only the A100 possesses. Portable intents — safe to rely on because any modern GPU can honour them: “This kernel is memory-bound; keep the working set resident.” Every GPU has a memory hierarchy; the boundary just moves. “Use the widest reduced-precision type the device supports.” BF16 and FP16 exist across vendors even where TF32 does not. “Overlap host-to-device transfer with compute.” PCIe, Infinity Fabric, and Intel’s Xe links all reward this. Locked assumptions — the ones that turn a migration into a rewrite: Hardcoded tensor-core tile shapes (e.g. 16×16×16 fragment sizes) written into the algorithm rather than requested from a library like cuBLAS or CUTLASS. Collective-communication code that assumes NVLink bandwidth and NVSwitch topology, so it falls over on PCIe-connected or differently-wired multi-GPU boxes. Precision thresholds calibrated against TF32 rounding. Explicit HBM2e bandwidth constants used to compute batch sizes or prefetch depths. The divergence point is architectural, not incidental: it is the moment you write an A100 characteristic into your algorithm instead of into a tuned-but-swappable layer. A CUTLASS call that says “give me the best GEMM for this problem on this device” is portable in spirit even if the CUDA implementation is not — you can swap the backend. A hand-rolled kernel with A100 tile sizes stamped in is not. If your inference path is already CUDA-bound and you want to see what unwinding it costs, porting a GPU inference path off the CUDA lock-in with HIP walks through the concrete mechanics. How do I size a Lambda Labs A100 instance without over-provisioning? Over-provisioning on rented GPUs is not usually a compute problem — it is a memory-and-utilisation problem. Teams pick the 80GB, 8-GPU instance because it “won’t run out,” then run a workload that keeps two GPUs at 30% utilisation. The hourly meter does not care. A workable sizing rubric, in order: Measure the memory high-water mark first. Run your workload on the smallest instance that fits and read peak allocated memory (torch.cuda.max_memory_allocated() or equivalent). If it clears 40GB comfortably, you never needed the 80GB part. Check whether you are memory-bandwidth-bound or compute-bound. Profile with Nsight Systems or the PyTorch profiler. A memory-bound kernel will not go faster on more tensor cores; a compute-bound one will not benefit from more HBM. Only add GPUs when a single A100 is genuinely saturated. Multi-GPU adds NVLink/NCCL overhead and communication code. If one A100 runs at 85%+ sustained utilisation and you still need more throughput, scale out — otherwise you are paying for idle silicon and inviting topology lock-in. Match precision to the workload, not the hardware. If INT8 or BF16 holds your accuracy, you get more effective throughput per rented hour without touching the instance type. Getting instance sizing right is one input into a broader picture of where GPU cost actually hides — the kind of analysis we run in a full GPU performance and portability review. Sizing tells you what to rent; the audit tells you why the workload costs what it does. If I tune for the A100 now, how much carries over to AMD or Intel later? Honestly: it depends entirely on where you put the tuning. This is not a hedge — it is the whole point. If your A100 optimisations live in a swappable acceleration layer — a GEMM library call, a precision policy chosen at runtime, a collective-communication abstraction — then a move to an AMD MI300 or an Intel data-centre GPU is a re-tune. You re-profile, adjust tile heuristics, pick the target’s best reduced-precision type, and re-measure. In our experience that is a matter of weeks, and much of it is measurement rather than code change (observed across porting engagements; not a benchmarked figure). If your A100 optimisations are fused into the algorithm — tile shapes, bandwidth constants, NVLink-shaped collectives, TF32-calibrated thresholds — then the same move is a rewrite, and the difference is months, not weeks. The code does not “run slower” on the new target; it produces wrong results or fails to compile, because the assumptions no longer hold. The structural lesson is the one that shows up across the A100 line generally, including on a physical box — we cover the on-prem version in what an A100 workstation means for portable GPU code. The rented A100 changes the cost math (you pay by the hour, so waste is visible) but not the engineering principle: separate the intent from the device. How should I structure A100-tuned code so a future migration is a re-tune, not a rewrite? The pattern is a boundary. On one side, hardware-agnostic logic that expresses what the computation is. On the other, a thin, replaceable layer that expresses how to make it fast on the current device. A quick self-check: Are your matmuls going through cuBLAS/CUTLASS, or are they hand-tiled? Library calls carry the tuning; hand-tiled kernels carry the lock. Is precision a runtime policy or a compile-time constant? A dtype chosen from device capability is portable; TF32 baked into thresholds is not. Do your collectives go through an abstraction (NCCL behind an interface) or assume NVLink directly? An abstraction can be swapped for RCCL or oneCCL. Are memory sizes derived from a query (total_memory) or hardcoded? Derived values re-tune themselves; constants do not. Where you land on those four questions predicts your migration cost more reliably than any benchmark score. Characterising exactly how a workload behaves on a specific rented target like the A100 — which patterns are portable, which are locked — is one input to a GPU Performance Audit, the same discipline behind our OpenCL-to-Metal porting work for V-Nova. FAQ How does lambda labs a100 work? Lambda Labs rents NVIDIA A100 GPUs by the hour as bare instances or inside multi-GPU systems, with a standard CUDA driver stack. In practice it behaves exactly like a local A100 workstation you don’t own — which is why teams write device-specific code without noticing. The 40GB and 80GB HBM2e variants matter: choosing the right one is the clearest lever on hourly spend. What does an NVIDIA A100 actually offer — memory, tensor cores, NVLink — and how does that map to workloads? The A100 offers HBM2e (40/80GB at roughly 1.5–2TB/s per NVIDIA’s specs), third-generation tensor cores supporting TF32, BF16, FP16 and INT8, and NVLink/NVSwitch for fast multi-GPU links. Memory maps to large-batch inference and KV caches, tensor cores to matmul-heavy transformers and CNNs, and NVLink to multi-GPU sharding. Each is also a lock-in surface if its specifics get baked into your algorithm. Which A100-specific performance patterns are safe to rely on, and which lock your code to NVIDIA? Portable patterns express intent a runtime can re-satisfy elsewhere: treating a kernel as memory-bound, using the widest reduced-precision type the device supports, overlapping transfer with compute. Locked assumptions encode A100-only numbers: hardcoded tensor-core tile shapes, NVLink-topology collectives, TF32-calibrated precision thresholds, and explicit HBM2e bandwidth constants. The divergence point is writing a device characteristic into the algorithm rather than into a swappable layer. How do I size a Lambda Labs A100 instance without over-provisioning cloud spend? Measure your memory high-water mark on the smallest instance that fits before defaulting to 80GB or multi-GPU. Profile whether you are memory-bandwidth-bound or compute-bound, because more tensor cores won’t help a bandwidth-bound kernel. Only add GPUs when a single A100 is genuinely saturated (85%+ sustained), and match precision to the workload rather than the hardware. If I tune my code for the A100 now, how much of that carries over to AMD or Intel GPUs later? It depends on where the tuning lives. Optimisations in a swappable acceleration layer — library GEMM calls, runtime precision policy, collective abstractions — carry over as a re-tune measured in weeks. Optimisations fused into the algorithm — tile shapes, bandwidth constants, NVLink-shaped collectives — force a rewrite measured in months, because the assumptions no longer hold on the new target. How should I structure A100-tuned code so a future vendor migration is a re-tune, not a rewrite? Draw a boundary between hardware-agnostic logic that expresses what the computation is and a thin, replaceable layer that expresses how to make it fast on the current device. Route matmuls through cuBLAS/CUTLASS rather than hand-tiling, make precision a runtime policy, put collectives behind an abstraction like NCCL, and derive memory sizes from device queries rather than constants. How you answer those four questions predicts migration cost better than any benchmark. The A100 is excellent hardware, and renting it is a sensible way to get access without capital outlay. The mistake is confusing “this runs fast here” with “this is well-engineered.” Fast on one rented device is a benchmark result; portable-and-fast is an engineering property — and only the second one survives the day the invoice, the vendor, or the roadmap changes.
