Advantage actor-critic is usually described as a training recipe: two networks, a reward signal, and enough GPU headroom to iterate. That framing is fine for training and misleading the moment the policy has to run on a phone. On the edge, the question is not “did A2C converge?” but “which parts of what A2C built actually have to ship, and what do they cost per inference step?” That is where most edge-agent postmortems go sideways. A team trains a competent policy in a data-centre loop, packages the whole thing, and discovers the binary is twice the size it needs to be and the cold-start time blows the on-device budget — because half of what they packaged never runs at inference. Separating the training-time architecture from the inference-time footprint is the single most useful thing you can do before an actor-critic policy reaches the edge boundary. How does advantage actor-critic work, and what actually runs at inference? A2C keeps two learned components during training. The actor is the policy: given a state, it outputs an action (or a distribution over actions). The critic is a value function: given a state, it estimates the expected return from that point onward. The advantage term is the bridge between them — it measures how much better a chosen action was than the critic’s baseline expectation for that state. Training uses that advantage to scale the policy-gradient update, so the actor is pushed toward actions that beat the baseline and away from actions that fall short of it. The critical structural fact for deployment: only the actor is needed to act. At inference, an agent observes a state and needs a decision. The critic’s job was to reduce variance in the gradient estimate during training. Once training is over, the value estimate produces no action — it is scaffolding, not payload. In our experience reviewing edge-agent packaging, the value head and any critic-specific layers are the first thing that should come off before a policy is compressed, and they are frequently the last thing anyone checks. This is why the naive “two networks plus a reward signal” mental model is dangerous at the deployment stage. It is accurate about training and silent about the divergence that matters: the trained system and the shipped system are not the same object. What does the advantage term add over plain policy gradients? A vanilla policy-gradient method (REINFORCE-style) scales updates by the raw return from a trajectory. That works, but the return is high-variance — a good action inside a mediocre trajectory looks bad, and a bad action inside a lucky trajectory looks good. Learning is slow and noisy as a result. The advantage term replaces “how good was the total return” with “how much better than expected was this action.” By subtracting the critic’s state-value baseline, A2C removes a large chunk of variance without introducing bias in the gradient estimate. That is the whole point of carrying a critic: faster, more stable convergence during training. It is a training-time benefit. Understanding this is what tells you the advantage machinery buys you nothing at inference and can be discarded before packaging — a point the two companion explainers on how advantage actor-critic works in practice and A2C in practice develop from the training side rather than the deployment side. Vanilla actor-critic and A2C share the same inference-time story: the actor ships, the critic does not. What A2C changes is entirely upstream of the edge boundary. Which components are training-only, and which ship? Here is the split that governs the deployment footprint. Treat it as a checklist before packaging. Component Role Ships to edge? Actor policy network Maps observed state to action at every step Yes — this is the payload Critic / value network Estimates state value to compute advantage No — training-only Value head (if shared trunk) Produces the scalar value estimate No — prune the head, keep the trunk Advantage computation Scales the policy gradient No — exists only in the training loop Optimizer state (Adam moments, etc.) Drives weight updates No — never part of an inference artifact Reward / environment plumbing Supplies the training signal No — not present at inference The shared-trunk case is the one that trips teams up. Many A2C implementations share early layers between actor and critic and split into two heads late. When that architecture is exported naively — for example, a full torch.save of the module — the value head and its final layers travel along with the actor. Under a framework like PyTorch, the clean move is to trace or script only the actor forward path (via torch.jit or torch.export), which drops the value head and any critic-only branches from the exported graph. What remains is the runtime-footprint axis that actually matters against a phone-class ceiling. Sustained, per-step actor latency under realistic input — not training reward — is the operationally relevant measure for an edge-deployed A2C policy. That is the number the on-device budget is written against. How much does dropping the critic actually save? The honest answer is: it depends on how the network was built, and you should measure it rather than assume it. Two structural patterns bound the range. Separate actor and critic networks (no shared trunk): the critic is a completely independent parameter set. Dropping it removes its entire parameter count and binary contribution. If the critic was sized similarly to the actor — a common default — you are looking at a roughly halved inference-time parameter count. This is a structural consequence of the architecture, not a benchmarked rate; the actual figure depends on how each network was sized. Shared trunk, split heads: the trunk is shared and stays, so only the value head and its dedicated layers come off. The saving is smaller — often modest relative to the trunk — but the cold-start and binary-size wins still land because you are no longer loading, initialising, or JIT-compiling a code path that never executes. The reason cold-start matters as much as steady-state latency: on constrained hardware, the first inference after process launch pays for weight loading, memory mapping, and graph warm-up. Every parameter and every branch you did not need to ship is time the agent is not yet responsive. In edge agent work the practical outcome shows up in three numbers — the actor policy’s cold-start time, its memory residency against the device ceiling, and its per-step latency against the inference budget — not in training reward at all. When does the dropped-critic actor still not fit? Removing training-only components is necessary but frequently not sufficient. Pruning the critic gets you an honest actor; it does not guarantee the actor fits. Use this rubric to decide whether you are done or whether the policy needs a second compression pass: Memory residency. Load the exported actor and measure resident set size against your device ceiling. Over budget after the critic is gone? You need compression, not more pruning. Per-step latency tail. Measure p99 per-step latency under realistic input on the target class of hardware, not on the training GPU. A sub-hundred-millisecond tail budget is a common ask for interactive agents; if the actor blows it, the model — not the packaging — is the problem. Cold-start time. Time first inference after a cold process launch. If warm-up dominates, the parameter count is still too high for the memory-bandwidth profile of the device. When any of these fail after critic removal, the next lever is compression of the actor itself — distillation to a smaller student, or post-training quantisation to shrink weights and speed up memory-bound matmuls. That is a different discipline from the training-vs-inference split, and we cover it directly in model optimization for edge inference and, for the quantisation path specifically, in dynamic quantisation for edge-constrained agent inference. The important sequencing point: drop the critic first. Compressing a bundle that still carries a value head wastes effort optimising code that will never run. Meeting cross-platform edge memory and latency ceilings across CPU, mobile GPU, and NPU targets is its own body of engineering — the GPU and cross-platform edge inference work picks up once the policy is stripped to its runtime essentials. And whether an A2C policy is even the right tool for the agent in question is upstream of all of this; the generative-AI feasibility work is where the agent-architecture decision gets made before any training loop runs. How does the compressed actor feed back into the agent framework decision? An edge-aware agent framework decision is fundamentally a runtime-footprint decision: what can this device hold and how fast can it respond. The value of understanding A2C’s internals is that it lets you feed a true footprint into that decision rather than an inflated one. A team that reports “our A2C policy is 40 MB and 60 ms per step” after correctly stripping the critic is making a different — and better — framework choice than a team quoting the packaged-everything number. Data-centre training cost and edge inference cost are separate budgets, and conflating them is the most common way agent-architecture decisions go wrong; the data-centric approach for edge-constrained agent inference treats the same divergence from the data side. FAQ How does advantage actor-critic work in practice? A2C trains two networks: an actor that maps states to actions and a critic that estimates state value. The advantage term measures how much better an action was than the critic’s baseline expectation, and that advantage scales the policy-gradient update. In practice, only the actor produces actions — the critic and advantage machinery exist to stabilise training and do not run at inference. What does the advantage term add over a plain policy-gradient or vanilla actor-critic method? The advantage term replaces high-variance raw returns with a lower-variance “how much better than expected” signal by subtracting the critic’s state-value baseline. This reduces gradient variance without adding bias, giving faster and more stable training. It is a training-time benefit only; vanilla actor-critic and A2C share the same inference-time story where just the actor ships. Which parts of an actor-critic system are training-only, and which actually ship to an edge deployment target? The actor policy network ships — it is the only component that produces actions. The critic network, value head, advantage computation, optimizer state, and reward plumbing are all training-only and should be removed before packaging. In a shared-trunk architecture, prune the value head but keep the shared trunk that the actor needs. How does the actor policy’s runtime footprint compare to the full training-time architecture on phone-class hardware? The trained architecture includes the critic, value head, advantage computation, and optimizer state; the shipped artifact is just the actor forward path. With separate actor and critic networks of similar size, dropping the critic can roughly halve inference-time parameter count — a structural consequence of the architecture, not a fixed rate. With a shared trunk, the saving is smaller but cold-start and binary-size still improve. When does an A2C-trained policy fit an on-device latency and memory budget, and when does it need further compression? It fits when the stripped actor’s memory residency, p99 per-step latency, and cold-start time all clear the device budget measured on target-class hardware. It needs further compression when any of those fail after critic removal — the model itself, not the packaging, is then over budget. Distillation or post-training quantisation of the actor is the next lever. How does dropping the critic before packaging affect binary size and cold-start time at the edge? Removing critic-only parameters and code paths shrinks the binary and cuts cold-start time, because the process no longer loads, memory-maps, initialises, or JIT-compiles branches that never execute at inference. The steady-state latency win depends on architecture, but the cold-start win lands whenever any critic-specific weights or layers are removed. How does the compressed actor policy feed back into the edge-aware agent framework decision in the parent hub? The framework decision is a runtime-footprint decision, and stripping the critic lets you feed it a true footprint instead of a packaged-everything number. A team reporting an honest actor-only memory and latency profile makes a better architecture choice than one quoting the full training-time bundle. Data-centre training cost and edge inference cost are separate budgets that should not be conflated. The uncertainty worth naming: the split between actor and critic is architectural, so the savings from dropping the critic are only knowable once you inspect how a specific policy was built and export the actor path in isolation. When an edge-agent policy misses its budget after packaging, the failure class is almost always training-only components shipped as inference payload — inspect the exported graph for a stray value head before you reach for the quantiser.
