A facial recognition demo runs fine on a laptop. That is the trap. The demo processes one face in one image, and almost any GPU embeds a single 112×112 crop in a few milliseconds. Production is a different machine entirely, because facial recognition is not one workload — it is four, and each of the four stresses a different part of the box.

Teams routinely size a facial recognition workstation from that first happy demo. They read a GPU headline spec, buy the biggest card the budget allows, and assume they are covered. Then the system meets a real gallery, real concurrent camera streams, and a false-match threshold that has to hold across tens of thousands of identities — and the machine that embedded one face at 30 FPS stalls on a 500,000-identity search or chokes on eight simultaneous streams. The failure is not that the hardware was weak. The failure is that it was sized against the wrong stage.

The correct way to spec the machine is to profile each pipeline stage separately, because the stages do not share a bottleneck. Detection and alignment are lightweight. Embedding is GPU-bound. Gallery matching is memory- and index-bound, and it gets worse as the gallery grows. Spec the workstation stage by stage, and you buy a machine that runs the pipeline — not a demo dressed up as a deployment.

What are the four stages of a facial recognition pipeline?

Before you can size hardware, you have to stop treating facial recognition as one opaque box. In practice it decomposes into four sequential stages, and the compute demand of each is different enough that they belong in separate columns of a spreadsheet.

Detection finds faces in a frame. A modern detector — RetinaFace, SCRFD, or a YOLO-family face model — runs on the GPU but is cheap relative to embedding, because it operates on a downscaled frame and outputs a handful of bounding boxes. On a mid-range GPU this stage typically clears well above real-time for a single stream (observed across our engagements; not a published benchmark).

Alignment takes each detected box and warps the face to a canonical pose using five landmark points. This is almost pure image arithmetic — an affine warp per face — and it runs comfortably on CPU or as a trivial GPU kernel. It is the least demanding stage and almost never the bottleneck.

Embedding is the expensive stage. A recognition backbone — ArcFace, an IResNet-100, or an equivalent — converts each aligned crop into a fixed-length vector (commonly 512 dimensions). This is where GPU FLOPs and memory bandwidth actually get spent, and it is the stage that scales with the number of faces per frame and the number of concurrent streams.

Gallery matching compares each new embedding against the enrolled gallery to find the nearest identity. For a small gallery this is a trivial dot product. For a large gallery it is a nearest-neighbour search over hundreds of thousands of vectors, and the constraint shifts from FLOPs to memory footprint and index structure. Our breakdown of what “accelerate” actually means for facial recognition covers why a headline TOPS number tells you almost nothing about how these four stages behave together.

Which stage actually drives the workstation’s requirements?

The honest answer is: it depends which end of the pipeline your workload lives at, and most teams get this backwards. They provision for embedding because that is the stage with the obvious GPU cost, and then they are ambushed by gallery matching because that is the stage nobody profiled.

Here is the stage-by-stage compute profile that should drive the spec.

Stage	Primary constraint	Scales with	Where it lives	Spec lever
Detection	GPU compute (light)	Frame count, resolution	GPU	Mid-range GPU sufficient
Alignment	CPU / trivial GPU	Faces per frame	CPU or GPU	Rarely the bottleneck
Embedding	GPU FLOPs + memory bandwidth	Faces × streams	GPU	GPU tier, VRAM, batch size
Gallery matching	Memory footprint + index	Gallery size	RAM / VRAM	System RAM, index type

Read the table as a decision tool, not a summary. If your deployment is a handful of streams against a small watchlist — say, a few thousand enrolled identities — embedding dominates and a single mid-range GPU with enough VRAM to batch the streams is the whole story. If your deployment is a large enrolled population — a national ID gallery, a retail loss-prevention database, a border-control watchlist — gallery matching dominates, and the GPU almost stops mattering while system memory and index choice become the entire question.

The reason this matters commercially is that the two failure modes are symmetric and both expensive. Over-provision a multi-GPU rig for a workload a single card handles, and you have spent the budget on FLOPs that sit idle while a 200k-vector search runs on an under-sized memory subsystem. Under-provision the memory and index, and false-match tuning plus gallery growth force a hardware rebuild mid-project — the exact cost the spec exercise exists to avoid.

How much GPU do you actually need for embedding throughput?

This is the question the demo answers dishonestly. A single 512-dimensional embedding on an IResNet-100 backbone costs on the order of a few billion FLOPs — trivial for any current GPU in isolation. The demo shows you a few milliseconds and you conclude the GPU is oversized. It is not; you simply have not asked it to do the real job yet.

The real job is sustained throughput across concurrent streams at a working batch size. Eight 1080p camera streams at 15 FPS, each averaging two faces per frame, is roughly 240 embeddings per second — and that number is the floor, not the ceiling, because detection also runs on the same GPU for every frame. Whether a given card sustains that depends on how well you batch: embedding one face at a time wastes most of the GPU, while batching faces across streams into a single forward pass is what actually fills the silicon.

A worked example, with assumptions stated:

• Assume 8 streams, 1080p, 15 FPS, ~2 faces/frame → ~240 embeddings/sec target.
• Assume an IResNet-100 ArcFace backbone at FP16 with dynamic batching.
• Then a single mid-range-to-upper GPU with 16–24 GB of VRAM typically sustains this with headroom, provided the software stack batches across streams rather than processing them serially (observed pattern across our deployments; not a benchmarked rate).
• But double the streams or the face density, and you are either adding a second GPU or dropping precision.

Two levers change this arithmetic without buying more hardware. Reduced precision — FP16 or lower — cuts memory bandwidth pressure and can raise throughput materially; our note on what FP4 means for CV model precision covers where the accuracy trade-off starts to bite for recognition specifically. And graph compilation matters: running the backbone through a compiler like TensorRT, rather than eager PyTorch, fuses kernels and removes framework overhead that otherwise dominates at small batch sizes. Our walkthrough of how ML compilers optimize face recognition inference explains why the same GPU can deliver very different throughput depending on the compile step.

How does gallery size change the memory and index requirements?

This is the stage that quietly breaks single-workstation deployments, and it breaks them on an axis the GPU spec never touches: memory footprint and index structure grow with the enrolled population, not with the stream count.

The arithmetic is unforgiving but simple. A 512-dimensional embedding stored as FP32 is 2 KB per identity. A 100,000-identity gallery is therefore about 200 MB of raw vectors — trivial. A 1,000,000-identity gallery is about 2 GB, still fine to hold in RAM. The problem is not storing the vectors; it is searching them fast enough. A brute-force nearest-neighbour scan over a million vectors, per query, per frame, across concurrent streams, is a very different cost from a single dot product.

That is where the index choice enters the spec. An approximate nearest-neighbour index — FAISS with an IVF or HNSW structure — trades a small, tunable accuracy cost for a search that stays bounded as the gallery grows. But the index itself has a memory cost that can exceed the raw vectors, and HNSW in particular is memory-hungry. So the gallery spec is really two numbers: enough RAM to hold vectors plus index, and enough of it fast enough that search latency stays under your per-frame budget at your operating false-match threshold.

The single-workstation ceiling is real and worth naming. A workstation holding a few million vectors in RAM with a well-tuned index handles most enterprise galleries comfortably. Past that — tens of millions of identities, or strict latency guarantees under heavy concurrent load — you are into sharded search across multiple machines, and the question stops being “what workstation” and becomes “what architecture.” Knowing where that line sits before you buy is the entire point; crossing it accidentally is the mid-project rebuild the ROI case is built to prevent. Working through those trade-offs before committing hardware is exactly the kind of scoping a computer vision consultant does when framing edge-deployment decisions.

How do specs differ across cloud, on-device, and edge?

The same four-stage decomposition holds everywhere, but where you run it moves the constraints around.

Cloud gives you the most freedom to over-buy your way past a bad spec, which is exactly why cloud deployments hide sizing errors until the invoice arrives. You can attach a large GPU and abundant RAM on demand, so embedding and gallery matching both look easy — until the running cost of an idle multi-GPU instance, or the egress cost of streaming raw video to the cloud, becomes the real constraint. The spec discipline shifts from “will it run” to “what am I paying for the FLOPs I am not using.”

On-device — a workstation or server physically at the site — is the setting this article is really about, and the one where the four-stage spec pays off most directly. You size the GPU for embedding throughput at your stream count and the RAM plus index for your gallery, once, and the machine either fits the workload or it does not. There is no elastic escape hatch, which is precisely why the profiling has to be honest.

Edge — a small accelerator at or near the camera — inverts the priorities. Detection and embedding may run on a constrained NPU or a low-power GPU, precision is aggressively reduced to fit the memory envelope, and gallery matching often moves off the device entirely to a central server because the edge box cannot hold a large gallery or search it fast. Reading generic accelerator benchmarks in this setting is a trap of its own; our note on what GPU benchmarks like SPECviewperf actually mean for CV deployment hardware and the companion piece on what SPEC benchmarks measure and miss for CV inference both matter here, because a benchmark score for a workstation GPU tells you almost nothing about how a quantized recognition backbone behaves on an edge NPU. The full computer vision practice at TechnoLynx treats this cloud-versus-edge split as a deployment decision, not a hardware afterthought.

FAQ

How should you think about a spec workstation for facial recognition in practice?

A spec workstation for facial recognition is a machine sized against the four distinct stages of the pipeline — detection, alignment, embedding, and gallery matching — rather than against a single GPU headline number. In practice it means profiling each stage separately, because detection and alignment are light, embedding is GPU-bound, and gallery matching is memory- and index-bound. The workstation is “spec’d” when it sustains real embedding throughput and bounded gallery-search latency, not when it runs a single-image demo.

Which pipeline stage actually drives the workstation’s GPU, memory, and storage requirements?

It depends on your workload, which is why the profiling matters. Embedding drives GPU FLOPs and VRAM and scales with faces per frame times concurrent streams. Gallery matching drives system memory and index footprint and scales with the enrolled population. Small-gallery, few-stream deployments are embedding-bound; large-gallery deployments are memory-and-index-bound, and there the GPU almost stops mattering.

How much GPU capacity does a workstation need to sustain real embedding throughput versus a single-image demo?

A demo embeds one face in a few milliseconds on almost any GPU, which tells you nothing. Real throughput is sustained embeddings per second across concurrent streams at a working batch size — for example, roughly 240 embeddings/sec for eight 1080p streams at 15 FPS with two faces per frame. A single mid-range-to-upper GPU with 16–24 GB VRAM typically handles that with headroom when the stack batches across streams and uses FP16 plus graph compilation (observed pattern; not a benchmarked rate).

How does gallery size affect memory and index requirements, and where does a single workstation stop scaling?

A 512-dim FP32 embedding is 2 KB per identity, so even a million-identity gallery is only ~2 GB of raw vectors — storage is not the problem, search speed is. An approximate nearest-neighbour index (FAISS IVF or HNSW) keeps search bounded but adds its own memory cost, so RAM must hold vectors plus index. A single workstation handles a few million identities comfortably; past tens of millions, or under strict latency at heavy concurrency, you move to sharded search across multiple machines.

How do workstation specs differ between cloud, on-device, and edge inference settings?

The four-stage decomposition is constant, but the binding constraint moves. Cloud lets you over-provision, so the discipline becomes cost control rather than fit. On-device forces an honest one-time spec with no elastic escape hatch. Edge inverts priorities — reduced precision, constrained accelerators, and gallery matching pushed off-device to a central server because the edge box cannot hold or search a large gallery.

What practical spec questions should a buyer ask so the workstation supports false-match tuning and gallery growth without a rebuild?

Ask for the per-stage compute profile: sustained embeddings/sec at the real stream count and batch size, the gallery size the RAM-plus-index budget supports, and the search latency at the operating false-match threshold. Then ask where the single-workstation ceiling sits and how far the current spec is from it. Those answers tell you whether false-match tuning and gallery growth will fit the existing machine or force a hardware rebuild mid-deployment.

The workstation spec is not a shopping list; it is the output of a pipeline audit. Detection, alignment, embedding, and gallery matching each carry a distinct hardware demand, and the machine that fits your deployment is the one sized against all four at your real throughput and gallery scale — not the one that ran the demo. The remaining uncertainty for any given buyer is where their own false-match threshold and expected gallery growth push the ceiling, and that is precisely the question a facial recognition pipeline audit is built to answer before the purchase order goes out.