The team wired up the full Segment Anything Model at an inspection station, pointed it at a conveyor moving parts at line rate, and watched the queue back up. The model produced beautiful masks — one every second or so, on a station that needed a verdict every 150 milliseconds. The engineering instinct in the room was to blame the hardware and ask for a bigger GPU. That was the wrong diagnosis. The problem was not that the station was too slow; it was that the wrong segmenter had been chosen for a throughput-bound job.

Mobile SAM is the answer to that specific problem. It is a distilled, lightweight variant of the Segment Anything Model that replaces SAM’s heavy image encoder — a large ViT-H vision transformer — with a compact one, while keeping the same promptable mask decoder. The point is not that Mobile SAM segments better. It segments fast enough to run on every unit, and that is what turns segmentation from a demo into a production measurement.

How does Mobile SAM work in practice?

SAM has two parts that matter here: a big image encoder that turns an image into a dense feature embedding, and a small prompt-driven decoder that turns a point, box, or mask prompt into a segmentation mask. Almost all of SAM’s compute lives in the encoder. Mobile SAM’s core move is encoder distillation — training a small encoder to reproduce the embeddings of the original heavy encoder, so the decoder that was trained against those embeddings still works unchanged.

In practice that means you keep SAM’s promptable behaviour — click a point on a part, or feed a bounding box from an upstream detector, and get a mask back — but the per-image cost drops sharply because the expensive encoder pass is now a lightweight one. Per the published Mobile SAM work, the distilled encoder is roughly two orders of magnitude smaller than SAM’s ViT-H encoder in parameter count (benchmark, per the authors’ reported model sizes). That is the difference between a model you run once for a screenshot and a model you run thousands of times a shift.

For an inspection cell, the sequence is usually: a fast detector (often a YOLO variant) proposes a box around each part or region of interest, and Mobile SAM turns that box prompt into a pixel-accurate mask. If you have not looked at how detection and segmentation split the work, our explainer on instance segmentation models for manufacturing inspection walks through where the mask actually earns its keep on the line.

How does Mobile SAM differ from the full Segment Anything Model, and what is traded away for speed?

The honest framing: Mobile SAM trades a small amount of mask fidelity on hard, ambiguous boundaries for a large gain in speed and a much smaller memory footprint. On clean, well-lit parts with clear edges — which is most industrial inspection imagery — the masks are close enough that the difference rarely shows up in the downstream metric. On cluttered scenes with fine, overlapping structure, the full model’s richer encoder still has an edge.

What you do not trade away is the promptable interface. Both models take the same prompt types and produce the same mask format, so an inspection pipeline built against SAM’s decoder can swap in the Mobile SAM encoder without redesigning the prompt logic.

Full SAM vs Mobile SAM for a line-side inspection station

Dimension	Full SAM (ViT-H encoder)	Mobile SAM (distilled encoder)
Encoder size	Large (hundreds of millions of params)	~1% of the encoder param count (per authors)
Per-image latency	Too slow for per-unit line-rate scoring	Feasible at line rate on modest edge hardware
Edge memory footprint	Heavy; often needs a datacenter-class GPU	Fits comfortably on a compact accelerator
Prompt interface	Point / box / mask prompts	Identical — decoder is unchanged
Mask fidelity on clean parts	Reference	Close; differences rarely reach the SPC metric
Mask fidelity on cluttered/fine detail	Better	Slightly degraded
Right when…	Offline labelling, ambiguous scenes	On-line, throughput-bound scoring

The table is the whole decision in miniature: if the job is offline mask labelling or a visually ambiguous scene, the full model is defensible. If the job is scoring every unit as it passes a station, the encoder size is the binding constraint and Mobile SAM wins.

Why is a lightweight promptable segmenter useful for defect inspection on a production line?

Here is the divergence point that most teams miss. A station that can only run a detector emits an attribute signal — pass or fail, defect present or absent, a box with a confidence score. A station that produces a clean per-unit segmentation mask emits a variable signal: defect area in square millimetres, affected-region count, the fraction of a part’s surface flagged. That distinction is not cosmetic. It is the difference between attribute and variable data in statistical process control, and it changes what the control chart can see.

Attribute pass/fail charts (p-charts, np-charts) are coarse. They tell you the reject rate moved, but only after enough units have failed to be statistically visible. A variable measurement — defect area per unit — feeds an X-bar/R or individuals chart that can detect a process shift while parts are still nominally passing, because the distribution of the measurement drifts before the fail threshold is crossed. Continuous, quantitative signals tighten control limits and shorten time-to-detect a process shift (observed-pattern; this is the general behaviour of variable-vs-attribute SPC, and holds across the inspection cells we have worked on rather than being a single benchmarked number).

Mobile SAM is what makes that variable signal affordable to collect on every unit. Segmentation gives you the mask; the mask gives you area; area is a variable you can chart. Without line-rate segmentation you are back to bounding boxes and coarse pass/fail. If you want the detection side of this picture, our piece on fine-tuning YOLO for manufacturing-line defect detection covers what the detector fixes and what it leaves to the segmenter.

How do you turn Mobile SAM masks into a variable SPC input like defect area or region count?

The mask is not the metric. You have to reduce a per-unit mask into one or two scalars that a control chart can consume, and you have to do it the same way every time.

A worked example, with the assumptions stated explicitly:

• Assumption: the camera has a fixed working distance and a known calibration, so pixel area maps to physical area (e.g. one pixel ≈ 0.04 mm²).
• Step 1: YOLO proposes a box around each part; Mobile SAM returns the defect mask within it.
• Step 2: count the mask’s foreground pixels; multiply by the per-pixel area to get defect area in mm².
• Step 3: run connected-components on the mask to get region count (how many distinct defect blobs).
• Step 4: log both scalars per unit, timestamped and part-ID linked, into the SPC store.
• Step 5: the control chart plots defect area as an individuals/X-bar series; region count as a secondary series.

Two scalars — area and count — are usually enough to drive a variable chart. Linking each measurement to a part identity matters if you want to trace a shift back to a specific fixture or upstream operation; our note on Re-ID in CV inspection for SPC covers how the part identity is carried across the line.

What latency and edge-hardware footprint should you expect at line rate?

The specific numbers depend on your image resolution, prompt strategy, and accelerator, so treat any single figure as illustrative rather than a spec. The structural expectation is what matters: Mobile SAM’s compact encoder cuts both per-image latency and peak memory versus full SAM by a wide margin, which is precisely why it fits on a compact edge accelerator instead of demanding a datacenter-class GPU per station (benchmark, per the authors’ reported speedups over full SAM).

Two levers move the numbers on real hardware. First, prompt count — one box prompt per part is cheap; dense point-grid “segment everything” prompting is not, and you rarely need it for inspection. Second, runtime — compiling the model with TensorRT or ONNX Runtime, using FP16 where accuracy allows, and fusing the pre/post-processing off the critical path all reduce end-to-end latency. In configurations we have tuned, the encoder pass stops being the bottleneck once it is compiled and quantised, and the box-prompt decoder is already fast.

Do the arithmetic against your takt time before committing: if a station has 150 ms per unit, your entire detect-then-segment-then-reduce pipeline has to fit inside that, with headroom for jitter. Segmentation that averages fast but occasionally spikes will still drop units.

How do you validate Mobile SAM’s mask stability before trusting its output in SPC control charts?

This is the step teams skip, and it is the one that decides whether the SPC chart is measuring the process or measuring the model’s noise. A defect-area metric is only a valid SPC input if the measurement is repeatable: the same part imaged twice must yield nearly the same area. If the mask edge jitters run-to-run, that jitter shows up as variance in the control chart and inflates your control limits — you lose exactly the sensitivity you switched to segmentation to gain.

The validation is a gauge R&R in spirit: image a fixed set of known parts repeatedly, measure the run-to-run spread of the derived area, and confirm the measurement variance is small relative to the process variation you want to detect. Only then does the metric earn a place on the chart. This is precisely why the segmentation output has to be characterised — for latency, for mask stability, and for drift over time — before it feeds SPC limits, and why a production monitoring harness charts the segmenter’s own signal for drift alongside the process it is watching.

A lightweight segmenter chosen for the line still has to meet the reliability artefacts a hardened industrial CV deployment signs against; getting the model fast is necessary but not sufficient. If you are scoping that hardening, our computer vision practice and the way we structure [engagements around your specific problem](our service packs) both start from what the station has to guarantee, not just what the model can do.

When is Mobile SAM the wrong choice versus a purpose-trained defect segmentation model?

Mobile SAM is a promptable, general segmenter. It excels when you can give it a good prompt and the defect boundary is visually obvious once localised. It is the wrong choice when the defect is subtle, class-specific, and hard to prompt — a faint texture change, a sub-surface flaw, a defect that a general model has no concept of. In those cases a purpose-trained defect segmentation model, trained on your labelled defect data, will beat a prompted general model on both accuracy and consistency, and it does not depend on an upstream detector producing a clean box.

The practical read: use Mobile SAM to bootstrap — it gets a segmentation-based station running fast, and its masks can even help label a training set. If the derived metric proves valuable and the defect class is stable, graduating to a purpose-trained segmentation model is often the right next investment. The two are not rivals; they are stages.

FAQ

What’s worth understanding about mobile sam first?

Mobile SAM keeps the Segment Anything Model’s promptable mask decoder but replaces its heavy image encoder with a small one trained by distillation to reproduce the original encoder’s embeddings. In practice you still click a point or feed a box and get a mask back, but the per-image cost drops enough to run segmentation on every unit at line rate instead of only on offline images.

How does Mobile SAM differ from the full Segment Anything Model, and what is traded away for speed?

Almost all of SAM’s compute is in the encoder, and Mobile SAM shrinks that encoder to roughly a hundredth of the parameter count while leaving the prompt interface and decoder unchanged. The trade is a small loss of mask fidelity on cluttered or fine-detailed scenes; on clean industrial parts the difference rarely reaches the downstream metric.

Why is a lightweight promptable segmenter useful for CV defect inspection on a production line?

A detector alone gives you attribute data — pass/fail — while a per-unit mask gives you a variable measurement like defect area or region count. Variable data feeds a tighter SPC chart that can detect a process shift before parts start failing, and a lightweight segmenter is what makes collecting that mask affordable on every unit.

How do you turn Mobile SAM segmentation masks into a variable SPC input like defect area or region count?

With a calibrated camera you map mask foreground pixels to physical area to get defect area in mm², and run connected-components on the mask to get region count. Log both scalars per unit, timestamped and part-ID linked, and plot defect area on an individuals/X-bar chart with region count as a secondary series.

What latency and edge-hardware footprint should you expect from Mobile SAM at line rate?

Exact numbers depend on resolution, prompt strategy, and accelerator, so treat any single figure as illustrative. Structurally, Mobile SAM’s compact encoder cuts both latency and peak memory versus full SAM enough to fit a compact edge accelerator; compiling with TensorRT or ONNX Runtime, using FP16, and keeping prompts to one box per part are the main levers.

How do you validate Mobile SAM’s mask stability before trusting its output in SPC control charts?

Run a gauge-R&R-style repeatability check: image a fixed set of known parts many times and measure the run-to-run spread of the derived area. Only trust the metric on a control chart if that measurement variance is small relative to the process variation you want to detect, and monitor the segmenter’s signal for drift over time.

When is Mobile SAM the wrong choice versus a purpose-trained defect segmentation model?

Mobile SAM is a general, promptable segmenter, so it struggles with subtle, class-specific defects that are hard to prompt or that a general model has no concept of. When the defect class is stable and the metric proves valuable, a purpose-trained defect segmentation model trained on your labelled data will beat a prompted general model on accuracy and consistency.

The starting question is never “which segmenter is most accurate” — it is whether the station can produce a repeatable, line-rate measurement that a control chart can trust. Mobile SAM earns its place by making segmentation fast enough to be that measurement; whether the resulting signal is stable enough to set SPC limits is a separate question you have to answer with a repeatability study, not a benchmark leaderboard.