A bounding box around a crack tells you almost nothing that matters. It says a defect exists somewhere inside a rectangle, but a crack’s severity lives in its geometry — length, width, branching, and path — and none of that survives a box. On a demo photo of a wide, high-contrast crack against a clean surface, a detector draws a tidy rectangle and the demo looks solved. Take that same model to a real casting, a weld seam, or a concrete pour under mixed lighting, and the box either swallows half the part or misses the hairline fracture entirely. Crack segmentation is a pixel-level task. The output is a per-pixel mask that labels each pixel as crack or not-crack, and that mask is what carries the information an inspection decision actually needs. The moment you frame crack detection as “put a box around the defect,” you have thrown away the geometry before you started. This is the single most common reason crack-inspection prototypes look good in a slide deck and fall apart on the line. How should you think about crack segmentation in practice? A segmentation model assigns a class to every pixel in the image. For crack inspection that usually means a binary mask — crack versus background — though multi-class variants separate cracks from other defect types like porosity or scratches. Architecturally, most production crack segmenters are encoder–decoder networks in the U-Net family, or transformer-based segmentation heads; the encoder compresses the image into features, the decoder reconstructs a full-resolution mask. Frameworks like PyTorch with segmentation-models-pytorch, or a fine-tuned Segment Anything Model, are common starting points, and we cover how promptable masks from SAM can feed downstream geometry in a related piece. In practice, the mask is not the end product — it is the input to a measurement. Once you have a per-pixel crack mask, you skeletonize it to estimate crack length along its path, measure width perpendicular to the skeleton, and count branch points. Those measurements are what an inspection rule tests against: reject if maximum crack width exceeds a threshold, or if total crack length crosses a limit. A box cannot support any of that. This is why the task is segmentation and not detection: the decision depends on geometry that only a mask preserves. How is crack segmentation different from crack detection or bounding-box localization? The distinction is about what the output preserves, not about which is more advanced. A detector answers “is there a crack, and roughly where.” A segmenter answers “which exact pixels are crack.” For a defect that is compact and blob-like — a dent, a missing component — a box is often enough, and the choice between axis-aligned and oriented box geometry is where that decision gets made. Cracks are the opposite of compact. A single crack can span most of a frame while covering a fraction of a percent of its pixels, branch into a tree, and thin to two or three pixels wide. Here is the concrete failure. A crack running diagonally across a part produces a bounding box that covers the whole diagonal region, most of which is intact metal. The box’s area over-states the defect by orders of magnitude, and its dimensions tell you nothing about the crack’s actual width — the number that usually drives the accept/reject call. Report crack width from a box and you are reporting the box’s width, which is a property of the crack’s extent, not its severity. That is not a tuning problem you can fix with more data. It is the wrong output representation. Segmentation costs more: pixel-level labels are expensive to produce, masks are heavier to store and transmit, and the models are harder to make real-time. The honest trade-off is that you pay that cost precisely when the geometry drives the decision. For crack inspection it almost always does. Why are thin, branching, low-contrast cracks hard to segment? Three properties make cracks a worst case for segmentation, and each has a distinct cause. The first is class imbalance at the pixel level. If a crack occupies 0.3% of an image’s pixels, a model that predicts “background everywhere” scores 99.7% pixel accuracy while being useless. Standard cross-entropy loss will happily converge to that degenerate solution. In our experience across industrial-inspection work (an observed pattern, not a benchmarked rate), this is the number-one reason a first-pass crack model appears to “not learn” — it learned the majority class perfectly. The fix is a loss that weights the crack class heavily, or a region-overlap loss like Dice or Tversky that ignores the background’s dominance. The second is resolution and receptive field. A hairline crack two pixels wide at full resolution can disappear entirely when the image is downsampled to fit a model’s input size. Downsample a 4000×3000 sensor frame to 512×512 and a two-pixel crack becomes sub-pixel — it is gone before the network sees it. This is why tiling matters. Instead of resizing the whole frame, you cut it into overlapping patches at native resolution, segment each, and stitch the masks back. Our discussion of how image patching works and when it matters walks through the mechanics; for crack work the overlap is not optional, because a crack crossing a tile boundary must appear whole in at least one patch or the stitched mask will show a false gap. The third is contrast and confounders. Low-contrast cracks on textured or reflective surfaces sit close to the noise floor, and the model must separate them from scratches, casting marks, grinding lines, joints, and shadows that share the crack’s thin, elongated shape. A shadow cast by a fixture and a hairline crack can be nearly identical at the pixel level. No amount of loss reweighting fixes a confounder the training data never showed the model, which is why the label distribution — not just the label quality — decides how well confounders are handled. What metrics should evaluate a crack segmentation model? Pixel accuracy is the trap. Because of the imbalance above, it stays near-perfect while the model misses every crack. You need metrics that measure the crack region specifically, and you need more than one because they fail differently. The broader landscape of precision, recall, mAP, and IoU for inspection is worth reading alongside this — crack segmentation uses a subset with a specific emphasis on boundaries and per-defect recall. Crack segmentation evaluation surface Metric What it measures What it catches Evidence class IoU (Jaccard) on crack class Overlap between predicted and true crack pixels Overall mask quality; penalizes both misses and over-segmentation benchmark — computed per test set Boundary F1 (BF score) Agreement of mask edges within a pixel tolerance Thin-structure accuracy; a shifted-by-2px crack tanks BF while IoU barely moves benchmark — computed per test set Per-defect recall @ fixed FP budget Fraction of true cracks with any correct overlap, at a capped false-positive rate Missed hairline cracks (escape rate) at an operationally chosen FP ceiling benchmark — computed per test set Pixel accuracy Fraction of correctly classified pixels Almost nothing useful for cracks — reported only to expose the imbalance trap — The reason to track three metrics rather than one is that they localize different failures. IoU that drops while boundary F1 holds suggests the model is missing whole cracks (a recall problem). Boundary F1 that drops while IoU holds suggests the mask is roughly right but the edges are ragged (a resolution or label-precision problem). Per-defect recall at a fixed false-positive budget is the metric that maps most directly to the inspection outcome, because it answers the operational question: at the false-reject rate the line can tolerate, how many real cracks do we still let through? Track that frame-to-frame and you can target the specific failure — missed hairlines versus over-triggering on texture — rather than blindly retraining the whole model. What are the main failure modes, and what causes each? Four failures show up repeatedly, and each points at a different cause. Treating them as one lump — “the model is bad” — is exactly the diagnostic dead-end that a stage-level view is meant to avoid. Missed hairline cracks. Usually resolution or class imbalance. The crack was downsampled away, or the loss let the model ignore the minority class. Check whether the crack survives at the model’s input resolution before blaming the model. Over-segmenting texture. The model paints crack pixels across grinding marks or casting texture. Usually a label-distribution gap — the training set lacked enough clean-but-textured negatives — or a threshold set too low. Confusion with joints, seams, and shadows. A confounder problem. The model never learned to distinguish a real crack from a linear feature that looks like one, because the labels never made that distinction explicit. Broken masks at tile boundaries. A tiling artifact, not a model fault. Insufficient patch overlap or a stitching rule that drops edge predictions leaves gaps in what should be a continuous crack. Notice that two of these four are not model-quality problems at all — they are input-resolution and pipeline-assembly problems. If your segmentation stage is one opaque call, all four look identical from the outside: “the mask is wrong.” You retrain, the metric moves a little, and you have learned nothing about the cause. How do I make crack segmentation an observable, replaceable pipeline stage? The reframe that pays off is to stop treating the segmentation model as a black box that emits a verdict, and start treating it as an independently observable stage that emits a mask plus its quality telemetry. Every crack mask the stage produces carries an IoU, a boundary F1, and a per-defect recall figure against whatever ground truth or shadow-labeling you maintain. Those numbers are logged frame-to-frame, the same way you would instrument any other production model. This is the principle we apply across inspection pipelines: each stage is separately measurable and separately replaceable, so a regression can be attributed to a stage rather than to “the system.” When mask quality degrades, that degradation is telemetry the reliability discipline monitors as model drift. A crack segmenter whose boundary F1 slides over a month feeds the same drift-detection harness as any other model regression — there is nothing special about it being a segmentation stage. It is the same operational contract: measure the stage, alert on the measurement, replace the stage without disturbing its neighbours. Our broader computer vision engineering practice is built around exactly this stage-level observability. How do I tell whether an error comes from the labels, the model, or the resolution? This is the question the metrics exist to answer, and it has a repeatable procedure. Resolution first. Take the failing crack and check whether it is even visible at the model’s input resolution. Crop the native-resolution patch, downsample it the way the pipeline does, and look. If the crack disappears, the model was never given the information — no retraining fixes that. Fix the tiling or the input size. Labels second. Pull the training examples nearest to the failing case. If cracks like this one are absent, mislabeled, or the confounder (shadow, seam) was never labeled as background explicitly, the ceiling is set by the data, not the weights. Boundary F1 that is low across the whole test set — not just hard cases — often points here, because ragged human labels teach ragged masks. Model last. Only once resolution and labels are cleared do you attribute the error to the model itself: capacity, loss weighting, or training schedule. A model-side failure typically shows as a metric that improves smoothly with more or better-balanced data, whereas a resolution or label failure plateaus no matter how long you train. Running these in order matters. Teams that jump to step three first — retraining because the mask looks wrong — spend weeks chasing a resolution bug that a two-minute crop would have exposed. The order is cheapest-and-most-common-cause first. FAQ How does crack segmentation actually work? A segmentation model classifies every pixel in an image as crack or not-crack, producing a per-pixel mask. In practice that mask is not the deliverable — it is the input to a geometric measurement. You skeletonize the mask to estimate crack length, measure width perpendicular to the path, and count branches, then test those numbers against an inspection rule. How is crack segmentation different from crack detection or bounding-box defect localization? Detection answers “is there a crack and roughly where”; segmentation answers “which exact pixels are crack.” A crack running diagonally produces a bounding box covering mostly intact material, and the box’s dimensions describe the crack’s extent, not its width or path — the properties that drive the accept/reject decision. Cracks are thin, branching, and non-compact, so a box discards the geometry the decision needs. Why are thin, branching, low-contrast cracks hard to segment, and how do resolution and tiling affect results? Three causes compound: pixel-level class imbalance lets a model score high accuracy by predicting background everywhere; downsampling erases hairline cracks before the network sees them; and low contrast makes cracks hard to separate from scratches, seams, and shadows. Tiling at native resolution with overlapping patches keeps thin cracks visible and prevents false gaps where a crack crosses a patch boundary. What metrics should I use to evaluate a crack segmentation model? Use IoU on the crack class for overall mask quality, boundary F1 for thin-structure edge accuracy, and per-defect recall at a fixed false-positive budget for the operational escape-rate question. Pixel accuracy is misleading because class imbalance keeps it near-perfect while the model misses every crack. The three metrics localize different failures — misses versus ragged edges versus over-triggering. What are the main failure modes and what causes each? Missed hairline cracks come from resolution loss or class imbalance; over-segmenting texture comes from a training set short on clean-but-textured negatives or a low threshold; confusion with joints and shadows is a confounder the labels never distinguished; broken masks at tile edges are a stitching artifact from insufficient overlap. Two of these four are input or pipeline problems, not model-quality problems. How do I make crack segmentation an observable, replaceable pipeline stage rather than an opaque model call? Have the stage emit each mask together with its quality telemetry — IoU, boundary F1, and per-defect recall — logged frame-to-frame. A degrading segmenter then feeds the same drift-detection harness as any other model regression, so a problem is attributed to the stage and the stage can be replaced without disturbing its neighbours. How do I tell whether a segmentation error comes from the labels, the model, or the input resolution? Check in cheapest-cause order. First confirm the crack is visible at the model’s input resolution — crop and downsample the way the pipeline does. Then inspect the nearest training examples for missing, mislabeled, or unlabeled confounders. Only after clearing resolution and labels do you attribute the error to the model itself; a model-side failure improves smoothly with better-balanced data, while resolution and label failures plateau. Where this leaves the inspection stage The practical payoff is narrow and concrete: when a crack escapes to the field, or the line stops on a false reject, you want to name the cause the same day, not after a week of retraining. A segmentation stage that reports IoU, boundary F1, and per-defect recall against a fixed false-positive budget lets you do that, because the metric that moved tells you where to look. That is the difference between a crack model you operate and a crack model you merely hope about — and it is what the A2 pipeline assessment checks for at this stage: is the segmentation observable and replaceable, or is it one opaque call standing between you and the reason your masks went wrong?
