A team swaps in BEVDet, sees the mAP number climb, and carries the old evidence structure forward unchanged. That is the moment the trouble starts. BEVDet is not a cleaner box detector you can drop into the slot the previous stack occupied — it is an architecture that reprojects camera features into a shared bird’s-eye-view grid, and that reprojection introduces failure surfaces the old image-plane detector never had. The distinction matters because a safety reviewer does not clear a network on its benchmark. They clear the evidence that each function producing a safety-relevant detection is bounded and traced. BEVDet changes which internal steps carry that burden, and a strong NDS score says nothing about whether the depth-lift or the cross-camera fusion can silently degrade when a camera’s calibration drifts a fraction of a degree. What does BEVDet actually do? BEVDet is a multi-camera 3D object detection architecture that produces detections in a top-down bird’s-eye-view (BEV) coordinate frame rather than in each camera’s image plane. Instead of predicting a 2D box per image and stitching them together afterwards, it fuses all camera views into one shared grid centred on the ego vehicle, then detects objects directly in that grid. The reason for working in BEV is structural, not cosmetic. Downstream planning and control need object positions in a metric, ego-centric frame — where a car is relative to your vehicle in metres, not where its pixels sit in one camera. Detecting in the image plane forces a fragile post-hoc lift from 2D to 3D, where a small pixel-level error becomes a large metric error at range. BEV detection moves that lift inside the network and lets the detection head reason about the whole scene at once, which is why it handles objects spanning camera boundaries more gracefully than per-camera detectors that must reconcile overlapping predictions. That relocation of the lift is the whole point — and it is also where the new evidence obligations live. The BEV grid is only as trustworthy as the geometry that populated it. The four stages, and what can go wrong at each BEVDet is best understood as a pipeline of four stages, each with its own failure mode: Stage What it does Dominant failure surface Image encoder Extracts per-camera 2D feature maps (typically a ResNet or Swin backbone) Domain shift, low light, motion blur — the same failures any 2D backbone has View transformation / depth lift Predicts per-pixel depth and “lifts” 2D features into 3D, then splats them onto the BEV grid Depth estimation error; features placed at the wrong metric distance BEV encoder Refines the aggregated BEV feature grid with convolutions Aggregation artefacts where sparse or conflicting features overlap Detection head Regresses 3D boxes, orientation, and class from the BEV grid The usual localisation and classification errors, now in metric space The stage that separates BEVDet from an image-plane detector is the view transformation. In the “lift-splat-shoot” style transform that BEVDet builds on, the network estimates a depth distribution for every pixel and uses it, together with the camera’s intrinsic and extrinsic parameters, to place each feature at a 3D location. Two things feed that placement: the learned depth, and the geometry from calibration. Both can be wrong independently, and both errors compound in the same grid cell. This is not a subtle academic point. If the depth head places a feature one metre too far, the object appears in the wrong BEV cell, and every downstream consumer inherits that offset. If the extrinsic calibration is off by half a degree, features from that camera land in systematically shifted positions across the whole grid. The detection head cannot recover geometry the transform destroyed — it only sees the grid it was handed. Why calibration drift matters more here than you expect An image-plane detector that loses calibration still produces boxes on the image; the error surfaces later, in the 2D-to-3D lift you own explicitly. BEVDet folds that lift into the network, so calibration error propagates silently — the model still emits confident 3D boxes, just in the wrong places. There is no obvious symptom on the detection output that screams “your extrinsics drifted.” The mechanism is worth naming precisely. The view transform uses the camera-to-ego transform (extrinsics) and the focal length and principal point (intrinsics) to project depth-lifted features. An error in either rotates or translates where features land. Because the transform is applied per camera and then aggregated, a drift on one camera degrades exactly the region of the BEV grid that camera dominates — often the boundary zones where cross-camera fusion is already hardest. We treat the geometry that feeds the transform as a first-class evidence surface for this reason; the role of camera extrinsic calibration in your safety evidence is not a preprocessing footnote when the perception network consumes those parameters directly. The same fragility shows up downstream. A tracker that assumes stable BEV positions will break in ways that trace straight back to calibration drift, because the tracked positions inherit the transform’s geometric error frame by frame. The failure does not originate in the tracker; it originates two stages upstream, in a step the benchmark never isolated. Why a strong mAP or NDS score is not enough Benchmarks like nuScenes report mAP and NDS aggregated over a dataset collected with well-calibrated rigs under a fixed distribution. Those numbers are real signal — for what they measure. But a safety reviewer for a high-integrity perception function is not asking “does this network detect well on average?” They are asking “where can this function silently produce a wrong detection, and what bounds the error when it does?” An aggregate benchmark answers the first question and stays silent on the second. It does not tell you how the depth head behaves at the range where a detection feeds an ASIL D emergency-braking decision. It does not tell you how fused detections spanning two cameras degrade under realistic extrinsic drift. It does not decompose the error into the transform stage versus the detection stage. A benchmark number is a single scalar over a monolith; ASIL evidence is graded per function. This is the same trap that catches teams who carry benchmark scores forward as if they were safety evidence — the failure modes that matter for safety are precisely the ones an aggregate metric averages away. On nuScenes-style splits (published benchmark, not your operational distribution), BEVDet variants report competitive NDS, and that is exactly why the number is seductive and insufficient. Mapping each stage to an evidence surface The productive move is to stop treating BEVDet as one thing to validate and start treating each stage as an evidence surface whose depth is set by the ASIL of the function it feeds. A detection that only informs a comfort feature does not need the same rigour as one that feeds automatic emergency braking, even if both pass through the same network. A stage-to-evidence rubric Use this as a scoping checklist, not a compliance template — the specific ASIL and functions are yours to assign: Depth lift. For any detection feeding a high-ASIL function, bound the depth error as a function of range and show the bound holds under the lighting and weather distribution the vehicle will see. An aggregate mAP does not carry this; a range-stratified depth-error analysis does. Calibration sensitivity. Show how detection error responds to bounded extrinsic and intrinsic perturbation. If a half-degree extrinsic drift moves a high-ASIL detection outside its allowed error box, that is a finding a reviewer will want to see addressed, not discovered in the field. Cross-camera fusion. For detections spanning camera boundaries, trace how conflicting per-camera features are aggregated and bound the error in exactly those boundary regions, which behave differently from single-camera zones. Detection head. Apply the localisation and classification evidence you would apply to any 3D detector — but in metric BEV space, where the error units are metres, not pixels. Monitoring. Because calibration drift is silent at the output, the strongest field defence is a runtime monitor that watches the geometry, not just the detections. The pattern is graded rigour: the depth and fusion steps that carry the highest-ASIL detections get the deepest evidence, and the stages feeding lower-integrity functions get evidence proportionate to their consequence. This is the same discipline our production AI monitoring harness is built to support — turning each pipeline stage into a traceable evidence surface rather than a black box behind a single score. It is also the general reliability posture: BEVDet’s transform and fusion stages are precisely the internal failure surfaces a reliability-evidence discipline expects to be bounded and traced. FAQ How does BEVDet work in practice? BEVDet is a multi-camera 3D detector that lifts per-camera image features into a shared bird’s-eye-view grid centred on the ego vehicle, then detects objects directly in that metric top-down frame. In practice it means detections arrive already in the ego-centric coordinates planning needs, rather than requiring a fragile post-hoc 2D-to-3D lift. The trade-off is that the lift now lives inside the network, so its correctness depends on both learned depth and camera calibration. What is the bird’s-eye-view (BEV) representation, and why does BEVDet lift image features into a shared BEV grid rather than detecting in the image plane? The BEV representation is a top-down, metric, ego-centric grid where positions are measured in metres from the vehicle. BEVDet lifts into that grid because downstream planning and control reason about where objects are in metric space, and image-plane detection forces a brittle conversion where small pixel errors become large metric errors at range. Working in BEV also lets the detection head reason about the whole scene at once, which handles objects spanning camera boundaries better than reconciling separate per-camera predictions. What are BEVDet’s main stages — image encoder, view transformation/depth lift, BEV encoder, detection head — and what can go wrong at each? The image encoder extracts 2D features and inherits the usual backbone failures like low light and motion blur. The view transformation predicts per-pixel depth and splats features onto the grid, where depth error or calibration error places features at the wrong location. The BEV encoder can produce aggregation artefacts where features conflict, and the detection head carries the usual localisation and classification errors — now in metric space where the transform’s geometry cannot be recovered. How does camera calibration drift or depth-estimation error propagate through the BEV transform, and why does that matter for high-ASIL detections? The transform uses extrinsics and intrinsics to place depth-lifted features, so a calibration drift systematically shifts every feature from that camera, and a depth error places an object in the wrong grid cell. Both propagate silently: the network still emits confident 3D boxes, just in the wrong places. For a high-ASIL detection feeding something like emergency braking, a silent geometric offset that a benchmark never exposed is exactly the risk a safety reviewer needs bounded. How do we map each BEVDet stage to an evidence surface whose depth matches the ASIL of the function it feeds? Treat each stage as its own evidence surface and set its rigour by the ASIL of the detections that pass through it. The depth lift and cross-camera fusion feeding high-ASIL functions get the deepest analysis — range-stratified depth-error bounds and calibration-sensitivity results — while stages serving lower-integrity functions get proportionate evidence. The goal is graded rigour rather than uniform benchmark evidence spread flat across the whole network. Why isn’t a strong mAP or NDS benchmark on BEVDet enough to satisfy a safety reviewer for a high-integrity perception function? Aggregate benchmarks report average performance on a fixed, well-calibrated distribution and stay silent on where a function can silently fail. A reviewer for a high-integrity function needs to know how the depth head behaves at the relevant range and how fused detections degrade under realistic calibration drift — questions a single scalar averages away. The benchmark is genuine signal for average quality but does not decompose error by stage or bound it per function. How does cross-camera fusion in BEVDet affect how we bound and trace errors for detections that span camera boundaries? Cross-camera fusion aggregates features from multiple cameras into the same BEV region, so boundary zones behave differently from single-camera zones and are where calibration drift on one camera does the most damage. Bounding errors there means tracing how conflicting per-camera features are combined and characterising the error specifically in those boundary regions rather than assuming grid-wide uniformity. A detection that spans two cameras inherits both cameras’ geometric errors, so its evidence must account for that combined uncertainty. BEVDet earns its place in a perception stack precisely because it moves the 2D-to-3D lift where the network can reason about it — but that same move is what turns depth estimation and calibration into safety evidence surfaces rather than preprocessing details. The open question for any team adopting it is not “does the benchmark clear?” but “which stage produces each high-ASIL detection, and can we bound its error where the benchmark stayed silent?”
