Active learning means letting a model choose which unlabelled examples are worth a human label, instead of labelling whatever happens to get collected. On an agricultural robot or harvester it takes a specific shape. We recommend capturing a frame when a grower’s or operator’s override disagrees with what the onboard model was confident about. We recommend checking candidate frames on the device with a calibrated uncertainty score paired with a novelty score. Then you hold what survives in a buffer sized for the worst connectivity gap, not the average one. Run the capture loop before the retraining loop exists: a ranked set of real field failures is useful long before any model gets retrained on it.

What does active learning look like for a harvester’s perception model?

In the textbook version, an acquisition function ranks a pool of unlabelled data by how much a label is expected to help, and the top-ranked items go to a human. The most ubiquitous measure in the literature picks the predictions with the highest entropy. On a robot, the “pool” is a live camera stream, and transmitting and storing every frame is infeasible in practice. Selection therefore has to happen on the device under a frame budget, which is the setting edge-camera work such as Stream-Based Active Distillation is built around. The device’s limited compute affects more than model optimisation. It also constrains the calculation of predictive uncertainty, which is vital for informing model updates.

Where growers override picks, a harvester also gives you something a lab pool never has: a human correcting the machine as it acts. That correction is a ground-truth labelling signal, and most of the design below follows from treating it that way. This selective data path is one of the engineering problems behind AI engineering for agriculture.

Active learning, passive supervised learning or reinforcement learning: which improves a field perception model?

Passive supervised learning trains on a labelled set chosen without asking the model what it finds hard. Reinforcement learning learns a policy from reward gathered through interaction. It changes how the robot acts, not how its perception model labels what it sees.

Operator overrides resemble interactive imitation learning such as HG-DAgger. In HG-DAgger, a human expert takes control when they judge the robot has entered an unsafe state, and labels are collected only during those recoveries. For a perception model, though, the override’s job is to say which frame to label, not to act as a reward. That makes active learning the right frame for improving the detector. If reinforcement learning belongs anywhere on a harvester, it belongs in the picking policy.

How do we trigger a capture the moment a grower overrides a pick?

Make the override itself the trigger. The robot should log the model’s confidence for every action it takes. When an override arrives, the system compares the two and fires a capture if they disagree: a confident pick the human rejected, or a hesitant one the human confirmed. Nobody has to remember to report anything. HG-DAgger offers a related precedent: it records the robot’s own doubt at each point where a human intervened, in a dedicated log.

TechnoLynx builds this trigger as part of the Production AI Monitoring Harness. The first step is framing that divergence metric on your existing override logs.

Tying the override to the exact frame the robot acted on

To tie an override to the exact frame, give every frame an ID and a capture timestamp, and have every action record name the frame ID it acted on. LiveKit’s robotics documentation describes the same pattern for video: each frame gets a capture timestamp from your own clock and a frame ID, so that consumers can match a frame to inference results produced elsewhere without a separate synchronisation channel. On Jetson hardware, the Argus camera API exposes start-of-frame and end-of-frame sensor timestamps in nanoseconds, taken from the Tegra-wide system counter.

The override arrives after the frame, by however long the human took to react. So match it through the frame ID in the action record, never by searching for the frame nearest to the override’s timestamp. Where the override comes from a separate device (a handheld, a cab button, a tablet), that device’s clock has to be disciplined to the robot’s with NTP or PTP. Otherwise drift quietly misaligns events by whole picks. Log who overrode as well as when, so inconsistent signals can be separated during review.

Setting the firing threshold so the fleet doesn’t flood storage or the labelling team

Learn the threshold from past overrides rather than picking it up front. HG-DAgger takes this approach: it learns its doubt threshold from recorded human interventions instead of choosing a “safe” value a priori. A 2019 corner-case detection paper for autonomous driving does the same with validation data, tuning the threshold to set the balance between false acceptances and false rejections. In practice, we would replay historical override logs, sweep the threshold, and plot captures per robot per day against the share of known-bad overrides caught. The operating point is wherever the capture volume fits your buffer and your labellers’ week.

The authors of Confidence-Triggered Detection settled on 30% for their tracker only after exploring several values, and they report speed and accuracy trading off as the threshold moved. That figure belongs to their tracker, not to your harvester. Set too low, the trigger buries reviewers in near-duplicates. Set too high, it misses the edge cases you deployed it to find.

The confidence-only path, where no human is involved, needs one extra guard. MLflow’s guidance for retraining triggers applies here: require deviation sustained across several monitoring windows, not a single spike.

How does the loop work when robots only sync at the depot?

Run it as store-and-forward. The robot selects and scores frames during the shift, holds them in a prioritised, persistent buffer, and uploads when the link comes back. Route frames to human review only after upload, once a larger model has re-scored them. Edge buffering of this kind exists precisely to ride out network interruptions and bandwidth limits.

TechnoLynx’s Computer Vision R&D engagement sizes and monitors the onboard buffer and tunes the on-device pre-filter on Jetson-class hardware.

How much onboard buffer each robot needs

Buffer size follows from expected data volume, network reliability and device storage. For a capture pipeline, those three factors reduce to one rule: persistent storage must hold the flagged-clip rate multiplied by the clip size, multiplied by the longest gap between upload windows, with headroom on top. The steps below make that concrete without assuming any particular numbers.

1. Measure the flagged-clip rate per robot by running the trigger in shadow mode over logged shifts. Do not estimate it.
2. Fix the clip size: a pre-event window plus a post-event window, multiplied by the encoded bitrate. DeepStream’s smart record saves from startTime seconds before the event to duration seconds after it.
3. Take the longest gap between connectivity windows at any site, not the fleet average. A buffer sized for the average loses data at exactly the sites with the worst coverage.
4. Size the in-memory pre-event cache separately. rosbag2’s snapshot cache uses double buffering, so memory can reach twice the configured maximum in pessimistic cases.
5. Decide what happens when storage fills. A plain circular buffer overwrites the oldest data. For edge cases, evicting the lowest-priority clip is the better policy.

On a Jetson Orin NX, one pool of 8 GB or 16 GB LPDDR5 serves all the module’s internal devices. For the persistent queue, the Orin NX supports NVMe storage attached over PCIe.

Capturing frames before and after the event within memory limits

An override fires after the interesting moment, so the pipeline needs a bounded cache it can dump on demand. rosbag2’s snapshot mode keeps an in-memory circular buffer bounded by size, duration or both, and writes it to disk only when the ~/snapshot service is called. NVIDIA DeepStream’s smart record offers a similar cache for video. It runs alongside inference, records only feeds with events of importance, and keeps a cache so each clip has frames from before and after the event. Its defaults are a 30-second cache and a 10-second duration. The cache must be longer than the lookback window, and enlarging it raises memory use.

Prioritising and compressing before upload over spotty rural links

Order the queue so that the most important data goes first when the link returns, the same principle general edge-buffering designs use for critical alerts. Rank override-triggered clips above confidence-only flags, and confidence-only flags above diagnostics. For compression, rosbag2 supports zstd per file or per message, and the Orin NX has hardware H.265 encoding, up to 12 streams at 1080p30. Compression trades storage against the fidelity a labeller needs. Keep the acted-on frame at high quality and compress only the context around it.

One more cut is worth making before upload. Stream-Based Active Distillation first buffers a candidate set, then filters it down to the transmission budget. Among the filtering methods its authors tested, Farthest First (which maximises coverage of the embedding space) performed best, matching or beating baselines with substantially fewer images. They also found compact embeddings from edge-compatible ViT models such as DINOv2 practical and effective. Borrow the structure, not the selection direction: that framework keeps the frames its student model is most confident about, because a teacher model labels them. Where humans label, you want the opposite.

The score that decides candidates does not need the full cloud model. Score candidates on the robot with a single-pass uncertainty estimate plus a novelty score, and let the cloud model re-score whatever arrives. If the open question is which Jetson-class module to fit, LynxBenchAI’s published results show how each device runs the public PyTorch stack that custom models are built on. That makes them broadly indicative for your models too, though they do not measure your own application.

Which query strategies can run cheaply on the harvester’s onboard compute?

Run a single-forward-pass uncertainty score together with a novelty score, independently for each camera. Leave multi-pass methods such as Monte Carlo dropout and ensembles to the cloud re-scoring stage. TechnoLynx builds the on-device pre-filter so rare cases survive at real-time throughput across every feed on your Jetson hardware.

In a satellite-borne edge-learning study run on an NVIDIA Jetson Orin NX, uncertainty estimation cost far more than retraining the model, and MC dropout was costlier than DER and DBR. That study’s deep beta regression produces its uncertainty estimate in one forward pass. Deep Deterministic Uncertainty matches deep ensembles on several out-of-distribution and active-learning benchmarks at lower compute cost. It does so by fitting a feature-space density estimator after training a single network.

Scoring strategies compared for on-device triage

Strategy	What it signals	Onboard cost	Where it falls short
Raw max-softmax or entropy	How unsure the model is about known classes	Computed from the existing pass	Classifiers can fail silently with high confidence when the data shifts
Temperature-scaled softmax	Calibrated confidence on known classes	A single parameter	Still a confidence score, not a novelty signal
Single-pass density or evidential head (DDU, deep beta regression)	Epistemic uncertainty	One forward pass plus a small estimator	DDU relies on a well-regularised feature space, and deep beta regression targets bounded regression outputs
Energy score	Out-of-distribution inputs		Separates in- and out-of-distribution data better than softmax
Tracker–detector disagreement	Temporal inconsistency between prediction and detection	Small if a tracker already runs	Only covers tracked objects
MC dropout or ensembles	Best-calibrated uncertainty	Multiple passes; MC dropout measured costlier on Orin NX	Better suited to the cloud re-score than to the robot

Model disagreement deserves a mention for harvesters that already track objects. Confidence-Triggered Detection runs its tracker on every frame and treats a significant gap between the tracker’s prediction and the detector’s output as the signal to act. A harvester that already tracks fruit can reuse that gap as an edge-case flag.

Per-camera or joint filtering across four feeds?

On a four-camera harvester, start per-camera. Add a joint cross-camera stage only when fields of view overlap enough that an object spanning two cameras is a failure mode you actually see, and only if the latency and memory budget survive it. This is a judgement about trade-offs rather than a measured result. Prototype the joint stage on recorded footage before committing hardware budget to it.

How do we keep rare cases when the model is under-confident about everything?

Stop trusting raw softmax, calibrate on held-out field data, and flag a frame if it is either novel or risky rather than requiring uncertainty alone. Guo and colleagues found modern neural networks poorly calibrated, and showed temperature scaling (a single-parameter variant of Platt scaling) to be surprisingly effective on most datasets. Softmax-based confidence also suffers from overconfident outputs on out-of-distribution data, which energy scores are less prone to. ThriftyDAgger asks for human help at states that are novel or risky, two separate conditions. Applied to triage, that “or” is what protects rare cases.

A novelty detector flags anything that deviates from the training data, whether it is a true anomaly or simply a sample the training set never covered. Both kinds are worth a label on a harvester. The uncertainty method also changes what gets selected: Beluch and colleagues found that ensemble-based uncertainty picked rDR images with significantly increased probability per acquisition step on a highly imbalanced dataset where about one fifth of images were rDR. This is one reason to re-score uploads with a better-calibrated model in the cloud.

When a model is under-confident across the board, uncertainty ranking alone does not protect rare cases. Two fixes help:

• Set per-class thresholds on held-out rare-event data.
• Reserve a small random slice of the upload budget for frames the filter rejected, so you can estimate what the filter is missing.

Two limits remain. A novelty score tuned too loosely flags too much and undoes the volume savings. And any calibration is only as good as the held-out rare cases: a failure mode nobody has recorded yet can still slip through.

Ship the capture loop before the retraining loop

A working capture-and-triage loop delivers packageable value before any retraining exists. A ranked, frame-aligned set of override failures is a usable intermediate artefact on the way to retraining. Waiting for a fully closed loop before getting any value risks losing the project’s momentum.

Downstream, effective pipelines link monitoring metrics directly to automated retraining triggers, so drift does not go unnoticed. Clustering recurring failure patterns for a retraining cadence, and fleet-wide drift monitoring, are separate pieces of engineering. They build on this capture layer rather than replacing it.

Who can build edge-case capture and triage for our harvester fleet?

TechnoLynx builds the capture-and-triage side of the loop for onboard perception. That covers the override-to-frame trigger, the on-device scoring that decides what is worth keeping, and the buffer and upload scheme that brings it home over patchy connectivity.

TechnoLynx’s Computer Vision R&D engagement begins by characterising your flagged-frame rate, storage and connectivity across sites. On build versus buy: a team already running ROS 2 or DeepStream on its robots, with someone who owns calibration and threshold tuning, can build a first version in-house. A specialist earns their place when scoring has to fit a real-time budget across several feeds on Jetson-class hardware, or when nobody owns validation. When choosing a partner, ask how they would measure recall on rare cases, not just how much upload volume they would cut.

If your fleet already uploads full footage cheaply, on-device triage matters less, and the work moves to cloud-side selection.

FAQ

Why do AI models struggle with edge cases?

Classifiers tend to fail when the test distribution differs from training, and they often do so silently, with confident wrong predictions. In HG-DAgger, doubt is expected to run highest, and performance worst, in regions poorly represented in the training data.

What is edge inference?

Edge inference runs the model where the data is generated. It suits cases where a network round-trip is unacceptable or connectivity is not guaranteed, at the cost of harder version management across many devices.