Measure first, then split. The cut between sensor, gateway and central scoring belongs wherever measurements on your own device, radio and network show the best balance of three things: bandwidth saved, latency at each stage and battery cost. On a battery-powered sensor node the radio usually costs more energy than the processor, so filtering or classifying on the node pays off most when the raw payload is large. Safety-critical monitoring is a separate problem. There, the time a device has to raise an alarm, and the standard that sets it, decide what stays local before bandwidth costs come into it.

So edge to cloud computing needs two design processes, not one. A fleet of flow meters or vibration sensors on LoRaWAN or NB-IoT needs a cost model that trades backhaul savings against alert latency and battery life. A bedside monitor needs an architecture that keeps urgent detection on the device and can be shown to be just as accurate as a version that doesn’t split.

Three candidate split points and what decides between them

Split	What runs where	Choose it when	Main risk
Node pre-filter, hub or central arbitration	A cheap local filter settles clear cases; borderline readings are escalated	The node runs on battery and anomalies vary in how subtle they are	A badly calibrated threshold either drains the battery or hides real events
All central	Raw or compressed data streams to the gateway or data centre for scoring	Power is not constrained (mains supply, large battery) or radio cost is negligible	Backhaul volume, radio energy and network round-trip time
Bedside detector plus central analytics	Beat and arrhythmia detection on the monitor; trends, retraining and reporting off-device	Any function carries an alarm-time requirement	Accuracy loss from shrinking the on-device model, which must be demonstrated

Each row is a judgement, and measurement has to confirm it for your deployment.

Gateway scoring versus central scoring: what the backhaul saving depends on

Moving anomaly scoring to the gateway cuts backhaul by however much raw data the gateway stops forwarding. The cost is gateway compute and possibly a less accurate model. Keeping scoring central avoids that load but adds communication latency to every decision. TechnoLynx handles this as an engineering engagement. TechnoLynx sets the alert SLA, the duty-cycle and memory limits, and decides which signals are latency-critical. TechnoLynx then builds each candidate split point with stage-level timing and measures bandwidth reduction, latency and energy per split before deploying the one that fits the budget. For the gateway hardware itself, the first step is to compare candidate devices in LynxBenchAI’s published results, which show how each runs the public PyTorch stack your models are built on. That is a broad indication for custom PyTorch models, though it does not measure your own workload or application.

LynxBenchAI publishes results showing how candidate gateway devices run the public PyTorch stack, so the first step is to compare your shortlisted gateways there before building any split.

How much backhaul a gateway pre-filter actually removes

The published figures cover a wide range, and that spread is the useful lesson. An edge streaming architecture reported cloud-uplink bandwidth savings of 82–88% per device-hour, with 12.8 MB per device-hour against 110.4 MB for a cloud-only baseline in its industrial setting. The HiCH healthcare architecture cut traffic by 82% when it sent only one of six windows during normal conditions and the full data once an abnormal window appeared. A LoRa experiment that transmitted only the points DBSCAN flagged as anomalous cut data volume by 98.19%. An edge-AI patient-monitoring framework, which exchanged model gradients and event logs instead of raw data, cut bandwidth by about 38%.

Each of these systems used a different transmission policy. The saving comes from what the gateway decides not to send, so a percentage from another system says little about your mobile data bill.

Building a latency budget for each candidate split point

A latency budget splits end-to-end alert time into uplink (radio airtime, duty-cycle waiting and retries), gateway inference, backhaul and central scoring, and each stage has to be measured. In a TinyML predictive-maintenance study on mining machinery, inference on the sensor averaged 3.33 ms. Offloading to the gateway averaged 148.15 ms, because of compression and radio transmission, and the cloud path averaged 641.71 ms because of internet communication. A cold-chain deployment over LoRaWAN timed each hop and found that network-server processing and message passing made up almost half the latency before inference. On-node processing took about 283 ms, 228 ms of it inference, and a prediction arrived in about 555 ms, not counting the gateway-to-network-server hop. The Neurosurgeon study found that, for mobile DNNs, the status-quo cloud-only approach spent over 94% of its time transferring data and under 6% computing.

For a minutes-scale alert SLA, then, the binding stage is rarely the milliseconds of inference. It is the radio. LoRaWAN regulatory duty cycling stretches each transmission’s holding period to the airtime divided by the duty cycle. At a 1% duty cycle that is 100 times the airtime (1 / 0.01). A 2021 multi-RAT study put uplink latency for a 1600-byte payload at 42 minutes to 68 hours on the default 1% bands. NB-IoT guarantees delivery but varies widely: in an empirical study of 2,880 datagrams, 2,307 arrived within 18 s, 496 took 21–39 s and 77 (about 2.7%) took 256–270 s. A datagram in that tail uses more than four minutes on its own, most of a five-minute alert window. The gateway stage can work in your favour too. A LoRaWAN gateway that generated acknowledgements locally averaged 43 ms confirmed-messaging time, where each retransmission through a cloud network server adds more than 2 s.

The procedure we would follow for each candidate split:

1. Write the SLA as an end-to-end deadline at a stated percentile, not as a mean.
2. List the stages the split point puts on the critical path. Classification on the device carries only processing delay, while classification further up adds device processing, communication and remote processing.
3. Embed a timestamp at every hop so each stage is timed separately, as the cold-chain study did. Our article on instrumenting edge inference runtimes for placement decisions covers the runtime side of this.
4. Measure under the deployment’s real duty cycle, coverage and gateway load, and read the tail.
5. Count a deadline as missed when either latency or accuracy fails, the definition used in calibrated edge-offloading work.
6. Pick the split that meets the SLA and the duty-cycle limit with the largest bandwidth cut, which is not necessarily the one that pushes the most work to the edge.

Until this has been run for your deployment, no latency decomposition exists for it, and any quoted minutes-to-alert figure is a guess. Keeping detection working when the backhaul drops is a separate resilience problem. Our piece on designing split inference architectures across distributed sensor fleets deals with that.

Bedside patient monitoring: which detection stays on the monitor

Any detection with an alarm-time requirement stays on the bedside device, or on a hub on the same ward network, whatever the bandwidth cost. Trend analytics, model retraining and reporting can run centrally because they tolerate network delay and gaps in connectivity. IEC 60601-2-27 requires cardiac-standstill alarms within 10 seconds. A monitor with front-end filtering delay must therefore alarm sooner after the last detected beat. GE’s monitor waits five seconds, and Philips’ has about one second of filtering delay and waits up to four. The standard lists time to alarm for heart-rate alarm conditions among its essential performance requirements.

The mining study shows the same ordering outside medicine, with sensor-local inference in milliseconds and the cloud path above half a second. HiCH’s authors point out that disconnection or bandwidth and latency variation harms remote monitoring, and a connection delay in an emergency can be fatal. Their design gives gateways enough intelligence to keep monitoring and notifying locally when the cloud is unavailable.

Moving the detector onto the device needs evidence about accuracy as well as latency. In one edge-AI framework tested on MIT-BIH arrhythmia data, median inference latency was 118 ms on a Jetson-class edge device against 246 ms for a cloud-only baseline. Its p95 of 188 ms sat inside the sub-200 ms threshold the authors cite for clinical decision support. INT8 quantisation in that work cost about 0.4 accuracy points. A wearable cardiovascular study found a much larger drop, with an 8-bit model reaching 0.9055 accuracy against 0.9748 in floating point. Quantisation loss varies by model, so it has to be measured against a baseline that runs without the split. HiCH offers a pattern worth borrowing: train in the cloud and periodically update the local decision-maker, so the bedside detector stays light and the central side carries the personalised modelling.

Power then decides which tier runs what. On the wearable patch, continuous NPU inference drew 0.086 mW, about 7.4 J per day. Streaming raw signal at realistic rates drew 1.145 mW at 2,048 B/s and 2.64 mW at 12,000 B/s, 13 and 31 times as much. Transmitting only when an anomaly is detected keeps the patch’s energy dominated by inference rather than radio. Gateway-class hardware is another matter. A Jetson Nano drew 4.8–5.3 W during continuous inference, which the authors judge unsuitable for microcontroller wearables. Their two-tier test gave sensing to an ESP32-S3 and inference to a smartphone or Jetson Nano gateway, which cut MCU memory demand by 82% and ran the MCU over seven hours on a 750 mAh battery.

What can a battery-powered flow sensor decide on its own?

A battery-powered sensor node must work within strict latency, memory and power limits. Suitable checks include standard-deviation thresholds, moving averages, control charts, change-point tests and small tree ensembles. A survey of embedded anomaly detection reports that statistical techniques need no training, infer in constant time and fit in under 10 KB. Isolation Forest can run on ultra-constrained microcontrollers in under 50 ms and 160 KB. The same survey finds that quantised LSTM autoencoders and compact CNNs handle time series better but often need more than Cortex-M-class devices can supply. Among change-point detectors measured on edge test-beds, non-parametric CUSUM used the least CPU and memory, and Bayesian online change-point detection used the most.

The decision between the three options follows from that. Deciding everything on the node fits rare, simple anomalies and fleets where battery life dominates. Sending everything to the hub fits mains-powered nodes or links where radio cost is negligible. A tiered split fits the middle case of a battery-powered node watching anomalies that range from obvious to subtle. The survey reports that effective systems increasingly use exactly this cascade: a lightweight unsupervised filter screens first, and a heavier model runs only when triggered.

Where the line between flagging locally and escalating to the hub sits

Early-exit designs draw this line with a confidence threshold on the local model’s output. Clear cases are decided on the node; anything below the threshold goes to the hub. Calibration of that threshold matters. In calibrated edge-offloading experiments, conventionally trained models overestimated their confidence and fell short of the target accuracy, while calibrated ones met or came close to the threshold, which made it a usable control. Escalation can also adapt to anomaly history. One hierarchical design moves inference to higher layers when anomalies are frequent and back down when they are rare.

Gating transmission saves energy. On real hardware, sending only anomalous LoRa readings cut energy by 98.10% with DBSCAN and 97.32% with Isolation Forest. DBSCAN caught all 58 anomalies in that dataset and produced 33 false positives. Those savings cover transmission energy only. The authors ran detection offline and deliberately left computation out. Compression has to go into the node’s budget as well: compressing a 5.86 KB signal with Zlib cost 10.67 mJ, small next to 1,570 mJ for the radio but not zero.

Two failure modes pull against each other. A filter that is too aggressive hides real anomalies that never reach the hub, and a filter that is too loose wastes the battery saving. Hub capacity limits the design too. Under concurrent clients, detection delay rose two to four times for parametric CUSUM and Bayesian detectors, while non-parametric CUSUM stayed roughly stable.

Battery life with on-device inference versus sending raw readings

On-device inference saves battery because radio energy grows with data volume while inference energy stays fixed for a given model. In the ESP32-S3 study’s once-daily image case, sleep duration mattered more than the choice of radio and paradigm. TechnoLynx builds this cost model from measurements. TechnoLynx measures compute and radio energy on representative sensor and gateway hardware across a range of edge-scoring fractions, and data volumes under the current and proposed splits. TechnoLynx then projects the trade-off to full fleet scale and recommends an operating point that balances savings, battery life and latency. Where the remaining cost lies in inference on gateways or central servers, the Inference Cost-Cut Pack covers the optimisation step.

On an ESP32-S3, on-device CNN classification cut sensor-node energy by 1.7x to 6.9x compared with raw transmission over LTE-M, NB-IoT or LoRaWAN, depending on transmission frequency. Against the naive cloud baseline, the reductions were 6.8x for audio bird detection, 3.8x for image bird detection and 4.2x for plant disease. Batching raw data instead gave only 1.6x to 2.5x. It also added response latency and risked MQTT timeouts and reconnection energy, so the authors treat those gains as a best case. For an image system on a one-hour cycle, moving to embedded inference over LoRa raised estimated battery life from 1.17 to 5.93 years. In the mining study, the radio cost 1,570 mJ per offloaded cycle against 2.72 mJ for on-board inference, which gave battery-life bounds of about 65 hours offloaded and 104 hours on-board.

The NB-IoT term in the cost model

For NB-IoT, the payload is the cheap part of a message. Measurements on real devices found that sending and receiving data used orders of magnitude less energy than random access, attach and active waiting. Random access repeats on every connection and accounts for a large share of battery drain. Sending fewer messages therefore cuts the connection overhead that dominates energy at short intervals. Coverage changes the term too: low SNR raises energy per message enough to cut expected battery life to half or less. Payload size can flip the radio choice. One study found NB-IoT more energy-efficient than LoRaWAN above 240 bytes. Another found NB-IoT costing significantly more than LTE-M for large payloads, with operator configuration driving the small-payload differences.

Worked structure of a fleet-scale cost model

These are the terms the model multiplies out, stated as structure, without invented values:

• Annual data cost per split = nodes × messages per node per year under that split × the carrier’s price per message or per volume, using whichever structure your contract actually uses.
• Energy per cycle = sensing + inference (with model initialisation on each wake) + connection overhead + payload + sleep.
• Battery life = usable capacity, derated for temperature and ageing, divided by energy per cycle times cycles per year.
• Missed-anomaly rate under the pre-filter, validated separately, because savings that rest on a filter that misses events are not savings.

Is there a published benchmark for compute-versus-radio energy on this hardware?

Not one you can use as a substitute. A 2025 survey notes the lack of standardised benchmarks that evaluate detection performance together with latency, RAM and power. The ESP32-S3 study warns that energy depends heavily on the device and that FLOPs are a poor proxy. Processor and radio interact, so neither can be understood alone. The wearable study expects its trade-off to hold qualitatively but says the crossover payload depends on the radio and accelerator. Profile before deciding the split: the right placement comes from measurements in context, not from a rule of thumb about where inference “should” run. LynxBenchAI’s published device results inform the gateway tier. The microcontroller and radio term has to be measured on your own board, module and network.

Who can build and validate the split for our sensor or monitoring fleet?

TechnoLynx designs, builds and validates edge-to-cloud splits for IoT sensor and monitoring systems. TechnoLynx sets the SLA and battery budget, implements the node or gateway detector and the central model with stage-level timing, and measures bandwidth, latency and energy per split point before deployment. When choosing a partner, ask whether they will measure on your hardware and network, report tail latency per stage, and show accuracy against a non-split baseline.

The engagement does not suit everyone. For a clinical split, TechnoLynx delivers the engineering and measurement, not clinical validation or regulatory sign-off. If the split is already settled and you only need numbers for one module, the module vendor’s application engineers may be the quicker route. If you are buying hours against a fixed design, a contract embedded developer fits better.

FAQ

Is fog computing the same as edge computing?

The terms overlap. “Fog” usually means an intermediate tier of gateways or local servers between devices and the cloud, and “edge” often means the devices themselves or anything near them, though usage varies between authors.

What is the difference between delay and jitter?

Delay is the time a packet takes to get from sender to receiver. Jitter is how much that delay varies from one packet to the next. For an alert SLA, high jitter can matter as much as a high average delay, because the slow tail decides whether deadlines are missed.

What are some common wireless protocols used in the IoT?

For long-range battery-powered sensing, LoRaWAN, NB-IoT and LTE-M are common, and Bluetooth Low Energy is typical for body-worn and short-range devices. Which suits a node depends heavily on payload size: one measurement study found LoRaWAN best for small classification outputs and LTE-M best for larger images and audio.