A maintenance-monitoring gateway can host edge inference when its own measurements show room for it. That means per-unit CPU and memory headroom while it does its current job, battery state where it runs on battery, and the uplink volume that local scoring would save or spend. ThingsBoard’s IoT Gateway already publishes CPU, RAM, disk and bytes sent and received as statistics, and LwM2M objects define battery level, free storage, link utilisation and transmitted data per device. Read those feeds first, shorten their reporting interval, count what the telemetry itself costs, and add an agent only where a feed has a real gap, such as per-process CPU for the inference runtime or battery capacity on a Linux host.
What to instrument before a gateway hosts edge inference
This placement decision rests on four quantities: CPU headroom, memory headroom, power headroom and uplink cost. Each one has to be measured per gateway, over a window long enough to include the gateway’s busy periods.
LynxBenchAI measures AI performance across accelerators on one scale, so where a gateway class lacks the headroom a scoring model needs, the first step is to compare candidate accelerators in its published results before choosing new hardware.
TechnoLynx would start by working out the smallest set of metrics the placement decision actually needs. TechnoLynx would then adapt the inference runtime’s instrumentation hooks to emit CPU, memory and uplink-cost figures, aggregate them on the gateway to limit added uplink traffic, and check accuracy and overhead on a representative subset of gateway classes before rolling out across the fleet against the budget timeline. If the decision turns into which accelerator hardware a gateway class should carry, LynxBenchAI measures AI performance across accelerators on one scale.
In practice the metric set looks like this:
- CPU: host utilisation, plus the share used by the inference process itself, because host-level percentages cannot tell the runtime apart from the protocol connectors running beside it.
- Memory: RAM in use and RAM available, kept separate from storage space.
- Power: battery capacity, status and estimated time to empty, where a battery exists.
- Uplink: bytes sent and received per period, plus the depth of the gateway’s outbound buffer. ThingsBoard reports storage statistics, including the number of messages currently held in gateway storage.
Read these together, not one at a time. ThingsBoard’s own guidance is to chart a transfer metric and a host metric on the same axis, because throughput that flattens exactly as CPU saturates is a diagnosis, while either curve alone is only an observation.
Where the gateway already runs an inference runtime, or can run one, the cheapest option is to have that runtime report its own resource use instead of installing a second monitoring stack. AWS IoT Greengrass is a working example of runtime-level telemetry: its nucleus telemetry emitter publishes system health data, including CPU and memory usage, to a local topic every 60 seconds by default.
Collecting CPU, memory and uplink telemetry across thousands of mixed gateways without flooding the uplink
Yes, it can be done lightly, provided the telemetry stays as periodic metrics and does not grow into traces or logs. A 2024 Umeå University thesis on OpenTelemetry in microservice architectures found that adding observability as metrics cost less than adding tracing, across every performance type it measured. The same study recorded overheads as large as 42% for CPU, 22% for memory, 24% for latency and 30% for network. Those figures come from a containerised microservice system the thesis tested. Read them as a warning about what heavy instrumentation can cost, not as a prediction for your fleet.
The thesis’s batching and sampling tests were run on trace export; they show where the cost comes from. Smaller export batches raised CPU overhead steadily from 18.4% to 49.0%. Larger batches raised memory overhead only modestly, which makes batching an efficient way to trade memory for CPU and latency. Batching also delays the data’s arrival at the backend. Sampling cut CPU overhead from 17.8% to 3.6% but did nothing for memory, and it cannot be applied to metrics the way it is to logs and traces, because a sampled counter loses events. For gateway headroom, the useful levers are therefore the batch size and the reporting interval, not sampling.
The reporting interval has a direct price. AWS documents that lower publish intervals for the Greengrass telemetry emitter mean higher CPU usage on the core device, and recommends starting at the default and adjusting from observed CPU. On constrained links the encoding matters too: the LwM2M TLV format adds between 2 and 5 bytes of overhead per value.
The claim that per-gateway metrics scale to the whole fleet without materially raising traffic is a reasonable expectation, given how small a periodic metric payload is, but nobody has demonstrated it. Measure it on a pilot group before you rely on it.
How quickly can runtime telemetry be in place before a budget deadline forces the placement decision?
It depends on which of two paths the fleet is on. If the existing platform already collects the right metrics, the work is configuration: ThingsBoard’s remote configuration exposes a statistics tab where general and custom statistics are switched on and their send periods set. If the gateways need a new agent, the timeline follows your over-the-air rollout process, and the sources give no basis for estimating that.
The default cadence matters on both paths. ThingsBoard IoT Gateway statistics are enabled by default with a send period of 3600 seconds, which gives 24 samples per gateway per day (86,400 ÷ 3,600). That is too coarse to catch short CPU peaks or the drain of a battery overnight. The ThingsBoard general configuration example uses a 60-second period instead. Whatever interval you pick, the data needs enough elapsed time to cover a representative operating cycle before the budget date.
Measuring compute and battery headroom on each edge device
The edge-eligibility decision needs per-gateway figures, not fleet averages. An aggregate dashboard can look healthy while individual gateways sit below the threshold a scoring model needs. Keep the data at unit level and add up only after you have applied the threshold.
LynxBenchAI measures AI performance across accelerators on one scale, so where a gateway class falls short of a scoring model’s threshold, the first step is to look up candidate accelerators in its published results before choosing new hardware.
For this decision, compute headroom is what remains at the gateway’s busy times, not its average. ThingsBoard’s machine statistics report CPU, RAM and disk usage as current utilisation percentages of the gateway host, which is enough to estimate host-level headroom if the reporting interval is short enough to see peaks.
Two standard interfaces report battery state:
- On Linux gateways, the kernel’s power supply class exposes battery properties to user space through sysfs, with fixed units. These include CAPACITY in percent and TIME_TO_EMPTY in seconds while the battery powers a load.
- On LwM2M devices, the Device object defines Battery Level as a percentage, Battery Status (normal, charging, low battery, damaged and others), and voltage and current for each available power source.
Three caveats apply to battery data. A gateway that reports rarely or drops offline gives a battery figure that can lag the real state. Time-to-empty depends on the present load, so it shifts once scoring starts. And the telemetry you add draws power and CPU too, so measure its cost as part of the headroom it is supposed to reveal.
Can Kura or ThingsBoard give us per-gateway CPU and battery stats today, or do we need our own agent?
ThingsBoard gives you per-gateway CPU, RAM and disk today, as long as statistics reporting is enabled on the gateway. Battery is a different matter. ThingsBoard’s custom statistics run shell commands on the gateway and report each command’s output as a telemetry value, so a command that reads the kernel’s battery capacity can feed battery state through the same channel without a new agent. One documentation detail to check on your version: the configuration reference describes statistics as reported as device attributes, while the widget guide describes them as telemetry.
For Eclipse Kura, the sources show a framework that runs on anything from a Raspberry Pi to a high-performance computer, with a Java plugin API, a REST API, an AI inference service for NVIDIA Triton models, and Eclipse Kapua for managing many instances. They do not list built-in CPU or battery metrics. Whether your Kura deployment exposes them depends on the device agents and configuration you run, so check it against your current setup rather than assume it.
LwM2M’s memory figure is a trap for headroom work. The Device object’s Memory Free resource is the estimated storage space available for data and software, in kilobytes, not free RAM. It tells you whether a model file fits in storage, not whether there is RAM to run it.
What each feed gives, and what it does not
| Feed | Reports per gateway | Missing for a placement decision | Default cadence |
|---|---|---|---|
| ThingsBoard IoT Gateway statistics | CPU, RAM, disk usage %, bytes sent and received | Per-process CPU and battery, unless added as custom command statistics | 3600 s send period |
| LwM2M Device object | Battery level and status, power source voltage and current, storage free and total | Free RAM (Memory Free counts storage) | Not given in the sources |
| LwM2M Connectivity Statistics | Tx and Rx data in KB per collection period, including IP headers; link utilisation % | Compute and power | Collection period configurable; 0 means unset |
| Linux power_supply class | Battery capacity %, time to empty and time to full in seconds | Has to be read and shipped by something else | Read on demand from sysfs |
| Greengrass telemetry | CPU and memory in use | Battery and uplink | 60 s to the local topic; hourly aggregation and 24-hourly publication to EventBridge at MQTT QoS 0 |
The Greengrass row shows why the cadence column matters. The cloud-side agent publishes once a day on a best-effort basis and may fail to deliver while a device is offline. The local 60-second topic is the feed that supports a headroom decision.
When an agent earns its place
Add an agent when the decision needs a measurement the existing feed cannot provide. On gateways, the gap that matters here is per-process figures for the inference runtime. The OpenTelemetry Collector’s Host Metrics receiver is built for agent deployment and reports per-process CPU, memory and disk I/O on Linux, Windows, macOS and FreeBSD. Its collection interval defaults to one minute, and you can run several receivers at different intervals. Prometheus Node Exporter is another option: it exposes a wide range of hardware and kernel metrics, such as node_cpu_seconds_total, for scraping. Either agent adds a process to the gateway, so validate it on the pilot group under the same overhead test as everything else.
From headroom telemetry to an edge-or-cloud call per workload
Telemetry turns into a placement decision once each workload has a threshold and each gateway has a measured value to compare against it. The order matters, because thresholds should rest on measurements:
- Profile the scoring model on one gateway of each hardware class. Record its CPU, RAM and power draw at the intended scoring rate.
- Set eligibility thresholds per class. Peak CPU plus the model’s load must stay below a margin you choose, available RAM must cover the model’s working set, and time to empty under the new load must fit the gateway’s charge cycle.
- Compare each gateway’s measured peaks, not its averages, with its class threshold. Use a window that covers a full operating cycle.
- Price the uplink side. From Tx data per period, estimate what local scoring would remove (raw readings no longer sent) and what it would add (scores and alerts).
- Place each workload per gateway. Run it at the edge where the gateway passes and the uplink saving is worth it, centrally where the gateway fails, and decide case by case where the margin is thin.
Getting a model to run on a gateway is AI engineering for embedded devices in its own right. A scoring model that works in a central service is a different artefact from one that works within a gateway’s CPU, memory and power budget, and porting it is worth doing only against a measurable trigger such as a bandwidth cost or a latency limit. How to structure the split itself, including where the cut between gateway and central scoring falls, is the subject of designing split-inference architectures across distributed sensor fleets. Proving latency under production load, monitoring drift on the gateway and pushing model updates over the air are separate problems that come after placement.
Who can instrument our gateway fleet before the placement decision?
TechnoLynx would adapt gateway inference runtimes’ instrumentation hooks to emit per-unit CPU, memory and uplink-cost metrics for the placement decision. Where the question is which accelerator hardware a gateway class should move to, LynxBenchAI measures throughput, latency, memory and energy on that hardware.
This engagement does not suit every team. If your ThingsBoard or LwM2M feed already reports what the decision needs, the work is a configuration change and analysis your own platform team can likely handle. If the real question is fleet management or OTA rollout logistics, your IoT platform vendor is the better first call.
What the evidence does not yet settle
The open question is scale. Whether these feeds behave well across thousands of heterogeneous gateways with reporting tightened to a minute remains open. That makes the pilot group more than a formality. Its job is to measure the telemetry’s own CPU, power and uplink cost on your worst hardware class, so the fleet-wide rollout is sized from that measurement rather than from the documentation.
