The Internet of things connects devices that sense the world and transfer data, and AI is what learns from that data and acts on it. The ITU-T (2023) defines the artificial intelligence of things (AIoT) as AI technologies combined with IoT infrastructure, and notes that AI can be applied anywhere along that infrastructure but especially on the device and at the edge. In practice, the decision that most shapes an AIoT system is where inference runs: on the sensor, on an edge gateway, or in the cloud.
TechnoLynx helps IoT teams get each model running on the edge board or embedded target that placement calls for. That covers porting models to microcontrollers, edge boards and other embedded targets, with a reproducible runbook. TechnoLynx’s AI engineering for IoT and embedded devices work starts with a feasibility study on the target hardware.
What is the difference between AI and IoT, and how do they work together as AIoT?
IoT on its own connects distributed devices. Those devices sense their surroundings and pass data to each other or to cloud servers, edge servers and data centres, which provide higher-level services such as data management and network monitoring. A recent peer-reviewed survey of AIoT (Siam et al., ACM TOSN 2024) describes the field in terms of three components: sensing, computing, and networking and communication.
The ITU-T technical paper says plainly what changes when the two are joined: AIoT creates intelligent things that learn from the data they generate and use what they learn to make autonomous decisions. The paper’s conceptual model traces the data from source to application. Raw data comes from sensors and devices, is collected and processed, and then feeds AI-based analytics (data exploration, predictive analytics and decision making) for the application, with security and governance applied along the whole path.
The useful distinction for an engineer is this. A plain IoT system sends readings and lets something else decide. An AIoT system decides somewhere inside the system itself, and that “somewhere” is a design choice with real costs.
Does the Internet of Things use AI?
Not necessarily. The ITU-T notes that existing IoT-based AI relies on cloud platforms, which it calls unacceptable for delay-sensitive services. Combining AI with the IoT infrastructure is what moves a deployment from IoT to AIoT. Once that happens, AI can touch the whole data lifecycle, from the device that collects the data to the service that delivers results to users.
The analysis of machine learning systems in the IoT gives a concrete example of the shift: a camera that only streams video to a remote server is no longer enough, because it is now expected to count people or detect an intruder itself. That example captures the change well. The device stops being a pipe and becomes part of the decision.
How does AI help with the Internet of Things?
AI helps in three broad ways. It quickly analyses huge data volumes, supports high-quality decision making, and sends only the necessary data to the cloud. The edge-intelligence survey by Zhou et al. (Proc. IEEE 2019) describes deep learning finding patterns and anomalies in sensed data such as traffic flow, humidity, temperature, pressure and air quality, then feeding real-time decisions such as traffic control and driving alerts.
Pushing AI towards IoT edge computing also cuts cloud costs. According to the ITU-T, distributed and lightweight AI at the edge significantly reduces the need for cloud analytics, and its cost. With AI on the device, the bulk of the data is processed in real time where it is produced and only the necessary data goes to the cloud, which the ITU-T says protects privacy, reduces latency and improves reliability.
The reported applications span diverse industries. A survey of intelligent IoT cites traffic prediction in vehicular networks, automated disease detection, health-monitoring wearables and agriculture sensors that optimise resource use and crop yields. Medical IoT devices raise their own privacy concerns, because vital signs sent to a remote hospital are sensitive data.
Sensor, gateway or cloud: choosing the tier for each model
AI inference should run as close to the sensor as the model, the power budget and the accuracy target allow. Move it outward only when the device cannot carry the workload. The ITU-T places AIoT capabilities on devices, edges and clouds according to application requirements, and says the three should work together rather than compete.
Each tier has a cost. An analysis of machine learning systems in the IoT, covering work up to 2020, sets the options out as follows. Device-centric processing avoids offloading problems but is limited by the device’s compute and power supply. Gateway-centric computation depends on wireless links and inherits their variability. Cloud-centric processing offers unlimited storage and compute but carries heavy communication overhead. Edge servers sit in between, with stable power, processing close to the data and less data transfer than the cloud would need. The same paper names the core problem as deciding whether, when and how to offload computation when networks are unpredictable.
Zhou et al. give the other side of the cloud argument. Moving large data volumes across a wide-area network can be prohibitively expensive and slow, and it creates a privacy risk. On-device analytics, meanwhile, can suffer from poor performance and energy limits.
A decision table for inference placement
| Where inference runs | What it gives you | What it costs | Fits when |
|---|---|---|---|
| On the device (microcontroller, sensor node) | No server communication during inference, lower latency, raw data processed locally | Bounded by the device’s CPU, memory and power; the model must be shrunk | The decision must be immediate or private, and the model can be compressed to fit |
| On an edge gateway or edge server | Stable power, processing near the data, less transfer than cloud | The device-to-edge link adds variability; performance depends on bandwidth between device and server | Many sensors feed one site, or the model is too large for the devices |
| Split between device and edge | The device runs early layers and sends intermediate data, which helps both performance and privacy | Partitioning has to track bandwidth, device resources and server load | Bandwidth is tight but the device cannot run the whole model |
| Edge plus cloud | Serves highly constrained devices that only collect data | Performance depends heavily on network quality | Latency tolerance is loose and connectivity is dependable |
| Cloud | Training, storage, data normalisation and orchestration of model deployment | High transfer cost and delay; unsuitable for delay-sensitive services | Fleet-wide analytics, retraining, and decisions measured in minutes rather than milliseconds |
Latency has a precise meaning here. Zhou et al. define it as the whole inference process, including pre-processing, model inference, data transmission and post-processing, and note that real-time applications often carry deadlines such as 100 ms. A cloud round trip spends part of that budget before the model has run at all. For the wider architectural comparison, see our discussion of edge computing versus cloud computing.
One pattern divides the work across tiers. The ITU-T describes the cloud handling model iteration and data aggregation while the edge receives new algorithms and runs them in the field. Zhou et al. define one level as “on-device inference and cloud training”, where the model is trained in the cloud and run fully on the device.
Does an AI-enabled IoT device need an internet connection to make decisions?
No, provided inference runs on the device or a local gateway. The ITU-T lists “network-agnostic services” as a property of AIoT: unlike network-dependent cloud AI, AI-enabled devices can operate autonomously and keep providing service while disconnected. In Zhou et al.’s device-based mode, the device gets the model from the edge server and does not communicate with the server during inference.
Connectivity is still needed for the model’s lifecycle. Tiny and small-sized AIoT devices cannot train their own models, so, as the ITU-T notes, they run models trained and distributed from cloud and edge systems. The ITU-T sorts devices into three collaboration models: standalone (no external communication at all), server-client (a platform or edge node deploys and updates the models) and peer-to-peer.
Azure IoT Edge builds offline operation in. Microsoft’s Azure IoT Edge documentation states that after a device has connected to IoT Hub once, it and its downstream devices can keep working with intermittent or no internet connection, and can operate indefinitely while disconnected. In offline mode, the edge hub stores upstream messages until reconnection, authenticates modules and downstream devices on IoT Hub’s behalf, and relays local communication. Stored messages have a time-to-live, with a default of 7,200 seconds (two hours) that can be configured in the deployment manifest. AWS describes the same split for IoT Greengrass V2: local inference on locally generated data using cloud-trained models, keeping low latency and cost savings while training runs in the cloud.
The practical catch is the TTL. If a site can be offline longer than the message expiry, telemetry is lost, so we would set the expiry from the worst realistic outage rather than leave it at the default. Over-the-air updates bring a security problem of their own: the 2024 intelligent-IoT survey notes that updating many devices wirelessly opens a route for attackers to compromise firmware availability and integrity.
What it takes to fit a model onto an IoT device
Low memory capacity is the common constraint of embedded systems. The TensorFlow Lite Micro (TFLM) paper (2020) describes embedded systems that range from a few megabytes of flash and at most a megabyte of SRAM down to a few hundred kilobytes or less. TinyML models in that space are extremely small, a few hundred kilobytes, and can run continuously with little impact on battery life.
The framework design follows from those limits. TFLM cannot rely on an operating system for dynamic memory allocation, so it manages memory from a fixed-size arena supplied by the application, and allocates only during initialisation, never during inference. The interpreter itself takes less than 2 KB, and on the SparkFun Edge board the paper reports at most 13 KB for its convolutional and Google Hotword models and 26.5 KB of framework memory for the larger visual wake words model. The authors chose an interpreter over code generation partly so that models can be updated in the field without re-exporting code.
The model has to be cut down to fit. The ITU-T calls tiny AI one of AIoT’s most obvious features: cloud models must be tailored or reduced for size, power and computation without losing much function or performance. Zhou et al. list weight pruning, data quantisation and compact architecture design as the main compression techniques, which enable on-device inference with lower latency and fewer privacy concerns.
Training on the devices themselves is harder still. A 2023 survey of federated learning (FL) on constrained devices explains that FL lets devices share knowledge without sharing their private data, trading communication overhead against accuracy. The same survey adds that today’s IoT devices remain heavily limited in training capability, and that device heterogeneity is a key complication. FL is not a complete privacy guarantee either: the 2024 intelligent-IoT survey notes that risks remain even when no raw data is exchanged.
How is AI used for predictive maintenance on IoT devices in manufacturing?
AI for predictive maintenance analyses and models sensor signals from equipment. The TFLM authors list predictive maintenance and acoustic-anomaly detection among the TinyML applications built on low-latency analysis of signals from microphones, accelerometers and gyroscopes.
The placement logic from the table applies here. We would run first-pass anomaly scoring on the sensor node or a line-side gateway, so a bearing that starts to fail triggers a local alert even when the plant network is down. Fleet-wide modelling of how equipment degrades, which needs history from many machines, belongs in the cloud. Data quality has a critical impact on the outcome. The ITU-T stresses that training data needs to cover different operating situations to produce generic models, and that biased or malicious data must be removable before training. A model also faces open-world failure on unseen examples, which the 2020 trustworthiness analysis warns about.
Any figure for accuracy or savings should come from a pilot on your own equipment.
How can edge AI on IoT devices help fight air pollution and support smart-city environmental monitoring?
Edge AI helps environmental monitoring by detecting anomalies in sensed data such as air quality and sending only necessary data to the cloud. Zhou et al. (2019) name air quality alongside traffic flow, temperature and humidity as sensed data in which deep learning finds patterns and anomalies, feeding decisions such as traffic control and public transport planning.
The federated-learning survey uses an air-quality network as an example. A PM2.5 IoT sensor network continuously measures fine particulate matter to train an anomaly-detection model. Because the project is open source, its devices run on varied hardware, and outdoor units rely on energy harvesting, which limits how much energy they can spend on training. The survey adds that indoor and outdoor devices differ in energy constraints and in data distribution. That network shows uneven hardware, uneven power and uneven data. It argues for small models on the nodes, with aggregation at the edge or in the cloud.
The ITU-T’s “tiny disposable AIoT” category fits this kind of deployment: a coin-sized, battery-powered sensor that analyses data and makes inferences itself, transmitting over a low-power wide-area network such as LoRa only when needed. The 2024 intelligent-IoT survey describes water management and smart-grid consumption monitoring, and a federated GRU approach to traffic-flow prediction that avoids collecting traffic data in the cloud.
What role does AI play in IoT cybersecurity and safeguarding connected devices against threats?
In IoT, AI for security and defence plays two roles. It is a defender that spots compromised devices from their network behaviour, and it is itself an attack surface that needs protecting. The ITU-T notes that the relationships between devices, users, manufacturers and the network are untrusted, and that AIoT can suffer malware infection and hacking.
AI as the defender
These detectors learn from network traffic:
- Device identification and isolation. IoT Sentinel (ICDCS 2017) fingerprints a new device from 23 features of its setup traffic, none of which depend on the payload, so it works on encrypted traffic. It then places vulnerable device types in an isolated, untrusted network overlay. Its authors report about 150 ms on average to identify a device type, and found a Raspberry Pi II sufficient for a typical environment of around a hundred devices.
- Botnet detection. N-BaIoT (2018) uses deep autoencoders on network behaviour snapshots. Tested on nine commercial IoT devices infected with Mirai and BASHLITE, it detected attacks as they were launched.
- Unsupervised intrusion detection. Kitsune (NDSS 2018) uses an ensemble of autoencoders to learn normal traffic without labels. It detects attacks with performance comparable to offline detectors, even on a Raspberry Pi.
- Federated anomaly detection. DÏoT (ICDCS 2019) builds per-device-type communication profiles and aggregates them through federated learning. It reports a 95.6% detection rate at about 257 ms for devices compromised by malware such as Mirai, with no false alarms in a real smart-home deployment.
Placement matters for security too. The 2024 intelligent-IoT survey cites a Raspberry Pi cluster on a local network running XGBoost intrusion detection, with a predicted inference time of 130 ms against 735 ms in the cloud, and an estimated running cost of 201 against 2,051 Indian rupees a month. The same survey describes fog nodes that apply a drop rule to the MAC-IP pair of a device sending malicious packets.
AI as the target
Deployed models can be attacked directly. The 2024 survey separates poisoning attacks, which corrupt models by injecting malicious data before or during training, from adversarial attacks, which feed slightly modified inputs to a deployed model to cause misclassification. Black-box attacks need no access to the model’s internals, and the survey gives evading a deep-learning intrusion detector in an IoT network as an example. Model extraction attacks query a system to steal its parameters or structure.
The survey recommends three defences against poisoning: clean training data of noise and outliers, make the model resistant through methods such as regularisation or adversarial training, and protect distributed aggregation with cryptographic protocols such as homomorphic encryption or secret sharing. It describes ShadowNet and MirrorNet, which use a trusted execution environment to keep a model’s weights from leaking or being extracted. One consequence follows for design: an anomaly detector that retrains automatically on whatever traffic it sees can be taught by an attacker to accept the attack, so the retraining pipeline needs the same scrutiny as the model.
Who can help us decide where our IoT models should run and get them onto the hardware?
TechnoLynx’s AI engineering for IoT and embedded devices work gets AI workloads running on the edge boards and embedded targets of an AIoT system, starting with a feasibility study on the target hardware. The work also covers porting models to fit the target hardware, with a reproducible runbook. A specialist pays off when the model, the hardware and the connectivity constraints all have to be traded against each other at once. When choosing a partner, ask them to show the latency and memory budget on your target device before any accuracy claim is made.
That kind of engagement is not a fit for everyone. It does not help a team that only needs dashboards over cloud telemetry, and it does not replace device certification, radio design or sector regulatory sign-off, where we can supply the engineering but not the domain authority. Teams whose models already fit their hardware may be well served by the platform tooling itself: the Azure IoT Edge and AWS IoT Greengrass documentation both cover deploying cloud-trained models for local inference.
FAQ
What everyday items use AI?
Keyword spotting is the most widely deployed TinyML example. The TFLM paper notes that Amazon, Apple, Google and others run tiny neural networks on billions of devices for always-on wake-word detection. The 2024 intelligent-IoT survey expects devices to carry small on-device models for tasks such as OCR, face recognition and translation, which keeps private data off the cloud.
