Before anyone chooses a degradation or remaining-useful-life (RUL) model, the historian data has to pass an audit, and that audit decides more about the outcome than the model will. The checks are these: whether the sampling rate resolves the failure signatures you want to predict, whether the tags measure what their names suggest, where the gaps are and why, whether the failure labels can be trusted, whether units and sites are comparable, and how many complete failure histories exist. Siang and colleagues list the idea that data volume and variety can make up for quality as a common misconception about industrial data.
Our position is that data quality and quantity, access to the processes that generate more data, and realistic expectations all come before model selection. We hold that accuracy close to 100% is almost never plausible, so the workflow around any model needs fault tolerance and people willing to keep a human in the loop.
Can we build a degradation model from the signals the historian already logs, or do we need new instrumentation first?
You can start with the existing historian if two conditions hold for each target failure mode: the sampling rate resolves the time-scale of that mode’s early signature, and the existing tags cover the physical quantities the signature shows up in. The amount of data the historian holds answers neither question.
Sampling rate belongs to the failure mode, not the historian
The Nyquist-Shannon theorem sets the ceiling: a system can only represent frequencies up to half its sampling rate, and anything above that is either filtered out or aliased into false peaks. For rolling-element bearings, a bearing-monitoring vendor’s guide (IoT Bearings) places envelope analysis, the usual technique for spotting early defects, in the 2,000 to 10,000 Hz structural resonance band and says it needs at least 25,600 samples per second, about ten times the rate needed for simple overall vibration monitoring. The FEMTO bearing benchmark was recorded at 25,600 Hz in 0.1-second bursts every 10 seconds. The authors of one study on the FEMTO data halved that rate to 12,800 Hz and still kept the range up to 6,400 Hz, where they expected the signs of degradation to appear.
ISO 17359 states the general rule: the measurement interval depends on the type of fault and how fast it progresses, and transients may need high-speed acquisition even where steady-state monitoring does not. A historian that logs vibration as an overall value at 1 Hz cannot hold envelope-band information at all, however many years of it there are.
Tag coverage decides whether new sensors come first
Coverage is the second test: does any existing tag measure the quantity in which each target failure mode shows itself? Siang and colleagues advise checking that each tag of interest moves beyond background noise, because a static input carries no predictive power and may point to an instrumentation or historian fault. Amazon’s documentation for Lookout for Equipment (a service AWS discontinues on 7 October 2026) makes the same point plainly: the right data matters more than the quantity, and more sensors are not necessarily better. If the audit shows that a signature lives in a band or quantity that no tag records, then new instrumentation comes before modelling, and our guide to specifying sensor instrumentation and data infrastructure for predictive maintenance covers that step.
How do we audit years of historian data for gaps and mislabeled failure events before anyone starts modelling?
Data-cleansing practice puts the audit first: statistical and database checks locate anomalies and contradictions, and the cleaning workflow is written only after their causes are understood. For a historian feeding a degradation model, we would split the audit into three parts: what the historian did to the signals, where values are missing and why, and whether failure events are labelled where and when they actually happened.
What the historian did to the signals
Compression comes first. Thornhill and colleagues (2004) showed that swinging-door piecewise-linear compression, which many commercial historians apply before archiving, changed key statistics such as the mean and standard deviation, and they gave an automated way to detect how severely a series had been compressed. Siang and colleagues note that many sites have since switched compression off, but older slices of a multi-year extract may still be compressed even when recent ones are not.
Retrieval settings come next. Pulling every tag with the same gridding and interpolation (zero-order hold or linear) produces a mismatch between retrieval frequency and actual collection frequency, so these settings should be chosen per tag with plant staff. Tag names can mislead as well: a tag that looks like a raw measurement may be a calculated value that has been filtered, clipped or spliced, and only the control-system logic or historian calculation shows which. Where equipment from several vendors logs on separate systems, unsynchronised clocks introduce timing errors that can invalidate plant-wide analysis.
Gaps, and whether they follow the work
Missing data hurts most when it is not random. In a NIST-led study of HVAC work orders, Conte and colleagues found that data lost at random widened the uncertainty of maintenance KPIs but left their means stable, whereas data lost by type of work shifted the means sharply, even with large samples. The same audit showed that taking recorded close-out dates at face value would have overestimated median work-order duration. Some gaps are disguised as values: in the Fraunhofer IEE CARE wind dataset, the operator for two farms supplied missing values as zeros, so long runs of zeros have to be treated with caution. Nor can cleaning restore what was never recorded, because incompleteness cannot be inferred away after the fact.
Failure and maintenance labels
Maintenance confounds time-to-failure labels: an asset repaired partway through degradation gives misleading early labels, and an asset that never failed is unusable for simple supervised RUL methods. Fault start times can be uncertain too. The CARE authors note that one farm’s logbook held only fault start timestamps, so the true start of each anomaly could differ from the one they set.
Maintenance records carry five kinds of defect, set out in a 2026 study of 16,316 work orders from 280 wind turbines: missing or unsupported system codes, taxonomies too coarse to separate subassemblies, unstructured technician text, maintenance-type labels that contradict the description, and generic action labels such as “repair”. In that archive, records retrieved as corrective or predictive turned out on inspection to include preventive, inspection, reset and retrofit work. In a structured labelling workflow on those wind records, 73% of proposed code assignments across three tasks met the acceptance rule, a routing yield rather than an accuracy. The stakes are high: a benchmark of language models for labelling wind maintenance logs reports that earlier studies found the choice of cleaning method could change a component’s characteristic life by as much as a factor of three, and that two expert organisations labelling the same dataset produced subsystem failure-rate KPIs more than 300% apart.
Records can still be made fit for purpose. Hodkiewicz and Ho (2016) showed that missing or incorrect failure-mode and action fields can be repaired using the free text, knowledge of local maintenance practice and the asset’s position in its life cycle. Work on pairing sensor streams with operators’ written observations argues that the text adds context and ground truth for the state of the component. Known abnormal periods, such as downtime, should be labelled so they can be kept out of training data for normal-behaviour models.
SAE J1939 gives one diagnostic structure that dealer records can be compared against: a trouble code combines a suspect parameter number (SPN), a failure mode identifier (FMI) and an occurrence count. The cyclic DM1 message, however, is a snapshot of active faults rather than an event log.
A pre-modelling audit checklist for historian data
| Check | What to look for | If it fails |
|---|---|---|
| Sampling rate against failure time-scale | Nyquist frequency above the band of each target signature; high-rate capture for transient modes | Instrument or reconfigure acquisition before modelling |
| Tag semantics and movement | Raw measurement or calculated value; movement beyond background noise | Re-source the tag or drop it |
| Compression and retrieval | Compression in older time slices; retrieval gridding matched to each tag’s real collection rate | Re-extract raw data, or exclude compressed periods |
| Gaps and missingness pattern | Zero-filled or forward-filled values; gaps that follow work type or asset | Treat as biased, not just smaller, data |
| Clock alignment | Timestamps across vendor systems and logs | Realign before joining streams |
| Failure label timing and confounding | Uncertain fault starts; repairs mid-degradation; assets that never failed | Re-label, censor explicitly, or move to unsupervised targets |
| Maintenance-record coding | Missing codes, coarse taxonomy, contradictory type or action labels | Harmonise the taxonomy and review flagged records |
| Comparability across units and sites | Sensor generation, mounting, baseline level, failure definition | Normalise per unit, segment, or retrain |
| History length and failure count | Number of complete run-to-failure trajectories, not years logged | Choose simpler or physics-constrained models |
How much history per asset do we need before a degradation or RUL model can be trusted?
In sample-complexity terms, each sample is a complete degradation trajectory: one asset observed until it fails. Eker, Camci and Jennions report prognostic dataset sizes of 10 to 40 run-to-failure samples in the literature, treat about 10 as the minimum for effective data-driven modelling, and judge two or three failures insufficient.
A 2026 sample-complexity analysis of RUL prediction recommends physics-constrained or low-complexity models below roughly 50 complete trajectories and keeps deep architectures for large datasets. Where the degradation signal is strong relative to noise, it suggests that 10 to 20 trajectories may be enough for physics-informed models. Censoring is expensive: for bearing-like power-law degradation, units observed for only half their life retain so little information that matching complete data takes 32 times as many censored trajectories. Fleet variability sets a floor: if unit-to-unit spread exceeds the target accuracy, no amount of data or model complexity will help, and the fix is to segment the fleet or adapt to each unit. The authors flag their own bounds as conservative by a factor of about two and as built on assumptions (Gaussian noise, independent censoring, no train-to-field shift).
The CARE benchmark’s builders required at least one full year in every sub-dataset so that seasonal effects could be learned, a design criterion for that dataset rather than a general minimum.
Mapping a new site’s tags and sample rates onto the features the model was trained on
A new site’s or acquired fleet’s historian can be reconciled with a trained model’s feature space when sensor type, mounting and sampling rate are close enough that the original feature definitions survive the mapping. When they do not survive (a missing tag, a sampling rate below the feature’s band, different sensor physics), the model needs partial or full retraining, whatever the recalibration looks like on paper.
Map tags by what they physically measure, checking control logic rather than names. Re-extract at retrieval settings that match each tag’s true collection rate. Re-derive the features from raw signals where possible. The smallest instrumentation change that keeps the existing model valid is the one that restores the inputs its features depend on: a sampling rate covering the feature’s band, comparable mounting, and the same measured quantity.
In a study of aircraft compressors using Collins Aerospace health-management records, units at different installation positions ran at different baseline surge margins, so the same absolute value could mean early or late life. The authors therefore built features relative to each unit’s own first-window baseline and normalised them by accumulated flight hours. They also found that service lives in a real fleet can differ by a factor of three or four, and that a model calibrated to the source fleet’s lifespans makes systematic errors on longer-lived units that feature alignment alone cannot remove. Recalibrating the temporal encoding from early operating data cut mean absolute error by 67% across 18 leave-one-out folds, with improvement in 15 of them.
Is there a standard way to harmonise multi-vendor historian schemas for predictive maintenance models? The standards that exist address neighbouring problems, not the mapping of tags onto model features. ISO 13374 defines an open architecture so that condition-monitoring data can move between sensors, acquisition hardware and analysis platforms. ISO 14224 standardises the categories and format of reliability and maintenance data. In wind, RDS-PP and the ZEUS state-event-cause code have been used to make maintenance reports from different organisations comparable. Each mapping from a historian to a feature set remains site-specific and has to be re-validated each time.
Two cautions apply. A lab-to-field or site-to-site shift in the data distribution adds error that the training-set bounds do not cover. A clean validation at handover also does not keep the model valid, so input drift needs monitoring from the first day the new site feeds the model.
Combining stations whose monitoring started at different times
Condition data from stations retrofitted at different times, or on different sensor generations, should not go into one degradation model raw: older streams need re-baselining against a common reference, and units with short histories should carry weight in line with how mature their data is.
TechnoLynx’s AI engineering for manufacturing and automotive work can turn condition data from mixed sensor generations into one consistent input for the degradation model: characterising the sensor generations, retrofit timelines and gaps across stations, designing the re-baselining protocol and history-length weighting, building the recalibration pipeline into the degradation model, and testing it against overlap periods. The first step is characterising the sensor generations, retrofit timelines and data gaps at each station.
ISO 17359 defines baseline data as measurements taken while operation is known to be acceptable and stable, recommends allowing for a wear-in period on new or overhauled equipment, and notes that a baseline may have to be re-established after component changes or adjustments. The same guidance recommends that monitoring records also include machine configuration and component changes. We would re-baseline an old stream against a newer one only where both ran together or a reference standard exists.
On weighting, the compressor study above showed prediction intervals that were wide early in each unit’s life and narrowed as service hours accumulated. We would carry that uncertainty forward for short-history units rather than drop them, because heavy down-weighting can under-represent the newest assets’ true condition. If degradation mechanisms change between asset generations, analysing each type separately is safer, as the wind reliability study authors did when they isolated ageing effects by turbine type.
Are we capturing the signals around extreme events, or averaging them away?
If the historian stores only fixed-interval averages around an extreme event such as a large swell, the transient detail was never captured and cannot be inferred later. In the CARE wind SCADA dataset, for example, each sensor is available as a 10-minute average, and some also carry 10-minute minimum, maximum and standard deviation values. ISO 17359 notes that transients may need high-speed acquisition, and data-cleansing practice is clear that facts not captured at recording time cannot be inferred afterwards.
Our recommended forward fix is event-triggered capture. Keep a rolling raw high-rate buffer, and when a load or sea-state threshold is crossed, save the window before and after the event.
Can simulation fill the rare-failure gap? Partly. In Wind Energy Science (2025), researchers working with nine 2 MW turbines and a single documented gearbox failure found that a model trained on that one event confused the season of the failure with fault features, and that adding synthetic run-to-failure signals of the same failure under different seasonal conditions reduced those false positives. They also warn that performance may degrade if the real failure mode differs significantly from the simulated one. A digital twin that misses the true failure physics under extreme load will produce training data that is consistently wrong, so we would validate synthetic data against whatever real extreme-event records exist before using it to retrain. Published work such as this is a cross-check, never a replacement for validating against your own assets.
Explaining the path from raw vibration to an RUL distribution to a committee
An auditable method is a documented chain of named stages, each with its inputs, transformations and assumptions written down, so that a committee or later auditor can trace any RUL distribution back to raw signals. ISO 13374 and the RUL literature both give that chain a vocabulary.
ISO 13374, as summarised in a condition-monitoring glossary (Vibromera), describes a pipeline of functional blocks:
- Data acquisition: sensor power, signal conditioning and analogue-to-digital conversion, producing a raw time waveform.
- Data manipulation: FFT spectra, RMS values, integration and envelope analysis.
- State detection: comparing processed values against alarm thresholds or baseline changes to assign states such as Normal, Alert or Danger.
- Health assessment, then prognostic assessment, which combines the diagnosis with historical trends to estimate remaining useful life.
The RUL literature describes the same path as four processes: data acquisition, health-indicator construction, health-stage segmentation and RUL prediction. The compressor study used Gaussian process regression whose prediction interval informed the removal rule: act when the lower bound crosses the safety threshold.
Explainability has to cover the whole chain. A review of explainable predictive maintenance argues that later prognostic outputs cannot be interpretable if the earlier stages were carried out incomprehensibly. Keeping intermediate outputs and source data at each stage makes the paths by which errors pass from one stage to the next auditable. Citing an established framework does not prove a specific implementation is correctly validated, and a plain-language summary will simplify real statistical uncertainty, so present both the stage diagram and the validation evidence.
Proving the model works on a prospect’s own fleet
The convincing evidence for a sceptical operator is a backtest on their own historical logs, showing which recorded events the model would have caught, how much lead time it gave, how often it raised false alarms, and what it missed. Treat any claimed industry norm for proving generalisation to a new fleet with caution. A backtest says nothing about operating conditions not yet seen, and the package should claim no more than the prospect’s event count supports.
What the audit result says about model choice
Fewer than about 50 complete trajectories point to physics-constrained or low-complexity models. Fleet spread wider than the target accuracy points to segmentation or unit-specific adaptation. Where run-to-failure data is lacking, Shimizu and colleagues (2023) argue that, for railway assets, a fault-severity level plus an approximate remaining time is often all maintenance planning needs. They also concede the limit plainly: without maintenance records or run-to-failure data, the predicted remaining life stays an estimate that cannot be validated.
Siang and colleagues observe that plant staff usually prefer a durable model with slightly lower accuracy to a fragile high performer that needs frequent retraining.
FAQ
How is predictive maintenance different from preventive maintenance?
Preventive maintenance repairs or replaces equipment on a schedule based on population-level statistics. Predictive maintenance aims to determine the condition of each individual piece of equipment and intervene at the right time, using degradation curves for specific failure modes.
Are CAN bus and J1939 the same?
No. SAE J1939 is a higher-level protocol that runs on CAN and uses its 29-bit extended frame identifier, and it is widely adopted in commercial vehicles.
What is the difference between data cleansing and data cleaning?
The two terms mean the same thing: identifying and correcting or removing corrupt, inaccurate or irrelevant records from a dataset.
