A model that was accurate at deployment is not necessarily accurate a year later. Suppliers change, surfaces change, lighting ages, seasons turn, and accuracy decays quietly while the dashboard still shows green. MLOps for devices is the discipline that makes that decay visible and fixable — model versioning, staged over-the-air delivery, drift monitoring and a retraining path that does not require the original team.
Deployed models rarely fail loudly. They degrade — a new supplier's surface finish sits slightly outside the training distribution, a lamp dims over eighteen months, a seasonal change alters the ambient contribution. Each shift is small. The system keeps returning confident answers, and the confidence is no longer warranted.
The only reliable defence is measurement: track the input distribution, track the output distribution, sample and review real decisions, and define what triggers a retrain before anyone needs it.
Every deployed model is traceable to its training data, code, hyperparameters and validation result. Without that, a regression cannot be diagnosed and a rollback is a guess.
Models are delivered over the air to a canary group first, with automatic rollback on a defined failure signal, before reaching the fleet.
Image statistics, sensor distributions and feature histograms compared against the training distribution, so a change in the world is detected before it becomes a change in accuracy.
Prediction distribution, confidence distribution and reject rate tracked per line, per site and per unit — because a fleet-wide average hides a single failing installation.
Low-confidence and disagreement cases routed to a human, generating exactly the labelled data the next training round needs.
A pipeline that can be run by the customer's team, with validation gates that prevent a worse model from being promoted.
Which model made which decision, when, on what input — required in regulated production and useful everywhere else.
| Type | What changed | How it is detected | Response |
|---|---|---|---|
| Covariate drift | Input distribution — new supplier, aged lighting, seasonal ambient | Input statistics diverging from the training distribution | Recalibrate or retrain on recent data |
| Label drift | Class balance — a defect mode becomes more or less common | Output distribution shift against process records | Rebalance and retrain |
| Concept drift | The relationship itself — acceptance criteria tightened | Rising disagreement between model and human review | Re-annotate to the new standard and retrain |
| Hardware drift | The sensor — lamp ageing, lens contamination, sensor degradation | Reference target imaged on a schedule | Maintain hardware; retraining will not fix it |
The fourth row matters most in practice. A maintenance problem misdiagnosed as a model problem produces an expensive retraining cycle that fixes nothing.
Model drift is the decay in a deployed model's accuracy as conditions diverge from those it was trained on. Causes include changes in the input distribution, changes in class balance, changes in the acceptance criteria itself, and degradation of the sensing hardware.
Through staged over-the-air delivery: the new model reaches a canary group first, is monitored against defined signals, and only then reaches the fleet. Automatic rollback on a failure signal limits the blast radius of a bad release.
It depends on how fast the process changes, and the honest answer is to trigger on monitoring rather than on a calendar. Some inspection models are stable for years; others need attention whenever a supplier changes.
That is the intent. The pipeline, annotation standard, validation gates and documentation are handed over so retraining does not depend on the original engineering team remaining available.
Tell us the specification, the constraint and the deadline. Programmes that cross silicon, radio, embedded and AI are where Faststream is strongest.