Production ML & MLOps

Monitoring & Drift

ML monitoring tracks data, predictions, model quality, business outcomes, and operational health after deployment. Drift means production has moved away from the assumptions under which the model was trained or validated.

Data drift means input distribution changes
Concept drift means the relationship changes
Prediction drift can be an early warning
Ground truth often arrives late
Slices reveal hidden regressions
Operational monitoring is necessary but insufficient

What to monitor
Signal	Examples
Input data	Feature distributions, missing rates, schema changes
Predictions	Score distribution, class balance, output length
Quality	Delayed labels, human review outcomes, eval samples
Slices	Region, language, cohort, tenant, device, plan
Operations	Latency, errors, cost, fallback rate
User outcomes	Complaints, corrections, conversions, escalations

Monitoring loop

Drift alerts should trigger investigation, not automatic panic.