MLOPS & DEPLOYMENT · QUESTION 71
When should a production model be retrained?
On this page
Retrain when evidence shows the current model no longer meets its task or operational requirements, and when suitable recent labels and data are available. Triggers may include scheduled review, validated performance decline, or meaningful data changes, but drift alone is not proof. Train a candidate, compare it with the current model, and promote only after quality, safety, and deployment checks pass.
LET'S MAKE IT SIMPLE
First, the idea in plain English
A model can become less useful as customers, products, or conditions change. Retraining gives it a chance to learn from newer examples, but an automatic update can also learn bad labels or amplify a data problem. The team needs evidence that updating is appropriate and a way to test the candidate before replacing the current model.
A USEFUL COMPARISON
Picture it this way
A map needs an update when roads change, but not every construction notice means the entire map should be redrawn. First confirm what changed, then check the new map before giving it to drivers. Retraining is a response to verified need, not a reflex to every signal.
FOLLOW THE JOURNEY
Use production evidence to gate a candidate retraining cycle
Show monitoring, diagnosis, candidate evaluation, and controlled promotion.
This simplified visual highlights the main relationships. In practice, the exact design depends on the requirements; A new model can regress even if retraining completed successfully.
Performance and data signals trigger investigation; valid labeled data trains a candidate that is compared with the current model before approval and deployment.
How to read this visual
This simplified visual highlights the main relationships. In practice, the exact design depends on the requirements; A new model can regress even if retraining completed successfully.
UNDER THE HOOD
How it works, step by step
Observe signals
Review task outcomes, input changes, and operational failures.
Why it matters: Different signals imply different root causes.
Confirm the need
Investigate labels, data quality, seasonality, and user impact.
Why it matters: Retraining does not fix a broken source or metric.
Train a candidate
Use representative recent data and a versioned training recipe.
Why it matters: New data and code should be traceable.
Compare and promote
Test quality, safety, and serving compatibility before rollout.
Why it matters: A new model can regress even if retraining completed successfully.
OUTSIDE THE TEXTBOOK
A practical example
A demand forecast receives a drift alert after a holiday period. The team checks whether the shift is expected seasonality and waits for verified sales outcomes. If forecast error remains high after the period, it builds a candidate with new data, evaluates by region and season, then deploys gradually if agreed criteria pass.
GO A LITTLE DEEPER
Technical details that matter
- Scheduled retraining is predictable but can waste compute or learn low-quality data.
- Event-triggered retraining needs thresholds, confirmation, and protection against repeated noisy triggers.
- Feedback labels can be biased by prior model decisions; retraining data should be reviewed for selection effects.
MAKE A GOOD CHOICE
When is this useful?
- Tasks whose input or target relationships evolve over time.
- Models with regular arrivals of verified labels.
- Operational processes that require predictable model refresh decisions.
If the issue is a broken feature pipeline or changed product definition, fix the source or redefine the task rather than retraining.
WATCH OUT FOR
Common misunderstandings
- Retraining automatically on every drift alert.
- Using unverified or biased new labels without review.
- Deploying the candidate because its training job succeeded.
REAL-WORLD TRADE-OFFS
Limitations and failure cases
- Labels may arrive slowly or reflect prior model decisions.
- Frequent retraining increases compute and release risk.
- Historical data may not represent the next operating period.
WHAT TO REMEMBER
Key takeaways
- 1Retrain based on task evidence, not drift alone.
- 2Validate data and labels before training.
- 3Compare candidate and incumbent under release gates.
PRACTICE THE FOLLOW-UP
What might an interviewer ask next?
Is scheduled retraining always safer?
It is easier to plan, but can train on unnecessary or poor-quality data. Scheduled runs should still validate inputs and candidates.
How do you avoid feedback-loop bias?
Review how labels were selected, preserve exploration or control data where appropriate, and evaluate on representative outcomes not solely chosen by the current model.
SAY IT OUT LOUD
How to explain this in an interview
“I retrain when verified task performance or operating requirements justify it and suitable data is available. Input drift can trigger investigation, but not automatic promotion. I validate labels and data, train a versioned candidate, compare it with the incumbent across important segments, and deploy only when acceptance criteria and rollback plans are in place.”
Connect this question to the roadmap topic it supports.