MLOPS & DEPLOYMENT · QUESTION 73
What is shadow deployment, and when is it useful?
On this page
In shadow deployment, a candidate model receives production-like inputs and generates predictions without controlling the user-facing decision. Teams compare its behavior with the current model, test serving compatibility, and measure resource use before exposing users. Shadowing reveals real-input differences but cannot measure how users respond to the candidate, and it can add compute and data-handling costs.
LET'S MAKE IT SIMPLE
First, the idea in plain English
A candidate model can be run alongside the current one while users still receive the existing result. This lets engineers test the real request path and inspect differences without letting the new model make decisions. It is a useful safety step, but it is not the same as an experiment that measures user outcomes.
A USEFUL COMPARISON
Picture it this way
A trainee may observe a live operation and propose what they would do while the experienced operator remains in control. This gives a comparison under real conditions, but it does not show how the operation would change if the trainee’s decision were used.
FOLLOW THE JOURNEY
Run candidate predictions beside the live decision path
Distinguish shadow outputs from user-facing traffic and subsequent rollout.
This simplified visual highlights the main relationships. In practice, the exact design depends on the requirements; Shadow evidence does not establish user benefit.
Production requests feed both incumbent and shadow candidate models; only incumbent outputs reach users while candidate predictions and resource use are logged for comparison.
How to read this visual
This simplified visual highlights the main relationships. In practice, the exact design depends on the requirements; Shadow evidence does not establish user benefit.
UNDER THE HOOD
How it works, step by step
Mirror eligible inputs
Send safe copies of production requests to the candidate.
Why it matters: Real request shapes expose integration and distribution issues.
Keep output non-authoritative
Prevent candidate predictions from changing user-facing decisions.
Why it matters: Shadow mode limits user impact.
Compare and inspect
Measure prediction differences, latency, errors, and resource use.
Why it matters: Differences need interpretation against expected behavior.
Choose next step
Promote, run a controlled experiment, or reject the candidate.
Why it matters: Shadow evidence does not establish user benefit.
OUTSIDE THE TEXTBOOK
A practical example
A new fraud model receives the same eligible payment requests as the current model, but its decision is recorded only for analysis. Analysts compare disagreement cases and infrastructure load. If the candidate passes checks, the team may move to a canary or randomized experiment with a defined rollback plan.
GO A LITTLE DEEPER
Technical details that matter
- Shadow requests may need feature snapshots so both models see equivalent inputs.
- Asynchronous mirroring can reduce impact on the live request but may miss strict latency behavior.
- Sensitive request payloads should not be duplicated without access, retention, and privacy review.
MAKE A GOOD CHOICE
When is this useful?
- Testing model integration with real production inputs before user exposure.
- Comparing prediction distributions and disagreement examples.
- Measuring capacity needs for a candidate serving path.
Use offline replay when production data cannot be copied safely. Use canary or A/B rollout when user-facing outcomes must be measured.
WATCH OUT FOR
Common misunderstandings
- Claiming shadow results prove user outcomes improve.
- Letting candidate errors affect the primary request path unintentionally.
- Ignoring doubled compute and duplicated sensitive data.
REAL-WORLD TRADE-OFFS
Limitations and failure cases
- No direct measurement of user response to candidate decisions.
- Additional serving cost and possible production load.
- Shadow inputs may differ from what the candidate would receive after rollout.
WHAT TO REMEMBER
Key takeaways
- 1Shadow deployment observes candidate predictions without using them for decisions.
- 2It tests real inputs and integration, not causal user impact.
- 3Control data handling and choose a measured rollout as the next step.
PRACTICE THE FOLLOW-UP
What might an interviewer ask next?
How does shadow deployment differ from a canary?
Shadow outputs do not control user decisions; a canary serves candidate outputs to a limited portion of users or requests.
What should you compare in shadow mode?
Check serving errors, resource use, prediction differences, important segments, and feature parity; decide which differences are expected.
SAY IT OUT LOUD
How to explain this in an interview
“Shadow deployment sends production-like inputs to a candidate while keeping its predictions out of the user-facing decision. It helps test integration, real input behavior, and capacity before exposure. It cannot tell us whether users respond better, so I would use it as a pre-release check and then choose a canary or controlled experiment if needed.”
Connect this question to the roadmap topic it supports.