MACHINE LEARNING · QUESTION 47
What is the difference between linear and logistic regression?
On this page
Linear regression predicts a numeric value by modeling a weighted combination of input features. Logistic regression models the probability of a class, commonly by applying a sigmoid to a linear score for binary classification. Despite its name, logistic regression is a classification method. Both are interpretable baselines, but their assumptions and output meanings differ.
LET'S MAKE IT SIMPLE
First, the idea in plain English
A house-price model should output a number such as an estimated price, so linear regression is a natural starting point. A spam classifier must estimate whether an email belongs to a class, so logistic regression maps a score to a probability. A decision threshold then turns that probability into a class label.
A USEFUL COMPARISON
Picture it this way
A ruler estimates how much something weighs, while a weather forecast estimates the chance of rain. One produces a numeric measurement; the other a probability used for a decision. Real models require assumptions about data and may need calibration or nonlinear features.
FOLLOW THE JOURNEY
Numeric prediction and class probability are different outputs
Compare the output and decision path of the two regression models.
This simplified visual highlights the main relationships. In practice, the exact design depends on the requirements; Different objectives need different evaluation.
Features feed a linear weighted score; linear regression returns a numeric estimate, while logistic regression maps the score to class probability and applies a threshold.
How to read this visual
This simplified visual highlights the main relationships. In practice, the exact design depends on the requirements; Different objectives need different evaluation.
UNDER THE HOOD
How it works, step by step
Build a feature score
Combine input features with learned weights.
Why it matters: Both methods start with a linear score.
Choose output mapping
Return the score for linear regression or map it to a probability for logistic regression.
Why it matters: The output must match the target type.
Evaluate the task
Use numeric error measures or classification metrics as appropriate.
Why it matters: Different objectives need different evaluation.
OUTSIDE THE TEXTBOOK
A practical example
For delivery-time prediction, linear regression might estimate 32 minutes and be evaluated with an error metric such as mean absolute error. For whether a delivery will be late, logistic regression estimates a probability; the team chooses a threshold based on the cost of missed warnings and unnecessary alerts.
GO A LITTLE DEEPER
Technical details that matter
- Linear regression commonly minimizes squared error; logistic regression commonly minimizes log loss.
- A logistic probability threshold is a separate decision policy and need not be 0.5.
- Feature transformations can represent some nonlinear relationships in either model.
MAKE A GOOD CHOICE
When is this useful?
- Linear regression for numeric targets and interpretable baselines.
- Logistic regression for binary classification and probability estimates.
- Simple benchmarks before considering more complex model classes.
Use tree-based or nonlinear methods when relationships are complex, but compare them using the same representative validation strategy.
WATCH OUT FOR
Common misunderstandings
- Assuming logistic regression predicts a continuous numeric outcome.
- Treating 0.5 as the universally correct classification threshold.
- Comparing regression and classification metrics as if they meant the same thing.
REAL-WORLD TRADE-OFFS
Limitations and failure cases
- Both models can underfit nonlinear patterns without suitable features.
- Correlated features can complicate coefficient interpretation.
- Logistic probabilities may need calibration.
WHAT TO REMEMBER
Key takeaways
- 1Linear regression predicts numeric values; logistic regression estimates class probabilities.
- 2The name logistic regression does not mean it is a regression-output model.
- 3Select metrics and thresholds based on the task.
PRACTICE THE FOLLOW-UP
What might an interviewer ask next?
Why is logistic regression called regression?
It models a linear relationship to the log-odds of a class, even though its common use is classification.
Can logistic regression handle more than two classes?
Yes. Multinomial or one-versus-rest approaches extend it to multiclass problems, with different probability and decision behavior.
SAY IT OUT LOUD
How to explain this in an interview
“Linear regression predicts a numeric target from a weighted feature combination. Logistic regression converts a linear score into a class probability and is commonly used for classification. I would choose metrics for the output type and set a classification threshold based on error costs. Logistic regression is called regression because of its statistical formulation, not because it predicts a continuous value.”
Connect this question to the roadmap topic it supports.