Quiz: Underfitting, Overfitting, Bias & Variance

Questions

Q1. A model performs poorly on both training data and test data. What is this called?

• A) Overfitting
• B) Underfitting
• C) High variance
• D) Good generalization

---

Q2. A model achieves 99% accuracy on training data but only 60% on test data. What problem does this indicate?

• A) Underfitting
• B) Overfitting
• C) High bias
• D) Low variance

---

Q3. Which of the following best describes bias in machine learning?

• A) Random fluctuations in model predictions
• B) Systematic error from oversimplified model assumptions
• C) Error caused by noisy data
• D) Difference between training and test accuracy

---

Q4. High variance in a model means:

• A) The model is too simple
• B) The model's predictions are stable across different training sets
• C) The model is overly sensitive to the training data
• D) The model has high bias

---

Q5. A linear regression model is used to fit a highly nonlinear dataset. This will likely result in:

• A) Overfitting
• B) Underfitting
• C) Low bias
• D) High variance

---

Q6. Which combination describes an overfitting model?

• A) High bias, high variance
• B) High bias, low variance
• C) Low bias, high variance
• D) Low bias, low variance

---

Q7. Adding more features to a simple model will generally:

• A) Increase bias
• B) Decrease variance
• C) Decrease bias
• D) Have no effect

---

Q8. Regularization techniques (like L1/L2) are primarily used to combat:

• A) Underfitting
• B) Overfitting
• C) High bias
• D) Data imbalance

---

Q9. A decision tree with maximum depth (no pruning) trained on a small dataset will likely have:

• A) High bias, low variance
• B) Low bias, high variance
• C) High bias, high variance
• D) Low bias, low variance

---

Q10. Which is true about the bias-variance tradeoff?

• A) Reducing bias always reduces variance
• B) Simple models have low bias and high variance
• C) Complex models have low bias and high variance
• D) Bias and variance are independent of each other

---

Q11. A model that memorizes the training data instead of learning patterns is exhibiting:

• A) High bias
• B) Underfitting
• C) High variance
• D) Good generalization

---

Q12. Cross-validation helps detect:

• A) Only underfitting
• B) Only overfitting
• C) Both underfitting and overfitting
• D) Neither

---

Q13. If you train the same model on different random samples of data and get very different results each time, your model has:

• A) High bias
• B) Low variance
• C) High variance
• D) Good stability

---

Q14. Which action would help reduce underfitting?

• A) Reduce the number of features
• B) Use a simpler model
• C) Increase model complexity
• D) Add more regularization

---

Q15. The training error is very low but the validation error is very high. The gap between them indicates:

• A) High bias
• B) High variance
• C) Underfitting
• D) Optimal model performance

---

Q16. Increasing the size of the training dataset typically helps reduce:

• A) Bias
• B) Variance
• C) Model complexity
• D) Training time

---

Q17. A polynomial regression model of degree 1 (linear) fitting a cubic relationship demonstrates:

• A) Overfitting
• B) Low bias
• C) High bias
• D) High variance

---

Q18. Dropout in neural networks is a technique to prevent:

• A) Underfitting
• B) Vanishing gradients
• C) Overfitting
• D) High bias

---

Q19. When both training error and test error are high and similar to each other, the model is:

• A) Overfitting
• B) Underfitting
• C) Well-generalized
• D) Showing high variance

---

Q20. The "sweet spot" in model complexity is where:

• A) Bias is zero
• B) Variance is zero
• C) Total error (bias² + variance) is minimized
• D) Training accuracy is 100%

---

---

Answer Key

Q	Answer	Explanation
1	B) Underfitting	Poor performance on both training and test data indicates the model is too simple to capture patterns.
2	B) Overfitting	High training accuracy but low test accuracy means the model memorized training data but doesn't generalize.
3	B) Systematic error from oversimplified model assumptions	Bias is the consistent error arising from a model's structural limitations.
4	C) The model is overly sensitive to the training data	High variance means small changes in training data cause large changes in predictions.
5	B) Underfitting	A linear model cannot capture nonlinear patterns, resulting in systematic errors (high bias).
6	C) Low bias, high variance	Overfitting models are complex enough to fit training data (low bias) but too sensitive to it (high variance).
7	C) Decrease bias	More features allow the model to capture more complex patterns, reducing bias but potentially increasing variance.
8	B) Overfitting	Regularization penalizes model complexity to prevent fitting noise in training data.
9	B) Low bias, high variance	An unpruned decision tree can fit any pattern (low bias) but will memorize noise (high variance).
10	C) Complex models have low bias and high variance	This is the core of the bias-variance tradeoff — as one decreases, the other typically increases.
11	C) High variance	Memorizing training data means the model is overfitting, which is characterized by high variance.
12	C) Both underfitting and overfitting	Cross-validation reveals if the model performs consistently across different data splits.
13	C) High variance	Unstable predictions across different training samples indicate high variance.
14	C) Increase model complexity	Underfitting means the model is too simple; adding complexity helps capture patterns.
15	B) High variance	A large gap between training and validation error is the hallmark of overfitting (high variance).
16	B) Variance	More data helps the model learn general patterns rather than noise, reducing variance.
17	C) High bias	A degree-1 polynomial cannot fit a cubic curve, causing systematic underfitting errors.
18	C) Overfitting	Dropout randomly disables neurons during training, preventing co-adaptation and overfitting.
19	B) Underfitting	High error on both sets with small gap means the model is too simple for the data.
20	C) Total error (bias² + variance) is minimized	The goal is to find the complexity level where the combined error is lowest.

---

Quick Reference Summary

Condition	Training Error	Test Error	Bias	Variance
Underfitting	High	High	High	Low
Overfitting	Low	High	Low	High
Good Fit	Low	Low	Low	Low

Key Takeaways

1. Bias = systematic error from model being too simple
2. Variance = sensitivity to training data fluctuations
3. Underfitting = high bias, low variance → model too simple
4. Overfitting = low bias, high variance → model too complex
5. Goal = minimize total error by balancing bias and variance