A data scientist trains a deep neural network for image classification. They observe that the model achieves 99.5% accuracy on the training dataset, but when evaluated on the independent test dataset, the accuracy drops to 65.2%. What training phenomenon is this model exhibiting?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Alright, pay close attention here because this one bites people in production all the time. You build this complex model, you run it on your training data, and the accuracy is a spectacular 99%. You're feeling like an absolute genius! But then you throw some brand new, unseen test data at it, and the performance drops off a cliff. What happened? Your model memorized the training data—including all the random noise and quirks—instead of learning the actual underlying patterns. We call this overfitting or overlearning. Think of it like a student who memorizes the exact answers to a practice exam, but has no idea how to solve a slightly different question on the actual test. To fix this, you need regularization, dropout, or more data. Got it? Sweet.
Full explanation below image
Full Explanation
The discrepancy between a very high training accuracy and a significantly lower testing accuracy is the classic signature of overfitting, also known as overlearning. Overfitting occurs when a model learns the detailed patterns, noise, and statistical fluctuations in the training dataset to such an extent that it negatively impacts the model's ability to generalize to new, unseen data. - Option C (Overfitting / Overlearning) is correct because a model that overfits has high variance, meaning its predictions are highly sensitive to the specific training data points it was exposed to. - Option A (High bias) is incorrect because high bias indicates that the model is too simple to learn the underlying patterns, resulting in poor performance on both the training and testing datasets. - Option B (Underfitting) is incorrect because underfitting is characterized by low training accuracy, which is the opposite of the scenario described. - Option D (Low learning rate) is incorrect because a low learning rate affects the speed of convergence during training but is not the structural cause of a generalization gap between training and testing data.