Overfitting
When a model memorizes its training examples instead of learning the pattern.
Overfitting happens when a machine learning model fits its training data too closely. It learns not just the real pattern but also the noise, the quirks and the accidents of those particular examples. It looks brilliant on the data it trained on and disappoints on anything new.
Picture a model that predicts house prices. With a few hundred sales to learn from, a sensible model learns that size and location matter. An overfitted one learns that a house sold high because it had a blue door and was listed on a Tuesday. It gets every past sale exactly right and badly misjudges the next one. The opposite problem, a model too simple to capture even the real pattern, is called underfitting.
The standard way to catch it is to hide part of the data during training and test the model on it afterwards. If the score on the hidden part is much worse than on the training part, the model is overfitting. The usual cures are more and more varied data, a simpler model, stopping training earlier, and techniques grouped under the name regularization, which penalize the model for getting too complicated.
Keep in mind
- The model learns the noise in its examples, not just the pattern.
- Shows up as great training results and poor results on new data.
- Detected by testing on data the model never saw during training.
- Fixed with more data, simpler models, early stopping or regularization.
