Data Science Stack Exchange
Data Science Stack Exchange

Questions tagged [overfitting]

Modeling error (especially sampling error) instead of replicable and informative relationships among variables improves model fit statistics, but reduces parsimony, and worsens explanatory and predictive validity.

Models that involve complex polynomial functions or too many independent variables may fit particular samples' covariance structures overly well, such that some existing (and any potential, additional) terms increase model fit by modeling sampling error, not systematic covariance that is likely to replicate or represent theoretically useful relationships. When used to predict other data (e.g., future outcomes, out-of-sample data), overfitting increases prediction error.

The Wikipedia page offers illustrations, lists of potential solutions, and special treatment of the topic as it relates to machine learning. See also:

Leinweber, D. J. (2007). Stupid data miner tricks: Overfitting the S&P 500. The Journal of Investing, 16(1), 15–22. Available online, URL: http://www.finanzaonline.com/forum/attachments/econometria-e-modelli-di-trading-operativo/903701d1213616349-variazione-della-vix-e-rendimento-dello-s-p500-dataminejune_2000.pdf. Accessed January 6, 2014.

Tetko, I. V., Livingstone, D. J., & Luik, A. I. (1995). Neural network studies. 1. Comparison of overfitting and overtraining. J. Chem. Inf. Comput. Sci. 35(5), 826–833. doi:10.1021/ci00027a006.

364 questions
41
votes
8 answers

What would I prefer - an over-fitted model or a less accurate model?

Let's say we have two models trained. And let's say we are looking for good accuracy. The first has an accuracy of 100% on training set and 84% on test set. Clearly over-fitted. The second has an accuracy of 83% on training set and 83% on test set.…
EitanT
  • 519
  • 4
  • 3
28
votes
2 answers

When should one use L1, L2 regularization instead of dropout layer, given that both serve same purpose of reducing overfitting?

In Keras, there are 2 methods to reduce over-fitting. L1,L2 regularization or dropout layer. What are some situations to use L1,L2 regularization instead of dropout layer? What are some situations when dropout layer is better?
user781486
  • 1,305
  • 2
  • 16
  • 18
20
votes
2 answers

Can the number of epochs influence overfitting?

I am using a convolution neural network ,CNN. At a specific epoch, I only save the best CNN model weights based on improved validation accuracy over previous epochs. Does increasing the number of epochs also increase over-fitting for CNNs and deep…
user121
  • 369
  • 1
  • 3
  • 9
17
votes
5 answers

High model accuracy vs very low validation accuarcy

I'm building a sentiment analysis program in python using Keras Sequential model for deep learning my data is 20,000 tweets: positive tweets: 9152 tweets negative tweets: 10849 tweets I wrote a sequential model script to make the binary…
Amy.Dj
  • 173
  • 1
  • 1
  • 5
16
votes
3 answers

Overfitting in Linear Regression

I'm just getting started with machine learning and I have trouble understanding how overfitting can happen in a linear regression model. Considering we use only 2 feature variables to train a model, how can a flat plane possibly be overfitted to a…
Sachin Krishna
  • 199
  • 1
  • 1
  • 7
15
votes
2 answers

Can overfitting occur even with validation loss still dropping?

I have a convolutional + LSTM model in Keras, similar to this (ref 1), that I am using for a Kaggle contest. Architecture is shown below. I have trained it on my labeled set of 11000 samples (two classes, initial prevalence is ~9:1, so I upsampled…
DeusXMachina
  • 263
  • 1
  • 2
  • 6
13
votes
1 answer

How to know if a model is overfitting or underfitting by looking at graph

Just recently got my hands on tensorboard, but can you tell me what features should I look for in the graph (Accuracy and Validation Accuracy) And please do enlighten me about the concept of underfitting as well.
Nikhil.Nixel
  • 319
  • 1
  • 2
  • 9
13
votes
2 answers

Validation vs. test vs. training accuracy. Which one should I compare for claiming overfit?

I have read on the several answers here and on the Internet that cross-validation helps to indicate that if the model will generalize well or not and about overfitting. But I am confused that which two accuracies/errors amoung…
A.B
  • 316
  • 1
  • 3
  • 12
12
votes
2 answers

Is a 100% model accuracy on out-of-sample data overfitting?

I have just completed the machine learning for R course on cognitiveclass.ai and have begun experimenting with randomforests. I have made a model by using the "randomForest" library in R. The model classifies by two classes, good, and bad. I know…
Milan van Dijck
  • 123
  • 1
  • 6
12
votes
4 answers

How to know the model has started overfitting?

I hope the following excerpts will provide an insight into what my question is going to be. These are from here. The learning then gradually slows down. Finally, at around epoch 280 the classification accuracy pretty much stops improving. Later…
figs_and_nuts
  • 775
  • 1
  • 4
  • 13
10
votes
5 answers

In which epoch should i stop the training to avoid overfitting

I'm working on an age estimation project trying to classify a given face in a predefined age range. For that purpose I'm training a deep NN using the keras library. The accuracy for the training and the validation sets is shown in the graph…
Yiannis Ath
  • 188
  • 1
  • 1
  • 10
9
votes
1 answer

Why k-fold cross validation (CV) overfits? Or why discrepancy occurs between CV and test set?

Recently, I was working on a project and found my cross-validation error rate very low, but the testing set error rate very high. This might indicate that my model is overfitting. Why does my cross-validation not overfit while my test set…
user7117436
  • 298
  • 4
  • 11
8
votes
1 answer

Who invented the concept of over-fitting?

I list the references that I found so far. Shortly, the first appearance of the term was in 1670, first appearance in in close meaning was in 1827, first appearance in a biological paper was in 1923 and first appearance in statistics was in…
DaL
  • 2,623
  • 11
  • 13
8
votes
2 answers

Can a linear regression model without polynomial features overfit?

I've read in some articles on the internet that linear regression can overfit. However is that possible when we are not using polynomial features? We are just plotting a line trough the data points when we have one feature or a plane when we have…
Tim von Känel
  • 361
  • 1
  • 10
8
votes
2 answers

Is over fitting okay if test accuracy is high enough?

I am trying to build a binary classifier. I have tried deep neural networks with various different structures and parameters and I was not able to get anything better than Train set accuracy : 0.70102 Test set accuracy : 0.70001 Then I tried…
skrrrt
  • 304
  • 2
  • 13
1
2 3
…
24 25