1

I have a problem. I have trained a model. And as you can see, there is a zigzag in the loss. In addition, the validation loss is increasing. What does this mean if you only look at the training curve? Is there an overfitting?

And the model does not generalise. Accuarcy on test and val is 0.84 and on the test 0.1. Does the assumption confirm the overfitting? And can overfitting come from the fact that I have trained too little? I only used two dense layers.

enter image description here

hH1sG0n3
  • 1,978
  • 7
  • 27
Test
  • 89
  • 7
  • 1
    One or more of your hyperparameter values may be off. Too large learning rate? – noe Oct 27 '22 at 09:35
  • 1
    Overfitting is when there is *too much* training and/or not enough training data. As I said on your [other similar question](https://datascience.stackexchange.com/q/115550/64377), it's not normal that the val accuracy is higher than the training. Please give more detail about your data: size, number of classes and proportion by class. Imho this looks like you're counting that a complex system will compensate bad/small data... if so it won't work, I'm afraid. – Erwan Oct 27 '22 at 10:59
  • Does this answer your question? [Loss is very erratic in the 100s and val\_loss is at 0, something - what is the reason for that?](https://datascience.stackexchange.com/questions/115550/loss-is-very-erratic-in-the-100s-and-val-loss-is-at-0-something-what-is-the-r) – Pluviophile Oct 27 '22 at 13:46

1 Answers1

3

Please notice that your loss oscillates between 175 and zero. In which case I would look for potential problems in the code with respect to

  • loss calculation
  • batch size (increase)
  • train/validation set split strategy (stratification wrt class)

In a more general sense:

  • size of your network may be small
  • activation function saturation (avoid saddle points - use relu)
  • learning rate
  • normalisation before training

I hope these are helpful as a starting point. I would like to also point to this resource wrt training and fine tuning a deep learning model.

Hope this helps!

hH1sG0n3
  • 1,978
  • 7
  • 27