How can I prevent this model to learn more(less) :)))

Question

As you can see, GradientBoostingClassifier overfit with more training example. These are my parameter for the model:

{'learning_rate': 0.1, 'loss': 'deviance', 'max_depth': 6, 'max_features': 0.3, 'min_samples_leaf': 80, 'n_estimators': 300}

What should I to make my model better or stop training at 350?

score 3 · Answer 1 · answered Aug 04 '18 at 00:31

Well, by making your model better, I suppose you mean enforcing some sort of regularization on it, so that it doesn't overfit. Early stopping, however can be viewed as a regularization method by itself.

I would suggest sampling the performance of your model more frequently (than the 200 examples you are doing currently) and if you see the validation score is not improving after a few iterations stop the training.

Early stopping can be implemented through the monitor parameter in the GradientBoostingClassifier's .fit() method.

How I should add regularization to a GradientBoostingClassifier? I thought regularization reduce the complexity of a model, how it help here? — parvij, Nov 03 '18 at 05:29

How can I prevent this model to learn more(less) :)))

1 Answers1