1

My understanding of boosting is just training models sequentially and learning from its previous mistakes.

Can boosting algorithms be built with bunch of logistic regression? or logistic regression + decision trees?

If yes, I would like to know some papers or books that covers this topic in-depth.

haneulkim
  • 385
  • 2
  • 11

1 Answers1

2

Boosting is not limited to tree-based models. Find some more information here:

P. Bühlmann, T. Hothorn (2007), "Boosting Algorithms: Regularization, Prediction and Model Fitting", Statistical Science 22(4), p. 477-505.

I implemented L2 linear regression boosting from Section 3.3 (p. 483) from the paper above in this R-code. You may replace the L2 model by a logit model and see how it works.

Peter
  • 7,277
  • 5
  • 18
  • 47
  • Perfect! Will look into it, thanks! – haneulkim Sep 08 '21 at 10:27
  • 1
    @haneulkim BTW: the idea of combining estimators in a boosting estimator sounds interesting. IMO this would require averaging or stacking. I do not know if there is reasearch on this issue and/or if it brings an additional advantage. Would be happy to know if you gain some insights on this. Cheers… – Peter Sep 08 '21 at 21:27