2

In the book "Deep Learning with Python" by Francois Chollet (2018), in section 1.2.4 one can find:

Decisions trees learned from data began to receive significant research interest in the 2000s, and by 2010 they were often preferred to kernel methods.

...

In particular, the Random Forest algorithm introduced a robust, practical take on decision-tree learning that involves building a large number of specialized decision trees and then ensembling their outputs. Random forests are applicable to a wide range of problems—you could say that they’re almost always the second-best algorithm for any shallow machine-learning task. When the popular machine-learning competition website Kaggle (http://kaggle.com) got started in 2010, random forests quickly became a favorite on the platform—until 2014, when gradient boosting machines took over.

It sounds for me like the author trying to draw this kind of the evolution of classification methods:

| Method            | From | To           |
|-------------------|------|--------------|
| Kernel Methods    | ...  | 2000         |
| Decision Trees    | 2000 | 2010         |
| Random Forest     | 2010 | 2014         |
| Gradient Boosting | 2014 | Today (2019) |

Is Gradient Boosting the most popular library nowadays?

Is it can be universally applicable for any cases?

What do you think?

Community
  • 1
Ruben Kazumov
  • 211
  • 1
  • 4

1 Answers1

1

Gradient Boosting performs well for problems that have a medium amount of tabular data. Many problems on Kaggle are in that style.

Currently, Deep Learning holds many of the state-of-the-art benchmarks. Deep Learning generally performs better as the amount of data grows. Deep Learning performs better on other types of data (images and text).

No single machine learning algorithm is best in all cases, often called the "no free lunch theorem".

Brian Spiering
  • 20,142
  • 2
  • 25
  • 102