Data Science Stack Exchange
Data Science Stack Exchange

Questions tagged [ensemble-modeling]

In machine learning, ensemble methods combine multiple algorithms to make a prediction. Bagging, boosting, and stacking, are some examples.

197 questions
81
votes
5 answers

GBM vs XGBOOST? Key differences?

I am trying to understand the key differences between GBM and XGBOOST. I tried to google it, but could not find any good answers explaining the differences between the two algorithms and why xgboost almost always performs better than GBM. What makes…
Aman
  • 977
  • 1
  • 8
  • 8
27
votes
1 answer

Adaboost vs Gradient Boosting

How is AdaBoost different from a Gradient Boosting algorithm since both of them use a Boosting technique? I could not figure out actual difference between these both algorithms from a theory point of view.
CodeMaster GoGo
  • 768
  • 1
  • 6
  • 15
10
votes
2 answers

What does Negative Log Likelihood mean?

I have a data set which has continuous independent variables and a continuous dependent variable. To predict the dependent variable using the independent variables, I've run an ensemble of regression models and tried to compare them against each…
Minu
  • 795
  • 2
  • 8
  • 18
8
votes
1 answer

what is the difference between "fully developed decision trees" and "shallow decision trees"?

As reading Ensemble methods on scikit-learn docs, it says that bagging methods work best with strong and complex models (e.g., fully developed decision trees), in contrast with boosting methods which usually work best with weak models (e.g.,…
Mithril
  • 373
  • 6
  • 15
8
votes
2 answers

Why isn't dimension sampling used with gradient boosting machines (GBM)?

GBMs, like random forests, build each tree on a different sample of the dataset and hence, going by the spirit of ensemble models, produce higher accuracies. However, I have not seen GBM being used with dimension sampling at every split of the tree…
Nitesh
  • 1,615
  • 1
  • 12
  • 22
7
votes
1 answer

Assumptions/Limitations of Random Forest Models

What are the general assumptions of a Random Forest Model? I could not find by searching online. For example, in a linear regression model, limitations/assumptions are: It may not work well when there are non-linear relationship between dependent…
Gaurav Singhal
  • 263
  • 1
  • 3
  • 11
7
votes
5 answers

Does ensemble (bagging, boosting, stacking, etc) always at least increase performance?

Ensembling is getting more and more popular. I understand that there are in general three big fields of ensembling, bagging, boosting and stacking. My question is that does the ensembling always at least increase the performance in practice? I…
WCMC
  • 465
  • 1
  • 4
  • 11
6
votes
1 answer

Is there any difference between a weak learner and a weak classifier?

While reading about decision tree ensembles Gradient Boosting, AdaBoost etc. I have found the following two concepts weak learner and weak classifier. Are they the same? If there is any difference what is it?
Carlos Mougan
  • 6,011
  • 2
  • 15
  • 45
6
votes
3 answers

What is the meaning of the term "pipeline" within data science?

People often refer to pipelines when talking about models, data and even layers in a neural network. What can be meant by a pipeline?
n1k31t4
  • 14,663
  • 2
  • 28
  • 49
6
votes
3 answers

What are the individual models within a machine learning ensemble called?

I am aware that an ensemble machine learning model is a stack of two or more machine learning models. Is there a word to refer to those individual models that go into the ensemble model? (i.e. a specific term/jargon?) I wish to refer to them and am…
stevec
  • 211
  • 1
  • 7
6
votes
0 answers

How to tune weights in Voting Classifier (Sklearn)

I am trying to do the following: vc = VotingClassifier(estimators=[('gbc', GradientBoostingClassifier()), ('rf', RandomForestClassifier()), ('svc', SVC(probability=True))], …
Abhinav Gupta
  • 161
  • 1
  • 3
6
votes
1 answer

How predictions of level 1 models become training set of a new model in stacked generalization.

In stacked generalization, if I understood well, we divide the training set into train/test set. We use train set to train M models, and make predictions on test set. Then we use the predictions as input of a new model. Thus, the new training set…
Spider
  • 1,239
  • 1
  • 12
  • 12
6
votes
3 answers

Is it possible to build ensemble models without a decision tree?

Is it possible to build ensemble models without a decision tree? I know that the description of ensembles itself suggests otherwise. However, I am really new to machine learning and all the ensemble models I came across so far use or are described…
Raveen Diaz
  • 61
  • 1
5
votes
2 answers

Stacking and Ensembling methods in Data Science

I understand that using stacking and ensembling has become popular, and these methods can give better results than using a single algorithm. My question is: What are the reasons, statistical or otherwise, behind the improvement in results? I also…
Donald S
  • 1,889
  • 3
  • 7
  • 28
5
votes
1 answer

2 stage ensemble -- CV MSE valid in 1st stage but not in 2nd

I'm trying out a Kaggle competition, which puts me in the unusual position of being able to get feedback on my models' "true" performance (you can submit several predictions per day and they give you your score on a performance metric -- in this…
Hack-R
  • 1,919
  • 1
  • 21
  • 34
1
2 3
…
13 14