Data Science Stack Exchange
Data Science Stack Exchange

Questions tagged [ensemble-learning]

45 questions
5
votes
2 answers

How to apply Stacking cross validation for time-series data?

Normally stacking algorithm uses K-fold cross validation technique to predict oof validation that used for level 2 prediction. In case of time-series data (say stock movement prediction), K-fold cross validation can't be used and time-series…
Manimaran Subramanian
  • 83
  • 5
4
votes
1 answer

Are "Gradient Boosting Machines (GBM)" and GBDT exactly the same thing?

In the category of Gradient Boosting, I find some terms confusing. I'm aware that XGBoost includes some optimization in comparison to conventional Gradient Boosting. But are Gradient Boosting Machines (GBM) and GBDT the same thing? Are they just…
CyberPlayerOne
  • 392
  • 1
  • 4
  • 14
4
votes
1 answer

Geometric and harmonic means in ensembling methods

When using ensembling methods for regression, a common approach is to average (using the arithmetic mean) the outputs of the weak learners in order to obtain the output of the ensemble. Is there a reason why the average that people take is the…
David Masip
  • 5,981
  • 2
  • 23
  • 61
3
votes
0 answers

Stacking when the the target variable is categorical?

I'm trying to use stacking when predicting for the infamous Iris dataset. Also, I'd like to build to stacked classifier by myself which means I don't want to use mlxtend because it's too "easy" and out of the box. I'm using python 3 and working on…
Nathan Furnal
  • 265
  • 1
  • 3
  • 10
2
votes
1 answer

How can I improve my model on a very very small dataset?

I am starting as a PhD student and we want to find appropriate materials (with certain qualities) from basic chemical properties like charge, etc. There are a lot of models and datasets in similar works, but since our work is pretty novel, we have…
Sadegh
  • 23
  • 2
2
votes
1 answer

Neural Network Multiple | Averging predictions

I am training multiple neural networks with various parameters. I am trying to average their predictions, but I am not really sure what that means, I am confused about what to average exactly. Here is what I mean: For a single observation in binary…
U. User
  • 257
  • 1
  • 11
2
votes
1 answer

Why Extra-trees should only be used within ensemble methods?

I was reading scikit-learn documentation for Extremely Randomized Trees and I found this warning: Warning: Extra-trees should only be used within ensemble methods. Why is that?
Abdulrahman Bres
  • 221
  • 2
  • 15
2
votes
2 answers

How to boost the performance of a single decision tree by adding additional trees?

I have a binary classification task and the data has imbalance issue (99% is negative and 1% is positive). I am able to build a decision tree that is carefully tuned, weighted, and post-pruned. Take this as tree1 and it has a high recall and…
szheng
  • 21
  • 2
1
vote
1 answer

How to assign a weight for classifiers when using weighted majority voting?

I am trying to apply weighted majority voting on an ensemble as a combiner method. I read different papers and articles, however, I am still a bit lost on: How the weighted majority voting works How to assign a weight for every ensemble base…
s_am
  • 53
  • 5
1
vote
0 answers

Ensemble of different reservoirs (echo state networks)

Suppose I want to do reservoir computing to classify the input to the proper category (e.g. recognizing a handwritten letter). Ideally, after training a single reservoir and testing it, there would be an output vector y with one value close to 1 and…
Wouter
  • 111
  • 1
1
vote
1 answer

In XGBoost, how is a leaf index corresponding to the particular leaf node in actual base learner trees?

I've trained a XGBoost model for regression, where the max depth is 2. # Create the ensemble ensemble_size = 200 ensemble = xgb.XGBRegressor(n_estimators=ensemble_size, n_jobs=4, max_depth=2, learning_rate=0.1, …
CyberPlayerOne
  • 392
  • 1
  • 4
  • 14
1
vote
0 answers

Stacking - Appropriate base and meta models

When implementing stacking for model building and prediction (For example using sklearn's StackingRegressor function) what is the appropriate choice of models for the base models and final meta model? Should weak/linear models be used as the base…
thereandhere1
  • 715
  • 1
  • 7
  • 22
1
vote
2 answers

What is the difference between ensemble methods and hybrid methods, or is there none?

I have the feeling that these terms often are used as synonyms for one another, however they have the same goal, namely increasing prediction accuracy by combining different algorithms. My question thus is, is there a difference between them? And if…
Tibo Geysen
  • 163
  • 1
  • 3
1
vote
1 answer

What is the form of data used for prediction with generalized stacking ensemble?

I am very confused as to how training data is split and on what data level 0 predictions are made when using generalized stacking. This question is similar to mine, but the answer is not sufficiently clear: How predictions of level 1 models become…
rocksNwaves
  • 309
  • 1
  • 10
1
vote
0 answers

ValueError: Graph disconnected: cannot obtain value for tensor Tensor

I'm trying to perform a stacking ensemble of three VGG-16 models, all custom-trained on my personal dataset and having the same input shape. This is the code: input_shape = (256,256,3) model_input = Input(shape=input_shape) def…
shiva
  • 311
  • 3
  • 6
  • 15
1
2 3