0

I would like to train my datasets in scikit-learn but export the final Gradient Boosting Regressor elsewhere so that I can make predictions directly on another platform.

I am aware that we can obtain the individual decision trees used by the regressor by accessing regressor.estimators[].tree_. What I would like to know is how to fit these decision trees together to make the final regression predictor.

Ethan
  • 1,625
  • 8
  • 23
  • 39
Chong Lip Phang
  • 221
  • 2
  • 8
  • You can directly save the model to disk and load it on the other platform. Why you want to do this complex exercise. Please explain a bit more in detail – 10xAI Dec 23 '20 at 17:34
  • By 'another platform' I mean a software environment other than Python. It is MQL5 used for forex trading. – Chong Lip Phang Dec 23 '20 at 22:52

1 Answers1

2

There are two estimators i.e. The initial predictor and the sub-estimators

init_estimator
The estimator that provides the initial predictions. Set via the init argument or loss.init_estimator.
estimators_
ndarray of DecisionTreeRegressor of shape (n_estimators, 1)
The collection of fitted sub-estimators.

Prediction after the first (i.e. init) estimator is controlled by the learning rate.

You can get the prediction as done in the below code -

trees = model.estimators_

x  = x_test.iloc[10,:].values # A sample X to be predicted
y_pred = model.init_.predict(x.reshape(1, -1)) # prediction from init estimator

for tree in trees:
    pred = tree[0].predict(x.reshape(1, -1)) # prediction from sub-estimator

    y_pred = y_pred + model.learning_rate*pred  # Summing with LR
y_pred
Ethan
  • 1,625
  • 8
  • 23
  • 39
10xAI
  • 5,454
  • 2
  • 8
  • 24
  • Interesting. I will test it out tomorrow. So we just use the initial sample repeatedly without having to worry about deviation from y, ie. the gradient? – Chong Lip Phang Dec 24 '20 at 10:30
  • Not initial sample, it's the "x" to be predicted e.g. x_test. Trees are already fitted. – 10xAI Dec 24 '20 at 11:53
  • Great. The figures matched precisely! – Chong Lip Phang Dec 25 '20 at 03:32
  • You seem to be knowledgeable in this field. I have some other questions. Could you please take a look? https://datascience.stackexchange.com/questions/87122/how-to-obtain-the-final-values-of-a-decisiontreeregressor-in-scikit-learn https://datascience.stackexchange.com/questions/87121/what-quantile-is-used-for-the-initial-dummyregressor-for-gradient-boosting-regre – Chong Lip Phang Dec 25 '20 at 07:33