1

I have built nine meta models based on the model stacking principle, which I compare to a reference model for a number of time series. See the results below. The 22 base models that are trained on 70% of the training data produce forecasts on the last 30% of the training data on which the meta models are trained. These are then validated on the test set (last 20% of all data).

The Lin. SVR model's hyperparameters are set as follows, with other hyperparams. set to their default values:
C=0.1, fit_intercept=False, loss='squared_epsilon_insensitive', dual=False

I am surprised at how the Linear Support Vector Regression (SVR) models are greatly outperformed by the Linear Regression (LR) models. I don't know how this phenomenon can be explained, as the pitfalls of SVR discussed online are long runtimes (solved by using a linear kernel) and that the problem might not be linearly solved. The latter appears unlikely given how the LR model performs much better. results

Looking at the actual forecasts of the Lin. SVR model, it is evident that there is a strong bias while the pattern of the target value appears captured moderately well. enter image description here

What could explain the behaviour of these SVR models?

Tim Stack
  • 121
  • 3
  • Welcome to DataScienceSE. I don't see the "SVR with hyper-parameter tuning" in the graph? is it better than the basic SVR? If yes then this would explain it. In general my guess would be that there's something wrong with the way SVR is used or configured. – Erwan Jun 14 '22 at 15:39
  • Hi @Erwan, the two SVR models are almost identical in performance and therefore partially overlap in the graph. Hyperparameter tuning therefore seems to have little effect. Any thoughts what could cause this? – Tim Stack Jun 14 '22 at 21:14
  • I don't really know, it's strange especially when basic LR works well. Maybe lack of scaling the features? Or a problem with the scaling? SVR can be sensitive to this, but I doubt it would have this effect. – Erwan Jun 14 '22 at 21:36
  • The data seen by all models is the same, and has been standardised using `sklearn`'s `scale()` function – Tim Stack Jun 15 '22 at 09:37
  • @Erwan I have amended my post with forecasts of the Lin. SVR model. These forecasts don't seem terrible, were it not for the fact that they seem to have a strong consistent negative bias. Would this warrant a new question? – Tim Stack Jun 15 '22 at 09:49
  • Indeed, the constant bias is curious, and it's likely the cause of the low performance. Yes, I think it's worth asking a specific question. – Erwan Jun 15 '22 at 14:29

0 Answers0