0

I have applied data scaling techniques on my training dataset during training.

For evaluation, when scaling the test dataset, should it be scaled using the scalers fitted to the training dataset or the test dataset?

NIM4
  • 1
  • 1
  • 5
    This previous post answers probably your question: https://datascience.stackexchange.com/questions/39932/feature-scaling-both-training-and-test-data – Nicolas Martin Jun 30 '21 at 07:29
  • 1
    The point here as both Erwan and Nicolas have mentioned is that **only the training set is supposed to be known**, so feature transformations are derived **only from the train set**. Then in production the input data are transformed by the transformation derived from the train set. This includes the test dataset which is a simulation of a dataset used in production – Nikos M. Jun 30 '21 at 12:15
  • 1
    This fact that **only the training set is supposed to be known** to us, and all feature transformations are this derived from it only, has as consequence that in order to derive valid transformations the training set has to be representative of the underlying distribution. Thus the way the training set is collected should be such that it covers all cases in the percent that they occur – Nikos M. Jun 30 '21 at 12:28
  • thank you, makes sense – NIM4 Jul 02 '21 at 01:41

0 Answers0