0

I want to ask why Label Encoding before train test split is considered data leakage?

From my point of view, it is not. Because, for example, you encode "good" to 2, "neutral" to 1 and "bad" to 0. It will be same for both train and test sets.

So, why do we have to split first and then do label encoding?

Anar
  • 73
  • 4
  • Where does this presumption come from? I don‘t see why - as you described the problem - this should lead to leakage. – Peter Mar 01 '22 at 20:09
  • Imagine that after the split there is no "good" in the training data. If you had done the encoding after the split, then you would have no idea that there can be a "good". There you have your leakage. – noe Mar 01 '22 at 22:30
  • As you mentioned, we have problem when we do encoding after split. So why do you prefer encoding after split? Still didn't get why it is leakage. Can you please give some clarification? Thanks in advance. – Anar Mar 02 '22 at 06:14
  • The problem we encounter when doing the split before encoding is just the real world, where we do not have perfect information about the data that our system will be fed in production. That is why we must evaluate our model on **unseen data**. If you split after encoding, you are evaluating your model under a false premise of knowledge about that very unseen data. – noe Mar 02 '22 at 07:37
  • @Anar I condensed my comments as an answer. – noe Mar 02 '22 at 07:39

2 Answers2

0

If you perform the encoding before the split, it will lead to data leakage (train-test contamination) In the sense, you will introduce new data (integers of Label Encoders) and use it for your models thus it will affect the end predictions results (good validation scores but poor in deployment).

Suppose test data has new class which was not available in train data but you do label encoding it will be available in the model which leads to data leakage

After the train and validation data category already matched up, you can perform fit_transform on the train data, then only transform for the validation data - based on the encoding maps from train data.

Almost all feature engineering like standarisation, Normalisation etc should be done after train testsplit. Hope it helps

Ashwiniku918
  • 1,864
  • 3
  • 17
0

Imagine that after the split there is no "good" in the training data. If you had done the encoding after the split, then you would have no idea that there can be a "good". There you have your leakage.

Of course, as you mention in the comments, this is a problem. Nevertheless, this problem is just the real world, where we do not have perfect information about the data that our system will be fed in production. That is why we must evaluate our model on unseen data. If you split after encoding, you are evaluating your model under a false premise of knowledge about that very unseen data.

noe
  • 22,074
  • 1
  • 43
  • 70