0

In https://www.kaggle.com/c/santander-product-recommendation/data it mentions that

Please note: This sample does not include any real Santander Spain customers, and thus it is not representative of Spain's customer base.

What are the ways where the Santander can anonymize their customers yet the solutions by Kaggle can be useful for them?

william007
  • 755
  • 1
  • 9
  • 20

2 Answers2

1

If a model predicts useful information for a class of customers, maybe customers over 50, or those with more than 1000EUR, then that's useful even without knowing who the individuals in the model are.

The actual data doesn't seem to be anonymous data though, it is implied to be synthetic data or possibly from another bank altogether ("does not include any real Santander Spain customers").

Spacedman
  • 1,982
  • 11
  • 16
1

I think data masking technique is what you are looking for.

The main reason for applying masking to a data field is to protect data that is classified as personal identifiable data, personal sensitive data or commercially sensitive data, however the data must remain usable for the purposes of undertaking valid test cycles.

Take your santander problem for example, there is an age feature in the original dataset. As we all know, $1\leqslant \text{age}\leqslant 200$ (nobody survives 201 years right?). If we do something like $\text{new_age}:=(\ln{(\pi* \text{age}+\sqrt{2})})^2$, new_age is kind of "encrypted" and no one knows the actual age of John Doe.

References

https://en.wikipedia.org/wiki/Data_masking

Icyblade
  • 4,246
  • 1
  • 22
  • 34
  • Thansk, but $\text{new_age}:=(\ln{(\pi* \text{age}+\sqrt{2})})^2$ could make the age mean and medium looks pretty strange? – william007 Feb 07 '17 at 04:23
  • Yes, the distribution can be strange, which might impact our model. However, in real world, most variables don't obey "good" distributions([normal distribution](https://en.wikipedia.org/wiki/Normal_distribution#Central_limit_theorem) etc), and we can't assume that our data has good statistics all the time. That's why we have data preprocessing techniques([BoxCox](https://docs.scipy.org/doc/scipy-0.15.1/reference/generated/scipy.stats.boxcox.html) transform etc). – Icyblade Feb 07 '17 at 05:33