Parallel Data preprocessing

Question

I am looking for a suggestion. Is it possible to implement the data preprocessing steps like missing value imputation, outlier detection, normalization, label encoding in parallel? Can I implement cuda/openmp/mpi programming for data preprocessing?

Thank you.

score 2 · Accepted Answer · answered Sep 08 '22 at 15:14

2

Yes - there are a lot of approaches. Depending on the language you are using / packages.

Assuming Python:

Multiprocessing: Dask, pool.map, modin, pandarallel, spark
GPU: CuDF from RAPIDS
Multi-GPU: Cudf-Dask

If you have a Nvidia GPU - I would highly recommend the RAPIDs framework, they have plotting, machine learning, dataframes etc...

answered Sep 08 '22 at 15:14

GooJ

435
2
11

What if I am using C or C++. Please let me know. – Encipher Sep 08 '22 at 19:19
@Encipher If I were you I'd ask a C/C++ specific question on stack overflow - perhaps with an example. – GooJ Sep 08 '22 at 19:36

Parallel Data preprocessing

1 Answers1