Categorization of approaches to deal with imbalanced classes

Question

What is the best way to categorize the approaches which have been developed to deal with imbalance class problem?

This article categorizes them into:

Preprocessing: includes oversampling, undersampling and hybrid methods,
Cost-sensitive learning: includes direct methods and meta-learning which the latter further divides into thresholding and sampling,
Ensemble techniques: includes cost-sensitive ensembles and data preprocessing in conjunction with ensemble learning.

The second classification:

Data Pre-processing: includes distribution change and weighting the data space. One-class learning is considered as distribution change.
Special-purpose Learning Methods
Prediction Post-processing: includes threshold method and cost-sensitive post-processing
Hybrid Methods:

The third article:

The last classification also considers output adjustment as an independent approach.

Thanks in advance.

The very short answer: all of them are the best and all of them are the worst! Classification and data mining in general are very context sensitive. There is no one size fit all solution in this domain. By the way, the best approach, in very generic terms, is usually a combination of the best decisions at different levels from the feature extraction, to the evaluation scheme. — Mohsen Kamrani, Jun 12 '18 at 23:53
@mok Thanks. Could you please let me know the class-weight in the sklearn's classifiers e.g., logistic regression is classified into which category? — ebrahimi, Jun 13 '18 at 19:06
@ebrahimi, it should fall into algorithm level because only the weights are adjusted according to a passed dictionary or calculated (inferred) according to the values of y (class) and the data remains untouched. — Sanjay Krishna, Jun 18 '18 at 07:07
@SanjayKrishna Thanks a lot. In case of the first categorization, it falls into cost-sensitive learning, doesn't it? Also, in case of the second taxonomy, it would be classified into the third category i.e., cost-sensitive post-processing. is it true? The second answer to this: https://stackoverflow.com/questions/32492550/what-is-the-difference-between-sample-weight-and-class-weight-options-in-scikit is also useful. — ebrahimi, Jun 18 '18 at 07:52

score 5 · Answer 1 · edited Sep 24 '18 at 21:41

The way I see it all three categorizations agree in many things. For example, all three have a category for pre-processing steps.

I would tend to mostly agree on the third categorization as its more generic and encompasses more things.

The data-level category includes any pre-processing steps dealing with class imbalance (e.g. over/under sampling).
The algorithm-level could be considered to include the second categories of the first two articles. Any change to the algorithm that deals with class imbalance would go here (e.g. class weighting).
Finally, a hybrid category for combining the two.

The only thing missing from the first two articles are the post-processing steps, which to be honest, aren't used in practice as often as the other.

1 Answers1