I am working on my multi-class classification project and I have a question:
I have three classes in proportion: 50%, 47% and 3%. I decided to use class_weight="balanced" parameter in random forest classifier. Now I want to calculate accuracy. Should I use balanced accuracy or can I use common accuracy?
Accuracy is by definition "weighted" according to the classes proportions: it counts the number of correct instances in the full dataset, so a class with N% of the instances represents N% of the accuracy score. In other words, the 50% class accounts for 50% of the score, but the 3% class represents only 3%.
This is why in general accuracy should not be used to measure performance with a imbalanced dataset. This question about taking imbalance into account in the performance might also be of interest.
