1

I have a multi-class dataset and I generated based on it rules. That is, if certain features are seen then it must be a certain class. I chose only rules with precision 1 (with respect to the whole dataset)

It is worth mentioning the dataset is highly imbalance. The major class has 60K samples and other classes can have around 1K samples.

Now, consider a rule that applies to, let's say, 17 endpoints. My question is, which test should I apply to check if I can be confident about this rule? I guess the size of the class will have an effect on it.

greenButMellow
  • 131
  • 2

1 Answers1

1

The confidence interval is the main indicator to test confidence, and it is measurable thanks to the data volume indeed, but also whether or not the endpoints are reliable enough.

Therefore, testing data is necessary at each endpoint or set of endpoints: The percentage of success from a specific confidence interval is defined either manually thanks to the known business limitations or automatically through a Gaussian function.

Nicolas Martin
  • 4,509
  • 1
  • 6
  • 15