2

Aleatoric uncertainty refers to the notion of randomness that there is in the outcome of an experiment that is due to inherently random effects.

Epistemic uncertainty refers to the ignorance of the decision-maker, due to for example lack of data.

Aleatoric uncertainty is irreducible, while epistemic can be mitigated (Adding more data).

When we deploy a ML model in production. Can we distinguish between epistemic and aleatoric uncertainy?

rubengavidia0x
  • 267
  • 2
  • 15
Carlos Mougan
  • 6,011
  • 2
  • 15
  • 45
  • one way to quantify aleatoric uncertainty is as average uncertainty over various models, since then model mismatch will average out leaving only irreducible uncertainty – Nikos M. Jul 29 '21 at 17:36

1 Answers1

1

In Bias-Variance tradeoff theorem, aleatoric uncertainty is represented by the irreducible error (inherently and irreducibly random). The rest represents model mismatch due to imprecise knowledge of the generation of the problem.

One way to quantify aleatoric uncertainty is as average uncertainty over various models for the same problem, as then uncertainty due to model mismatch will tend to average out leaving only the irreducible uncertainty.

Nikos M.
  • 2,301
  • 1
  • 6
  • 11
  • That will be a way to measure uncertainty. But you can not say which one it is. Aggregating different models (similar than bagging) will allow you to calculate uncertainty in general. – Carlos Mougan Jul 29 '21 at 21:25
  • Well, theoretically the average over all possible models should provide the irreducible uncertainty. In practice it can at least provide an upper bound. – Nikos M. Jul 29 '21 at 22:05
  • If you dont have data about the current weather in France, and they ask you about the weather in France, even if you average several models, you will still have epistemic uncertainty. – Carlos Mougan Mar 13 '22 at 18:22