1

It is given that:

MSE = bias$^2$ + variance

I can see the mathematical relationship between MSE, bias, and variance. However, how do we understand the mathematical intuition of bias and variance for classification problems (we can't have MSE for classification tasks)?

I would like some help with the intuition and in understanding the mathematical basis for bias and variance for classification problems.

Any formula or derivation would be helpful.

Ben Reiniger
  • 11,094
  • 3
  • 16
  • 53
IamTheRealFord
  • 81
  • 7
  • I don't fully understand the question, what are you looking for exactly? – Djib2011 Jun 19 '19 at 13:32
  • oops sorry. Updated in the question itself. What to know mathematical intuition of bias variance for classification problem. Fore regression it has relation with MSE but classification how to relate them.? – IamTheRealFord Jun 21 '19 at 08:51
  • WHAT classification? Logit? – Peter Jun 21 '19 at 20:01
  • If you are looking for the concept, see https://datascience.stackexchange.com/questions/53758/math-behind-mse-bias2-variance and deeplearningbook. – Fatemeh Asgarinejad Jun 25 '19 at 06:43
  • 1
    ya already gone through that. But how will it work for classification problem.? (we dont have mse there know) – IamTheRealFord Jun 25 '19 at 07:17
  • I don't see why this was closed; the question seems pretty clear to me (after the June 25 edit anyway). MSE has a well-known bias-variance decomposition, so what about other (especially classification) losses? This doesn't depend on the specific model used. For a starting point, see https://stats.stackexchange.com/questions/393942/bias-variance-decomposition-for-non-squared-loss , but I haven't found a satisfactory answer for, e.g., log-loss. – Ben Reiniger Jun 29 '19 at 13:57
  • yes i am puzzled why my question is put on hold :( – IamTheRealFord Jun 30 '19 at 14:17
  • Why can’t we have MSE for a classification task? – Dave Jan 08 '22 at 05:49
  • My opinion is that the bias variance trade off is rooted in the Uncertainty principle. It behaves absolutely the same. – Eugen Jun 25 '19 at 11:04
  • yes. I am currently reading this to decompose bias-variance for general loss function. http://www-bcf.usc.edu/~gareth/research/bv.pdf.. Also searching(both intuition and mathematically) why decreasing bias increases variance and vice versa! – IamTheRealFord Jun 25 '19 at 11:13

1 Answers1

0

Bias and Variance in Classification problems

Check this link about Support Vector Machine.

You will directly understand bias and variance in classification. Basically, if your data is linearly separable you do not have a problem.

But imagine that your data is pseudo/semi linearly separable, however, few points land on the other side of their group.

Now imagine having a model that separates the data linearly, vs a model that will oscillate through the data so much to be able to classify correctly every point.

biasvariance

Additional link

ombk
  • 101
  • 2
  • This answer addresses the mathematically imprecise "bias-variance _tradeoff_", but not the mathematically precise "bias-variance _decomposition_" (which is well-known for MSE but not, to my knowledge, for other losses including common classification ones). – Ben Reiniger Sep 12 '22 at 01:31