0

enter image description here

While applying sigmoid activation function (in finding y label), I have calculated it as below:

y = 0.35 + (0.8 * 0.1) + (0.3 * 0.6) + (-0.2 * 0.4) = 0.53

sigmoid_y = 0.625

how do we take threshold value here?

now, how can we say that the given label (y) belongs to 0 or 1?

tovijayak
  • 37
  • 6

1 Answers1

1

This is a design decision.

You can take the naive approach of using 0.5 as the threshold value, and assign 1 to every value greater than or equal to 0.5, and 0 to the rest.

But you can also, choose your threshold to obtain a classifier that meets certain properties, like achieving a certain false positive rate, o a certain false positive rate. You can see these answers for different objectives when choosing the threshold value: this, this, this.

Note that your question is not specific to the sigmoid activation or to neural networks, but applied also to any probabilistic classifier.

noe
  • 22,074
  • 1
  • 43
  • 70
  • yes, my question not specific to 'sigmoid'. if we have balanced data then we can take 0.5 as threshold. but, still am not cleared how we choose this value incase of imbalanced ? how can we select threshold in case of soft max activation function (in balanced , imbalanced)? @noe – tovijayak Jun 20 '23 at 10:51
  • The links I included detail how to choose a threshold value depending on what exactly you what to optimize for, e.g. FPR, FNR. The specific criterion is problem-specific, so one should decide what is more important in their classification problem. This is applicable to both balanced and imbalanced problems. – noe Jun 20 '23 at 10:59
  • Thank you for the notes @noe, how can we choose in case of softmax activation function? – tovijayak Jun 20 '23 at 12:02
  • As I already commented, the fact that you are using softmax is not relevant to the way you choose the threshold. – noe Jun 20 '23 at 12:06
  • ok, Thank you!! will go through the link again.. – tovijayak Jun 20 '23 at 12:09