2

I am training multiple neural networks with various parameters. I am trying to average their predictions, but I am not really sure what that means, I am confused about what to average exactly. Here is what I mean: For a single observation in binary classification for example, the final node will give p a value between 0 and 1 (or -1 and 1 if you're using hyperbolic tangent Activation Function), then this p will be rounded to 1 or 0 if it's > 0.5, depending on your decision boundary.

Now, here is what I don't understand, should average p1, p2 and p3 produced by the models before rounding, or I should round the values to True/False responses and then compute the average? and how does that work exactly ?

U. User
  • 257
  • 1
  • 11
  • Can you specify the overview of the problem you are trying to solve... Ie. Binary classification, multi class classification, multi label classification Looks you are trying for Ensembling of various networks. – rajeshkumargp Oct 07 '19 at 17:52

1 Answers1

1

There are multiple ways you could do. All of them are categorized under Ensemble methods in machine learning.

Voting classifiers: which is the simplest way. You just take votes based on the label from all models and uses the majority label. That means, you should first round up all labels to 0 or 1 and then use the majority.

Weighted voting classifiers: similar to previous one, but some models have higher weights in voting.

For more information look here.

aminrd
  • 340
  • 3
  • 12
  • So what happens if I don't round up the values and used some majority function? I am just interested on the reasoning behind it if that's possible. – U. User Oct 07 '19 at 17:56
  • The reason is, if you have 100 models all are making a prediction between 0 and 1. In some cases, they might behave randomly and if you computes the average, the average may always converge to 0.5. – aminrd Oct 07 '19 at 17:59
  • God I've been working on this project for months now and nothing made sense really because I wasn't rounding :/ – U. User Oct 07 '19 at 18:44