5

You have a neural network. And you have, say, pictures of $100,000$ hand-written letters (A-Z). Now you make a typical Training and the neural network will recognize an A as an A, a B as a B, ...

Now you repeat Training but with a change: you tell the neural network for all pictures with the letter B that its an A too.

Now the questions after training:

  • Does the network recognize A as an A as good as before?
  • Does the network recognize B as an A too?
Green Falcon
  • 13,868
  • 9
  • 55
  • 98
jochen6677
  • 561
  • 2
  • 4
  • 9
  • The question title made me think this was trivial, but now I think it's actually a good question. Consider how it might work with linear models, or gaussian-kernel SVMs; and imagine As and Bs are well-separated, with other letters _between_ them. These models may do very well on the real data, but struggle to classify the As and Bs together! Neural nets are sufficiently less-parametric that I wouldn't expect them to have the same trouble, but I don't think it's obvious that they wouldn't take any performance hit. – Ben Reiniger Jan 08 '20 at 16:44
  • You can try an experiment like this fairly easily by considering two MNIST numbers to have the same category. I’m curious enough that I might try it. If I do (no promises), I’ll post my code and results. – Dave May 09 '20 at 17:16

1 Answers1

1

Does the network recognize A as an A as good as before?

Yes, it can. The reason is that if you try to illustrate the confusion matrix you can see that the FP, FN, TP and TN can be kept at a good level. What you have done is actually changing the distribution of your input data. Suppose that you train a network and it should recognize cats and dogs. You can train it just using white cats or with cats of different colours. The point here is that what you've done is not telling that $A$ and $B$ are of the same class. What you have done is classifying the first two letters as the same entity. Consider that what your network learns depends on the kind of network you use. Avoid using just Dense nets. Having image inputs, Dense nets try to find something like a mask while convolutional layers find appropriate features and those features which are better for classification would then passed to the dense layers.

Does the network recognize B as an A too?

Again yes! But it depends on the distribution of your data. It can recognize $B$s, because you have trained it in that way. But should consider that the performance depends on the data you have. How your data is obtained to be good for generalisation.

Green Falcon
  • 13,868
  • 9
  • 55
  • 98
  • Thank you very much! One thing I do not get, why does it not mean „A and B is the same class“? I tought I exactly do that? I say „A is class A“ and „B is class A“. – jochen6677 Dec 13 '18 at 16:38
  • A and B are not the same in real, you have tried to classify them as a same thing. Your network will consider them as a same thing but you have to provide appropriate data if you want your network classify them correctly. – Green Falcon Dec 13 '18 at 19:00