5

I'm looking for a solution to detect different moods/styles expressed by an image. Unfortunately, there is no multi-labeled dataset for this task.

The scenario of defining a multi-label classification model based on single labeled data doesn't seem too absurd to me but still I couldn't find any publications or other sources addressing this problem.

So I'm very thankful for any hint how scenarios like this could be tackled (e.g. deriving a multi-labeled dataset from available single label datasets in a weakly supervised manner).

Neongelb
  • 53
  • 4
  • 2
    My first idea would be to just train it in a supervised single-label manner and then use the top $k$ classes with the highest confidence as your multi-label prediction. – Jonathan Dec 17 '19 at 19:45

3 Answers3

3

I would train it using the loss appropriate for multi-label (sigmoid activation/binary cross-entropy loss instead of softmax activation/categorial cross-entropy loss). The model will give you probabilities per label, so it's up to you to decide how to interpret them in your application.

Having said that, some multi-label problems have mostly single labels, much fewer double labels, and almost no labels beyond that. For problems like this, you could try a single-label model and see if it works better than a multi-label one. Or have an ensemble :)

Andris Birkmanis
  • 154
  • 4
  • That would give you different degrees of a single label. Not sure it could be meaningfull to deal with 'orthogonal' class that could appears in 'moods'. – Lucas Morin Oct 31 '20 at 16:00
3

Two possible approaches:

  1. Treat it as a supervised learning problem by tagging each image with mood/style labels.

  2. Treat it as an unsupervised learning problem by applying topic modeling. Each image would probabilistically belong to a topic. Then label each topic with a mood/style.

Brian Spiering
  • 20,142
  • 2
  • 25
  • 102
1

If you go the route suggested by @andris (the only option you may have), the multi label classes remain arbitrary, with probability cut points chosen by you. This would not seem to add true value

HEITZ
  • 911
  • 4
  • 7