5

I was wondering if there's a good way to use ensembling when I have two or more algoritims producing ranked lists. That is, suppose I have the following datasets consisting of ordered lists (higher to the top means more relevant):

Method1_Rankings  Method2_Rankings GoldStandard_Rankings
item1             item2             item1
item3             item1             item3
item2             item10            item5
...

Is there a way to optimally combine methods 1 and 2 (e.g., give the rankings some weights or similar)? Thank you.

user3490622
  • 151
  • 2
  • 2
    Good question! This problem has been studied; cf. e.g., [Learning to Blend Rankings](http://www.yichang-cs.com/yahoo/cikm10_blending.pdf), [Reciprocal rank fusion outperforms condorcet and individual rank learning methods](https://plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf), and [Schulze method](https://en.wikipedia.org/wiki/Schulze_method). More simply, if you have attendant scores associated with the items in each list, you could average the scores. Welcome to the site. – Emre Apr 15 '18 at 00:19

1 Answers1

0

In this case, you might want to convert these into pair-wise relationships (e.g. item1 < item3), put together what you get from the different methods, and find a ranking which agrees with them the most.

You can look at the answer in the comment for some ideas - I would also suggest another paper that might help you: Aggregating inconsistent information: ranking and clustering

anymous.asker
  • 844
  • 4
  • 10