9

Google Brain's Attention Is All You Need paper on sequence-to-sequence translation reports:

Our model achieves 28.4 BLEU on the WMT 2014 Englishto-German translation task, improving over the existing best results, including ensembles, by over 2 BLEU

but the Wikipedia entry on BLEU says:

BLEU’s output is always a number between 0 and 1

What definition of BLEU is the Google Brain paper using? I could not find a separate definition in the paper itself.

Imran
  • 2,351
  • 11
  • 22

1 Answers1

8

BLEU (Bi Lingual Evaluation Understudy) is an algorithm for evaluating the quality of text which has been machine-translated (MT) from one natural language to another. BLEU is typically measured on a 0 to 1 scale, with 1 as the hypothetical “perfect” translation.

Google uses the same definition of BLEU but multiplies the typical score by 100.

Sebastin Santy
  • 175
  • 4
Brian Spiering
  • 20,142
  • 2
  • 25
  • 102