BLEU
/ BLOO /
BLEU is a number, from 0 to 1 (often shown as 0 to 100), that grades how close a machine translation is to one or more human translations of the same sentence. The problem it solves is practical: you can't have a person judge every output when a system translates millions of sentences, so BLEU offers a fast, automatic stand-in. The name is a wink — Bilingual Evaluation Understudy — an "understudy" stepping in for the human judge.
The way it works is to check overlap in short word-chunks, the n-grams. How many of the machine's single words, word-pairs, and word-triples also appear in the human reference? More overlap, higher score. It also adds a brevity penalty so a system can't cheat by emitting a tiny, ultra-safe output. The result is a single comparable number that you can track as you tweak a system — the reason BLEU became the standard yardstick for machine translation for two decades.
Here is the honest part: BLEU measures surface word overlap, not meaning. A perfectly good translation that uses different but equally correct words scores low; a clumsy translation that happens to reuse the reference's words scores high. It rewards matching phrasing, not being right. So BLEU is useful for comparing systems on the same test set and watching trends, but a single BLEU number tells you little about real quality — which is why human evaluation never fully went away.
Reference: "The cat is on the mat." Machine A: "The cat sits on the mat" — high BLEU (lots of shared words). Machine B: "A feline rests upon the rug" — a fine translation, but near-zero BLEU because it shares almost no words.
BLEU rewards matching the reference's words — not getting the meaning right.
BLEU scores word overlap, not meaning, so a correct paraphrase can score low and an awkward word-match can score high. Compare systems on the same test set; never read a lone BLEU number as a verdict on quality.