ROUGE
/ ROOZH /
ROUGE is the summarization world's counterpart to BLEU: an automatic score for how close a machine-written summary is to a human-written reference summary. When you generate summaries for thousands of articles, you can't have people grade them all, so ROUGE gives a fast, repeatable number to compare versions and track progress. The acronym unpacks to Recall-Oriented Understudy for Gisting Evaluation — "gisting" being the act of capturing the gist.
Where BLEU leans toward precision (of the machine's words, how many appear in the reference), ROUGE leans toward recall (of the reference's words, how many the machine captured) — which fits summarization, where the worry is leaving out something important. The common variants are simple to picture: ROUGE-N counts overlapping n-grams, and ROUGE-L rewards the longest run of words appearing in the same order in both texts, a rough measure of shared sentence flow.
Be clear-eyed about what it measures: word overlap with one particular human summary, not faithfulness or readability. A summary that paraphrases well but uses different words is unfairly punished; one that copies the reference's wording while missing the point is over-rewarded. And ROUGE cannot catch a hallucinated fact — a confidently wrong summary can still score well if it shares enough words. ROUGE is a handy steering tool during development, but final quality still calls for a human read.
Reference summary: "The bill passed after a long debate." Machine summary: "After much debate, the bill was passed." ROUGE-L sees a long shared word-run and scores high, even though the wording is rearranged.
ROUGE-L credits the longest run of words shared, in order, by both summaries.
ROUGE leans on recall (did you keep the key words?) and measures overlap, not faithfulness — it cannot flag a hallucinated fact. A high ROUGE summary can still be wrong.