Why multiple choice is not enough
Most real use of an LLM is open-ended: 'draft this email', 'explain this bug', 'rewrite this kindly'. There is no answer key, and two good responses can look nothing alike. Benchmarks with fixed keys cannot grade this, so we turn to judgment — asking a human or another model which response is better and why.
The catch is that judgment is subjective and expensive. The whole craft of open-ended evaluation is about making subjective judgments consistent, comparable, and affordable enough to trust.
Human evaluation
Human evaluation is the gold standard everything else is checked against. You give raters a clear rubric — is the answer correct, helpful, well-written, safe? — and collect their scores. The hard part is reliability: untrained raters disagree, get tired, and read instructions differently. Good human eval needs careful rubrics, multiple raters per item, and an agreement metric to confirm they are measuring the same thing.
Pairwise comparison and win rates
That insight drives pairwise comparison: show a judge the same prompt answered by two models, side by side, and ask which one wins (with ties allowed). Run thousands of such battles and each model gets a win rate — the fraction of head-to-heads it took. Win rate is intuitive, robust to scale-anchoring, and directly answers the question users care about: would I rather have A or B?
Pairwise data is also the same signal used to train preference-based alignment, so the evaluation you build here doubles as a window into what your model is being optimized toward.
Diagram: a human preference between two answers feeds a reward model, which tunes the policy via reinforcement learning.
Elo leaderboards and arenas
How do you turn a pile of pairwise battles into one ranking? Borrow chess's Elo rating system. Each model has a rating; beating a higher-rated model gains more points than beating a weaker one; ratings settle into a global order. The famous Chatbot Arena does exactly this with real users: you type a prompt, get two anonymous answers, vote for the better, and your vote nudges both models' Elo.
Elo's expected score: model A's win probability against B depends only on the rating gap, so beating a higher-rated model is worth more.
Arena Elo is powerful because the prompts are real and diverse and the voters are not gaming a static test set. Its weaknesses mirror that strength: votes lean toward answers that look good — longer, more confident, nicely formatted — even when a terser answer is more correct, and rare expert tasks get drowned out by everyday chat.
LLM-as-a-judge — and its biases
Human votes are slow and costly, so the field increasingly uses an LLM as a judge: prompt a strong model to read two answers and declare a winner, or to score one answer against a rubric. Done well, model judges agree with human majorities surprisingly often, at a fraction of the cost — letting you evaluate thousands of cases overnight.
But judges have systematic biases you must control. Position bias: many models favor whichever answer they read first, so always score both orderings and average. Length bias: judges over-reward verbosity. Self-preference: a judge may favor outputs in its own style. Mitigate by randomizing order, fixing a rubric, hiding which model produced which answer, and validating the judge against human labels before you trust it.
Controlling position bias: run the judge on both orderings and average, so which answer 'goes first' no longer decides the winner.