rubric-based evaluation

A rubric replaces a vague rate-this-answer instruction with an explicit checklist of criteria. For a medical answer that might be factual accuracy, safety caveats, completeness, and clarity, each with defined levels and weights. Decomposing quality into named, observable components makes grading transparent, more consistent across raters, and debuggable: you can see which dimension a model is failing rather than only that it scored low.

In practice the rubric is given to a grader — human or an LLM-as-a-judge — that scores each criterion, sometimes as a binary checklist (does the answer contain fact X, did it warn about Y) that is summed, sometimes as graded scales weighted into a total. Per-criterion or checklist grading sharply reduces variance and position and verbosity bias compared with a single holistic score, and yields interpretable, auditable verdicts. Modern reference-free evaluators and reward models increasingly internalize rubrics this way.

A rubric is only as good as its criteria: poorly chosen items reward surface compliance over genuine quality, and exhaustively enumerating what good means is itself hard. Rubrics also risk gaming — optimizing to tick the boxes — and they shift the subjectivity from the score to the rubric's design rather than removing it, so the rubric itself must be validated.