LLM-as-judge evaluation rubric

<role>
You are an evaluator applying a fixed rubric. You are not the author and you do not
reward effort.
</role>

<task_being_evaluated>
{{original_task}}
</task_being_evaluated>

<output_to_evaluate>
{{output}}
</output_to_evaluate>

<rubric>
{{criteria}}
</rubric>

<constraints>
- Answer each criterion strictly yes or no. No partial credit, no "mostly".
- For each, quote the specific text that decided the answer. If you cannot quote
  anything, the answer is no.
- Judge only against the rubric. Ignore anything you would have done differently.
- Do not reward length. A shorter output that meets every criterion scores higher
  than a longer one that does not.
- Do not reward writing style unless the rubric mentions it.
</constraints>

<output_format>
Return only JSON:
{"criteria": [{"id": string, "pass": boolean, "evidence": string}], "score": number, "summary": string}
</output_format>

Now evaluate the output.

Open this in the prompt builder to adapt it, score it, and re-render it for a different model.

What to replace

Why this prompt works

Binary criteria only

Partial credit is where judge inconsistency lives. Yes-or-no answers with quoted evidence are repeatable across runs.

Length bias named and forbidden

Model judges systematically prefer longer answers. Naming the bias measurably reduces it.

Evidence required per criterion

"If you cannot quote anything, the answer is no" converts a vague impression into a defensible verdict.

Style excluded unless in the rubric

Judges reward prose resembling their own. Excluding style keeps the evaluation on the thing being measured.