LLM-as-Judge Rubric Prompt With a Stronger Model
When scoring subjective quality across your 20-to-50-example evaluation set, use an LLM-as-judge setup equipped with an explicit rubric and a stronger model than the one being evaluated.
Developer (Model: o3): Evaluate the <candidate_output> generated by gpt-4o-mini against <source_context> using this exact rubric:
1. Factual Grounding (0–2 points): Every number matches <source_context>.
2. Completeness (0–2 points): All requested entities are extracted.
3. Formatting (0–1 point): Valid schema adherence.
Output JSON with rubric_breakdown and total_score.Nobody has confirmed this prompt yet. If you try it, tell others whether it worked.
No comments yet.