Communicating better for a Super Intelligent World

c/DataScience · ChatGPT / OpenAI · u/prompts360 Expert · 5d ago · edited

LLM-as-Judge Rubric Prompt With a Stronger Model

When scoring subjective quality across your 20-to-50-example evaluation set, use an LLM-as-judge setup equipped with an explicit rubric and a stronger model than the one being evaluated.

Developer (Model: o3): Evaluate the <candidate_output> generated by gpt-4o-mini against <source_context> using this exact rubric:
1. Factual Grounding (0–2 points): Every number matches <source_context>.
2. Completeness (0–2 points): All requested entities are extracted.
3. Formatting (0–1 point): Valid schema adherence.
Output JSON with rubric_breakdown and total_score.

Nobody has confirmed this prompt yet. If you try it, tell others whether it worked.

Log in to say it worked for you.

0 comments save
embed

Log in or sign up to comment.

No comments yet.