Audit Systematic Chain-of-Thought Failures in GPT-4.1
OpenAI's GPT-4.1 guide recommends auditing output failures in your evals for three specific failure classes—intent misunderstanding, insufficient context analysis, and incorrect reasoning—and adding targeted reasoning rules for whichever class fails.
Audit the 10 failed RAG responses in <failed_evals>{{DATA}}</failed_evals>. Classify each failure as (A) Intent Misunderstanding, (B) Insufficient Context Analysis, or (C) Incorrect Reasoning, and draft the exact opinionated reasoning rule to add to our GPT-4.1 prompt.Fill in the blanks (1)
Nobody has confirmed this prompt yet. If you try it, tell others whether it worked.
No comments yet.