Vertex AI Prompt Regression Testing With Versions
Save and version proven prompts in code alongside the exact model version they were tested on, and judge any prompt change against a fixed evaluation set rather than a single good answer.
<evaluation_check> Evaluate <candidate_prompt_outputs> against our fixed 30-case regression suite in <fixed_cases>. Flag any regression in schema validity or factual grounding compared to <baseline_outputs>. </evaluation_check>Nobody has confirmed this prompt yet. If you try it, tell others whether it worked.
No comments yet.