Calibrate Interviewers on the Same Bar
When to use it
Get a panel rating the same answer the same way, before their scores go into a debrief and get treated as comparable.
The prompt
Help me run a calibration exercise so this panel applies the same bar. THE ROLE: {{Job Title}}, {{level}} THE COMPETENCY WE'RE CALIBRATING: {{pick one — e.g. "system design", "stakeholder influence", "code quality"}} OUR RATING SCALE: {{paste it, or describe it}} THE PANEL: {{who, their roles, how often they interview}} WHAT'S GONE WRONG BEFORE: {{e.g. one interviewer never gives above a 3, scores cluster in the middle, seniority disagreements}} Give me: 1. Three sample candidate answers for this competency — one clearly strong, one genuinely borderline, one weak but confident and well-presented. Write them as realistic transcript excerpts, not summaries. 2. The rating each should get and the reasoning, so the panel can check their own calls against it. 3. What distinguishes the borderline answer from the strong one, stated as observable evidence rather than impression. 4. The specific ways interviewers diverge on this competency: the harsh scorer, the middle-clusterer, the one who rewards confident delivery, the one who rewards their own background. 5. How to run the session in 30 minutes — the order, and the question to ask when two people rate the same answer differently. 6. One thing to write into the scorecard afterwards so the calibration survives past this meeting. Do not make the borderline example secretly easy. It should genuinely split a reasonable panel.
Tip
Calibrate on the borderline answer, never the obvious ones. Everybody agrees about the strong candidate and the weak one — the entire disagreement in your debriefs lives in the middle.
How to use it
Paste the prompt into ChatGPT, Claude or whichever assistant you use, then replace every {{bracketed}} part with your own detail. The more specific and messier your input, the better the output — a model given raw notes has more to work with than one given a tidy summary you wrote first.
More Assessment & Scorecards prompts
Open the full tool for all 87 prompts, the glossary, and the JD decoder.