When Your Reputation Precedes You: How Labels Influence Assessment

When we evaluate skills in simulation, we aim to be objective and fair. But what happens when we already have preconceptions about the trainee's abilities? An interesting study by Tannenbaum et al reveals just how powerfully a resident's reputation can sway assessors' judgment - even when they're all watching exactly the same performance.

The Set-Up: A Clever Deception

The researchers recruited 43 faculty gynecologic surgeons for what they thought was a straightforward evaluation task. Each would watch a video of a resident performing a laparoscopic salpingo-oophorectomy and rate their performance. Simple enough.

But here's the clever bit - all the faculty actually watched the identical video footage. The only difference? Before watching, they were told one of three stories about the resident:

• This is a high-performing third-year resident
• This is an average third-year resident
• This is a struggling third-year resident undergoing remediation

The Results: Labels Matter

The impact was stark. When faculty believed they were watching a "high performer," they awarded significantly higher scores on both technical skills (median 15/20) and global assessment (4/5). The exact same performance by an "average resident" earned middling scores (13/20 and 3/5). But most concerningly, when faculty thought they were watching a "struggling trainee," they gave markedly lower ratings (11/20 and 3/5) to the identical technical display.

The reputation effect was particularly dramatic at the extremes - over 35% of assessors failed the "struggling" resident with scores of 2 or lower, compared to only about 7% for both "average" and "high performing" residents.

Why This Matters

This study exposes an uncomfortable truth about workplace-based assessment: our prior knowledge and assumptions about trainees significantly colour how we perceive their actual performance. This bias works both ways - inflating scores for those with good reputations while potentially trapping struggling residents in cycles of poor evaluations.

For high-performing residents, reputation-based score inflation could mean missing opportunities for improvement. For those branded as struggling, artificially low scores might create unfair barriers to advancement, even when their technical skills actually meet the required standard.

The Way Forward

The researchers suggest several practical approaches to combat this bias:

1. Video-based assessment that removes identifying information
2. Structured scoring rubrics with clear performance criteria
3. Training for assessors to recognize and mitigate their own biases
4. Multiple independent evaluations to balance out individual bias

For candidates preparing for surgical assessments, this research highlights the importance of managing perceptions early in training relationships. First impressions matter - but shouldn't determine destiny.

The Bigger Picture

This study joins a growing body of evidence about cognitive bias in medical education assessment. From gender bias in resident evaluations to the impact of interpersonal relationships on scoring, we're increasingly recognizing that assessment is rarely truly objective.

But recognition is the first step toward improvement. By understanding these biases, we can design better assessment systems that give every trainee a fair chance to demonstrate their true capabilities.

Takeaway: The next time you're assessing a trainee (or being assessed yourself), remember - reputations can be powerful lenses that distort what we think we see. The challenge is to look past the label and evaluate the actual performance in front of us.

References

Tannenbaum, Evan MD, MSc; Walker, Melissa MD, MSc; Sullivan, Heather MD, MPH; Huszti, Ella PhD; Farrugia, Michele MD, Med; Sobel, Mara MD, MSc. Effects of a Resident's Reputation on Laparoscopic Skills Assessment. Obstetrics & Gynecology 138(1):p 16-20, July 2021. | DOI: 10.1097/AOG.0000000000004426