Subtopic Deep Dive
Situational Judgment Tests in Medical Selection
Research Guide
What is Situational Judgment Tests in Medical Selection?
Situational Judgment Tests (SJTs) in medical selection are assessments using realistic scenarios to evaluate non-cognitive skills like professionalism, teamwork, and interpersonal competence in medical school applicants.
SJTs supplement cognitive tests by measuring procedural knowledge of effective behaviors in medical contexts. Research spans test development, predictive validity for academic success and job performance, and fairness across demographics. Over 20 papers from 2005-2020, with foundational works by Lievens et al. exceeding 150 citations each.
Why It Matters
SJTs improve medical admissions by predicting clinical performance beyond grades, as shown in Lievens and Sackett (2011) with 261 citations linking video-based SJTs to academic and job outcomes. They reduce bias in selection, with Lievens et al. (2005, 163 citations) demonstrating operational validity when matching predictor domains to criteria like patient interactions. Patterson et al. (2016) confirmed SJT predictive power for postgraduate training, enhancing patient-centered care through better interpersonal skill identification.
Key Research Challenges
Matching Predictor-Criterion Domains
SJTs must align scenario constructs with medical outcomes like teamwork in clinical settings. Lievens et al. (2005, 163 citations) showed mismatches reduce validity in college admissions. This requires tailored video-based designs for procedural knowledge.
Retest Effects in Selection
Repeated SJT administration causes score inflation, distinguishing within-person gains from between-person differences. Lievens et al. (2005, 127 citations) developed a framework tested in operational settings. Mitigating this preserves fairness in high-stakes medical admissions.
Incremental Validity Over Cognitive Tests
SJTs add value beyond aptitude scores, but evidence varies by criterion. Lievens and Sackett (2011, 261 citations) validated interpersonal SJTs for success prediction. Gardner and Dunkin (2017) found weak support for some traits in resident selection.
Essential Papers
The validity of interpersonal skills assessment via situational judgment tests for predicting academic success and job performance.
Filip Lievens, Paul R. Sackett · 2011 · Journal of Applied Psychology · 261 citations
This study provides conceptual and empirical arguments why an assessment of applicants' procedural knowledge about interpersonal behavior via a video-based situational judgment test might be valid ...
Why do medical graduates choose rural careers?
John Henry, Brian Edwards, Brendan Crotty · 2009 · Rural and Remote Health · 168 citations
Based on the literature review and interviews, 11 strategies are suggested to increase the number of graduates choosing a career in rural medicine, and one strategy for maintaining practitioners in...
Overview: what’s worked and what hasn’t as a guide towards predictive admissions tool development
Eric Siu, Harold Reiter · 2009 · Advances in Health Sciences Education · 166 citations
The Operational Validity of a Video-Based Situational Judgment Test for Medical College Admissions: Illustrating the Importance of Matching Predictor and Criterion Construct Domains.
Filip Lievens, Tine Buyse, Paul R. Sackett · 2005 · Journal of Applied Psychology · 163 citations
This study is part of a trend of examining noncognitive predictors, for example, a situational judgment test (SJT), as supplements to cognitive predictors for making college admission decisions. Th...
RETEST EFFECTS IN OPERATIONAL SELECTION SETTINGS: DEVELOPMENT AND TEST OF A FRAMEWORK
Filip Lievens, Tine Buyse, Paul R. Sackett · 2005 · Personnel Psychology · 127 citations
This study proposes a framework for examining the effects of retaking tests in operational selection settings. A central feature of this framework is the distinction between within‐person and betwe...
The UKCAT-12 study: educational attainment, aptitude test performance, demographic and socio-economic contextual factors as predictors of first year outcome in a cross-sectional collaborative study of 12 UK medical schools
I. C. McManus, Chris Dewberry, Sandra Nicholson et al. · 2013 · BMC Medicine · 112 citations
Belonging, Respectful Inclusion, and Diversity in Medical Education
Laura Weiss Roberts · 2020 · Academic Medicine · 103 citations
Belonging is the experience of being accepted, included, and valued by others. A fundamental human motivation, belonging positively influences an individual's health, abilities, relationships, and ...
Reading Guide
Foundational Papers
Start with Lievens and Sackett (2011, 261 citations) for interpersonal validity concepts; Lievens et al. (2005, 163 citations) for operational SJT evidence in admissions; Lievens et al. (2005, 127 citations) for retest frameworks.
Recent Advances
Study Patterson et al. (2016) for postgraduate SJT validity; Gardner and Dunkin (2017) for resident selection evidence; Roberts (2020) for inclusion in assessments.
Core Methods
Core techniques: video-based scenarios for procedural knowledge (Lievens et al., 2005); criterion-related validity over multiple academic years; frameworks distinguishing retest effects.
How PapersFlow Helps You Research Situational Judgment Tests in Medical Selection
Discover & Search
Research Agent uses searchPapers and citationGraph to map SJT literature from Lievens and Sackett (2011), revealing 261 citations and clusters on validity; exaSearch uncovers fairness studies, while findSimilarPapers links to Patterson et al. (2016) for postgraduate applications.
Analyze & Verify
Analysis Agent applies readPaperContent to extract validity correlations from Lievens et al. (2005); verifyResponse with CoVe checks claims against abstracts; runPythonAnalysis computes meta-analytic effect sizes from citation data using pandas, with GRADE grading for predictive evidence strength.
Synthesize & Write
Synthesis Agent detects gaps like underrepresented diversity in SJTs via Roberts (2020); Writing Agent uses latexEditText for admissions tool sections, latexSyncCitations for Lievens papers, latexCompile for reports, and exportMermaid for validity framework diagrams.
Use Cases
"Compare retest effects in SJT medical admissions across studies"
Research Agent → searchPapers('retest SJT medical') → Analysis Agent → runPythonAnalysis(pandas meta-regression on effect sizes) → GRADE report on within- vs between-person effects from Lievens et al. (2005).
"Draft LaTeX review on SJT validity for med school selection"
Synthesis Agent → gap detection in Lievens/Sackett papers → Writing Agent → latexEditText(structure review) → latexSyncCitations(10 papers) → latexCompile(PDF with tables).
"Find code for SJT scoring models in admissions research"
Research Agent → paperExtractUrls(Lievens papers) → Code Discovery → paperFindGithubRepo(SJT validation scripts) → githubRepoInspect(analyze Python models for procedural knowledge scoring).
Automated Workflows
Deep Research workflow conducts systematic review of 50+ SJT papers: searchPapers → citationGraph → DeepScan(7-step validity analysis with CoVe checkpoints). Theorizer generates hypotheses on SJT fairness from Lievens et al. clusters, outputting Mermaid theory diagrams. DeepScan verifies incremental validity claims across McManus et al. (2013) and Patterson et al. (2016).
Frequently Asked Questions
What defines Situational Judgment Tests in medical selection?
SJTs present realistic medical scenarios to assess non-cognitive skills like teamwork via response rankings. Video-based formats evaluate procedural knowledge, as in Lievens and Sackett (2011).
What are key methods in SJT research?
Methods include video scenarios for interpersonal skills (Lievens et al., 2005), predictive validity modeling against academic criteria, and retest frameworks (Lievens et al., 2005). Fairness analyses address demographic biases.
What are foundational papers?
Lievens and Sackett (2011, 261 citations) on interpersonal validity; Lievens et al. (2005, 163 citations) on operational validity; Lievens et al. (2005, 127 citations) on retest effects.
What open problems exist?
Challenges include weak personality links (Gardner and Dunkin, 2017), domain matching, and diversity integration (Roberts, 2020). Incremental validity needs longitudinal medical outcome data.
Research Medical Education and Admissions with AI
PapersFlow provides specialized AI tools for Medicine researchers. Here are the most relevant for this topic:
Systematic Review
AI-powered evidence synthesis with documented search strategies
AI Literature Review
Automate paper discovery and synthesis across 474M+ papers
Find Disagreement
Discover conflicting findings and counter-evidence
Paper Summarizer
Get structured summaries of any paper in seconds
See how researchers in Health & Medicine use PapersFlow
Field-specific workflows, example queries, and use cases.
Start Researching Situational Judgment Tests in Medical Selection with AI
Search 474M+ papers, run AI-powered literature reviews, and write with integrated citations — all in one workspace.
See how PapersFlow works for Medicine researchers
Part of the Medical Education and Admissions Research Guide