Questioning the Validity of Step 2 CS: What’s the point?

David Wideman

Abstract

My paper discusses the role of requiring credentialing processes for certain jobs, and how exams should be designed when used for that purpose. Then, using supporting literature, it demonstrates how Step 2 CS fails to meet these criteria. Further discussion then focuses on the slowly increasing amount of literature demonstrating a lack of correlation between USMLE exams and residency performance, thus invalidating the only alternative use.

What is credentialing?

Regulates professions through licensure

Serves to protect public interests

Ensures practitioners are capable of fulfilling

minimum basic requirements

USMLE Step 2 CS

”[Step 2 CS] assesses the ability of examinees to apply medical knowledge, skills, and understanding of clinical science essential for the provision of patient care under supervision, and includes emphasis on health promotion and disease prevention”

“[Step 2 CS] ensure[s] that due attention is devoted to the principles of clinical sciences and basic patient centered skills that provide the foundation for the safe and effective practice of medicine”

Kane et. Al – Designing Licensing Exams

Make it an appropriate length There is far too much information to test over all topics

Goal is to assess mastery of ‘critical’ knowledge

Knowing enough to avoid directly harming people (aka P=MD)

Designed to assess minimum competence, NOT to create comparable assessment of knowledge

So…what is Step 2 CS for again?

”[Step 2 CS] assesses the ability of examinees to apply medical

knowledge, skills, and understanding of clinical science essential for the

provision of patient care under supervision, and includes emphasis on

health promotion and disease prevention”

[Step 2 CS] assesses the ability…to apply medical knowledge, skills, and

understanding…essential for the provision of patient care

Survey says:

Miller B, Dewar SB, Nowalk A. – No. Lack of correlation between USMLE scores and performance of pediatrics residency training

based on ACGME milestones ratings.

Violato C, Shen E, Gao H. – Also no. Does Step 3 of the United States Medical Licensing Exam Measure Clinical Competence?

But what about the other thing?

“[Step 2 CS] ensure[s] that due attention is devoted to the principles of

clinical sciences and basic patient centered skills that provide the

foundation for the safe and effective practice of medicine”

Survey says:

Prober CG, Kolars JC, First LR, Melnick DE. – No. A Plea to Reassess the Role of United States Medical Licensing Examination Step 1 Scores in Residency Selection.

Johnston JL, Lundy G, McCullough M, Gormley GJ. - No. The view from over there: reframing the OSCE through the experience of standardized patient raters

Kane MT, Crooks TJ, Cohen AS. – No. Designing and Evaluating Standard-Setting Procedures for Licensure and Certification Tests.

Puscas L, Chang CWD, Lee H-J, Diaz R, Miller R. – No. USMLE and Otolaryngology: Predicting Board Performance

What’s the point?

The USMLE step two CS exam has no useful purpose.

It fails to meet generally accepted standards for a

credentialing examination, and it also fails to serve

as a useful metric of general aptitude for medical

practice.