Medical Licensure in the United States Assessment of Clinical Skills

Ann C. Jobe, MD, MSN Executive Director Clinical Skills Evaluation Collaboration (CSEC)

Raising the Bar in Geriatrics Assessment

2010 Reynolds Grantee 9 th Annual Meeting October 25-27, 2010 St. Louis, Missouri (sample of slides from presentation)

• A Collaborative Partnership, established in 2003, between the Educational Commission for Foreign Medical Graduates (ECFMG) and the National Board of Medical Examiners (NBME)

Clinical Skills Evaluation Collaboration CSEC

A Look Back in Time

• • • USMLE introduced in early 1990’s Single examination pathway for initial medical license (graduates of US and international medical schools) A national standardized series of exams to assure minimal competency

United States Medical Licensing Examination (USMLE)

• • • Each of the three Steps of the USMLE complements the others No Step can stand alone in the assessment of readiness for medical licensure.

United States Medical Licensing Examination (USMLE)

• • Step 1 • understanding and application of important concepts of the sciences basic to the practice of medicine • Multiple choice exam Step 2 • application of medical knowledge, skills, and understanding of clinical science essential for the provision of patient care under supervision • Clinical Knowledge (CK) • Multiple choice exam • Clinical Skills (CS)

United States Medical Licensing Examination (USMLE)

• Step 2CS • • • • Standardized patients used to assess an examinee’s ability to

gather information from patients, perform physical examinations, communicate their findings to patients and colleagues

United States Medical Licensing Examination (USMLE)

• Step 3 • application of medical knowledge and understanding of biomedical and clinical science essential for the

unsupervised practice

of medicine • Multiple choice exam and computerized case simulations (CCS)

Life Cycle of a Physician in the United States

American Board of Medical Specialties (ABMS) Specialty Boards Board Certification Exams Recertification q 7-10 yrs NRMP Match Medical School Year 1 Year 2 Year 3 Year 4 Postgraduate Training 3-7 yrs Fellowship Maintenance of Practice Maintenance of Licensure Step 1 Step 2 CK Step 2 CS (USMLE) Step 3 Licensure Relicensure q 1-3 yrs State Licensing Authorities

Miller’s Pyramid

DOES SHOWS HOW KNOWS HOW KNOWS

Action Performance Competence Knowledge

Kirkpatrick Criteria

4.

3.

2.

1.

Results

Change in organizational practice Benefits to patients/clients

Behavior

Transfer learning to workplace Learners apply new knowledge and skills

Learning

Change attitudes/perceptions Change knowledge/skills

Reaction

Customer satisfaction related to participation in educational activities

COMPETENCY

• “Core Competencies” Accreditation Council for Graduate Medical Education (ACGME) American Board of Medical Specialties (ABMS) • • • • • • Patient Care Knowledge Communication and Interpersonal Skills Professionalism Systems-Based Practice Practice-Based Learning and Improvement

Correlations among Step scores

Step l Step 2 CK Step 2 CS Step 2 CK Step 2 CS Data-gathering Communication/IP skills Spoken English proficiency Patient note Step 3 ~.65

.19

.09

.09

.20

~.50

-------- .26

.16

.13

.30

~.70

---- ---- N/A

USMLE Step 2 Clinical Skills

• Mastery of clinical and communication skills, as well as cognitive skills, by individuals seeking medical licensure is important to the protection of the public. (from USMLE

Bulletin of Information)

Step 2 CS Examinees

YEAR

2009 2008 2007 2006 2005

TOTAL

34,837 35,224 33,832 32,843 31,939

USMGs

18,983 17,711 17,711 17,473 17,671

IMGs

15,854 17,513 16,121 15,132 14,268

Step 2 CS Fees

• USMG – increase of $100(10%) over 6 year period – 2004-2006: $975 – 2007: $1,005 – 2008: $1,025 – 2009: $1,055 – 2010: $1,075 • IMG – 2004-2009: $1,200 – 2010: $1,295

USMLE Step 2 Clinical Skills

• The cases cover common and important situations that a physician is likely to encounter in clinics, doctors’ offices, emergency departments, and hospital settings in the United States.

Presentation categories

• • • Include, but are not limited to, cardiovascular, constitutional, gastrointestinal, genitourinary, musculoskeletal, neurological, psychiatric, respiratory, and women's health. Examinees will see cases from some, but not all, of these categories. The selection of cases is also guided by specifications relating to acuity, age, gender, and type of physical findings presented in each case.

• Each Step 2 CS “session” includes 12 twenty-five minute patient encounters. • 15 minutes with patient • 10 minutes for patient note • The examination lasts approximately 8 hours. Two breaks are provided: • • 1 st 2 nd break is 30 minutes long (lunch) break is 15 minutes long (snack).

Step 2 CS Components

• • • Communication and Interpersonal Skills (CIS) Spoken English Proficiency (SEP) Integrated Clinical Encounter (ICE) • Data gathering (DG) • History & PE • Patient note (PN)

Step 2 CS Components

• USMLE Step 2 CS is a

Pass/Fail

examination •

Each of the three subcomponents (CIS, SEP, ICE) must be passed

in a single administration in order to achieve a passing performance on Step 2 CS

Assessment of Communication and Interpersonal Skills (CIS)

• CIS performance is assessed by the standardized patients • a global rating of these skills using a series of generic rating scales • • same CIS scale for all 12 encounters 3 sub-components: • Information gathering (questioning skills) • • Information sharing Professional manner and rapport

Assessment of Communication and Interpersonal Skills (CIS) Questioning skills - examples include:

•

use of

open-ended questions, transitional statements, facilitating remarks •

avoidance of

leading or multiple questions, repeat questions - unless for clarification, medical terms/jargon unless immediately defined, interruptions when the patient is talking •

accurately summarizing

information from the patient

Assessment of Communication and Interpersonal Skills (CIS) Information-sharing skills - examples include:

•

acknowledging

patient issues/concerns and clearly responding with information •

avoidance of

medical terms/jargon unless immediately defined •

clearly providing

• counseling when appropriate • closure, including statements about what happens next

Assessment of Communication and Interpersonal Skills (CIS) Professional manner and rapport - examples include:

•

asking about

• expectations, feelings, and concerns of the patient • support systems and impact of illness, with attempts to explore these areas •

showing

• consideration for patient comfort during the physical examination • attention to cleanliness through hand washing or use of gloves

Assessment of Communication and Interpersonal Skills (CIS) Professional manner and rapport - examples include:

•

providing

opportunity for the patient to express feelings and/or concerns •

encouraging

additional questions or discussion •

making

• empathetic remarks concerning patient issues/concerns • patient feel comfortable and respected during the encounter

Assessment of Spoken English Proficiency (SEP)

• SEP performance is assessed by the standardized patients • using rating scales; same scale for all 12 encounters • based upon • frequency of pronunciation or word choice errors that affect comprehension • amount of listener effort required to understand the examinee's questions and responses • clarity of spoken English communication within the context of the doctor-patient encounter (e.g., pronunciation, word choice, and minimizing the need to repeat questions or statements)

Scoring of the Step 2 Clinical Skills Subcomponents

• • The ICE subcomponent includes assessment of: Data gathering (DG) - patient information collected by history taking and physical examination Documentation (PN) - completion of a patient note summarizing the findings of the patient encounter, diagnostic impression, and initial patient work-up

Scoring of the Step 2 Clinical Skills Subcomponents

• Data gathering (DG) performance is assessed by the standardized patients • using checklists developed by committees of clinicians and medical school clinical faculty • checklists comprise the essential history and physical examination elements for each specific clinical encounter

Scoring of the Step 2 Clinical Skills Subcomponents

• The patient note is rated/scored by trained physician raters • The patient note (PN) consists of three areas • • • Medical History and Physical Examination Differential Diagnosis Diagnostic Workup

Scoring of the Step 2 Clinical Skills Subcomponents

• Scored holistically • • Relevant and correct information Congruency/consistency with specific case – based scoring guidelines • • Integration/synthesis of information Organization, coherence, cohesiveness, flow, legibility

Performance on Step 2 CS

• Failure rate for USMGs • 2004-2005: 4% • • • • 2005-2006: 2% 2006-2007: 3% 2007-2008: 3% 2008-2009: 3% • • • • Failure rate for IMGs • 2004-2005: 17% • • US Citizens: 11% Foreign Citizens: 18% 2005-2006: 15% 2006-2007: 23% 2007-2008: 28% • 2008-2009: 27%

Performance on Step 2 CS

• Passing rate for USMGs (first takers) • 2005-2006: CIS >99%; SEP >99%; ICE 98% • 2006-2007: CIS 99%; SEP 100%; ICE 97% • 2007-2008: CIS 99%; SEP >99%; ICE 98% • 2008-2009: CIS 99%; SEP >99%; ICE 98%

Performance on Step 2 CS

• Passing rate for IMGs (first takers) • 2005-2006: CIS 93%; SEP 98%; ICE 89% • 2006-2007: CIS 87%; SEP 99%; ICE 85% • 2007-2008: CIS 81%; SEP 92%; ICE 86% • 2008-2009: CIS 84%; SEP 94%; ICE 84%

• For more information about USMLE policies and procedures, and about specific exams – please go to: • www.usmle.org