Developing High Quality Clinical Skills Assessments University of North Carolina – Chapel Hill School of Medicine November 10, 2011 Ann Jobe, MD, MSN Clinical Skills Evaluation.
Download ReportTranscript Developing High Quality Clinical Skills Assessments University of North Carolina – Chapel Hill School of Medicine November 10, 2011 Ann Jobe, MD, MSN Clinical Skills Evaluation.
Developing High Quality Clinical Skills Assessments University of North Carolina – Chapel Hill School of Medicine November 10, 2011 Ann Jobe, MD, MSN Clinical Skills Evaluation Collaboration Philadelphia, Pennsylvania The most important consideration in developing a clinical skills assessment is to keep in focus the purpose of the assessment Formative or Summative? Formative: to provide feedback for improvement of performance, usually midway through a course of study. Summative: to distinguish between those who are competent, and those who aren’t, usually at the end of a course of study. Test Design Begin at the beginning… • What are you trying to assess? • What is the level of the examinee? • How much time will the examinee have? Can they complete the task in the time allotted? Test Design • What are you trying to teach and assess? Objectives for assessment – history taking skills, PE skills, communication skills, data interpretation skills? • What is the level of the examinee? Focus and length of checklists will be different for 1st year medical students and individuals seeking board certification USMLE Step 2 Clinical Skills Purpose is to assess ability to: Gather information from patients Correctly perform physical examination maneuvers Synthesize and communicate findings to patients and colleagues Examinees are moving to supervised patient care (PGY-1) Development Of An Examination Blueprint A test blueprint defines the requirements for each examination, regardless of where or when it is administered. Commonly seen cases. Blueprint (continued) Identify the criteria used to define an exam blueprint • Examination length • Case content • Examinee tasks (e.g. history, physical, communication) • Setting: inpatient or outpatient • Patient gender, age Case Selection Documentation of commonly seen cases • if specialty focused assessment, cases will be those common to that specialty Most common presenting complaints in clinical setting Health Department population statistics – especially for region, state or local area Case Pool Size Dependent on blueprint criteria Security concerns (more is better!) Case Content Cardiovascular Respiratory Gastrointestinal Musculoskeletal Constitutional Neurological Psychiatric Genitourinary Women’s health Other Case Acuity Acute Subacute/Chronic Form Patient age Age less than 18 Age 18 – 44 Age 45 – 64 Age 65 + Patient Gender Male Female Blueprint Criteria Define Case Needs This information is provided to faculty to guide their case development • Case content: Gastrointestinal • Case acuity: Acute • Patient age: 18-44 • Patient gender: Female Complete a “Medically Relevant Case Details” sheet Case Development Committee Physicians (medical school faculty): content experts SP trainers: training experts SPs: portrayal experts Process evolved from experience Usually break into several groups Each group focuses on developing several cases to fit blueprint needs Case Development Process Chief Complaint “Medically Relevant” case details Patient Personality Profile Case checklist Examinee instructions Patient note development Chief Complaint/ Differential Diagnoses Brainstorming process List all of the possibilities Then narrow the choices as development proceeds Medically Relevant Case Details Patient /Case Name Race • Any • African American • Caucasian • Other _________________ BMI • Any • Weight proportionate to height • Overweight: > 24 • Other Medically Relevant Case Details Gender • Male Female Either Age Range • 10 year age range – example 30 years old = 2535 Acuity • Acute • Subacute/Chronic Category (Blueprint) Exclusionary Medical conditions/scars Differential Diagnosis Medically Relevant Case Details Communication Tasks • • • • • Fostering the Relationship Gathering Information Providing Information Making Decisions Supporting Emotions Overview of Case Stimuli for Communication Tasks Thread of Encounter Primary MD Personality Profile Giving life and individuality to the patients who are being seen – more like “real patients” Not verbatim statements from a script – more general ideas of each patient’s perspective Personality Profile Who am I and how would I describe myself? Why am I here? What made me come in today? What do I think is going on and why? What are my expectations for the visit and how will I react if my expectations are not met? Personality Profile What questions do I have? What concerns/anxiety/fears do I have? How has this illness impacted my life? What do I bring with me? What do I look like? How do I behave/what’s my communication style (mood/attitude related to my illness)? Personality Profile What is my level of trust of the medical profession? What type of communication style do I appreciate in a health care provider? How will I respond to different styles of communication? How much information do I want the doctor to provide? Personality Profile What words won’t I understand (jargon)? What are my own words I use to describe what is going on? How motivated am I to accept medical advice? How do I feel about the use of complementary/alternative medicine? Case Checklist Recording instrument Checklist length One concept per item Evidence-based items Use lay language Example: Onset (“I’ve been coughing for about a month.”) Refining the Checklist Brainstorm a large number of items Role play Checklist revised • A – essential • B – important • C – relevant • F - delete Examinee Instructions Name Age Setting Presentation problem Vital signs Examinee Tasks • Refers to the skills that the examination assesses Role Play with SP A physician from another group “sees” the patient Following the encounter, the physician provides his/her differential diagnoses (see if it matches what the group selected): face validity Revise the checklist based on observing the physician Role Play with SP This allows an SP to provide feedback on any difficulties that arise in portrayal of the case – especially if an SP needs to repeat the case several times in an assessment session Development of Patient Note Key essentials and case-based scoring guidelines Questions?