Transcript Physical Examination Education in Graduate Medical

Physical Examination Education in
Graduate Medical Education –
A Systematic Review of the
Literature
•
•
•
•
•
Somnath Mookherjee, MD
Lara Pheatt, MA
Sumant R. Ranji, MD
Calvin Chou, MD, PhD
SDRME Summer Meeting 2012
Importance of PE skills?
Fletcher RH, Fletcher SW.
Has medicine outgrown
physical diagnosis? Ann
Intern Med.
1992;117(9):786-7.
• Flegel KM. Does the
physical examination have
a future? CMAJ.
1999;161(9):1117-8.
• Jauhar S. The demise of
the physical exam. N Engl J
Med. 2006;354(6):548-51.
•
•
•
•
•
Background
CXR from “The Practioner, 1904”
"Laennec examines a consumptive patient with a stethoscope in front of his students at the Necker Hospital". Painting by Théobald Chartran.
Physical diagnosis: a guide to methods of clinical investigation. George Alexander Gibson, William Russell. D. Appleton & Co., 1891. New York.
A pocket book of physical diagnosis: for the student and physician. Edward Tunis Bruen. P. Blakiston, 1881. Philadelphia
Methods
Results
Comment
Discussion
(Un)importance of PE skills?
ACGME Internal Medicine
program requirements
Background
Methods
Results
Comment
Discussion
Resurgence of PE
 Physical examination is still important
 Hypothesis based
 Evidence based
•
•
•
•
Verghese, A., Culture shock--patient as icon, icon as patient. N Engl J Med, 2008. 359(26): p. 2748-51.
Verghese, A., A touch of sense. Health Aff (Millwood), 2009. 28(4): p. 1177-82.
Yudkowsky, R., et al., Residents anticipating, eliciting and interpreting physical findings. Med Educ, 2006. 40(11): p. 1141-2.
Yudkowsky, R., et al., A hypothesis-driven physical examination learning and assessment procedure for medical students: initial validity
evidence. Med Educ, 2009. 43(8): p. 729-40.
• McGee, S., Evidence Based Physical Diagnosis. Second Edition ed2007, St. Louis: Saunders - Elsevier.
• David L. Simel and D. Rennie, The Rational Clinical Examination: Evidence Based Clinical Diagnosis. 2009, New York: McGraw-Hill
Professional.
Background
Methods
Results
Comment
Discussion
But skills are inadequate
Trainee
Type of Exam
Pediatrics
Ausculation
Emergency medicine
Cardiac exam
Emergency medicine
General PE
Internal medicine
General PE
Internal Medicine
Rectal
Internal medicine and Family medicine
Cardiac
Internal medicine and Family medicine
Pulmonary
Internal medicine and Family medicine
MSK
Internal medicine and Family medicine
Breast Exam
Internal medicine and OB / GYN
Pelvis and breast
Family medicine
Pelvic
Surgery
Vascular
Background
Methods
Results
?
References
↓
↓
↓
↓
↓
↓
Gaskin et al. Pediatrics, 2000
Dhuper et al. Clinical Pediatrics, 2007.
↓
↓
↓
↓
↓
↓
Mangione et al. American journal of respiratory and critical care
medicine. 1999.
Jones et al. Academic Emergency Medicine. 1997.
Mangione et al. Academic Emergency Medicine. 1995.
Li et al. Academic Medicine. 1994
Johnson et al. Archives of Internal Medicine. 1986
Ramani et al. JGME. 2010.
Wilt et al. JGIM. 1991.
Vukanovic-Criley et al. Archives of Internal Medicine. 2006.
Mangione et al. JAMA. 1997.
Mangione et al. AJM. 2001.
Vukanovic-Criley et al. Clinical Cardiology. 2010.
St Clair et al. Annals of Internal Medicine. 1992.
Meenan et al. The Journal of rheumatology. 1988.
Chalabian et al. Annals of surgical oncology. 1998.
Dugoff et al. American journal of obstetrics and gynecology. 2003.
Heiligman et al. JAMA. 1998.
Lang et al. F. Family medicine. 1990.
Endean et al. Journal of vascular surgery. 1994.
Comment
Discussion
Why focus on GME?
 Different than students
◦ Little “protected time” for didactics
◦ Constant time pressure
◦ Mainly experiential learning
◦ Skills needed for imminent practice
 New program requirements
Background
Methods
Results
Comment
Discussion
Developmental Milestones for Internal
Medicine Training – Patient Care
Green ML, Aagaard EM, Caverzagie KJ, Chick DA, Holmboe E, Kane G, et al. Charting the road to competence: developmental
milestones for internal medicine residency training. Journal of graduate medical education. 2009;1(1):5-20.
Background
Methods
Results
Comment
Discussion
Research Questions
 What teaching methods are used?
 What assessment methods are used?
 What teaching methods are
effective?
GME Program Curricula
Background
Methods
Results
Individual practices of teachers
Comment
Discussion
Search strategy





Background
Search 1 = "physical examination"[MESH] AND
"Education, Medical"[MESH]
Search 2 = physical examination AND "Education,
Medical"[MESH]
Search 3 = physical examination AND (resident OR intern
OR graduate OR residents OR interns OR graduates) AND
(teaching OR learning OR education OR teach OR learn)
Search 4 = "physical examination"[MESH] AND (resident
OR intern OR graduate OR residents OR interns OR
graduates) AND (teaching OR learning OR education OR
teach OR learn)
Personal files
Methods
Results
Comment
Discussion
Eligibility
1.
2.
3.
4.
5.
6.
7.
Background
English language
Inclusion of a description of the study population;
number of participants and level of training
Description of an educational intervention 
intention of improving PE skills
Inclusion of assessment of efficacy
Inclusion of a clear comparison group
Report of data analysis (descriptions of outcomes
without statistical analysis were not included)
Study subjects enrolled in GME
Methods
Results
Comment
Discussion
Data Extraction

Study Quality
◦ Medical Education Research Study Quality Instrument
(MERSQI)

Based on Best Evidence Medical Education
Collaboration protocol
◦
◦
◦
◦
Nation
Type of physical examination
Level and numbers of learners
Summary of intervention
 Human examinees
 Deliberate practice
◦ Summary of outcomes
 Benefit to learner
 Assessment methods
Background
Methods
Results
Comment
Discussion
Deliberate Practice?
1.
2.
3.
4.
•
•
•
•
Background
Repetitive performance
of skills by the learner.
Assessment of skills by
the teacher.
Specific feedback to the
learner by the teacher.
Observation of
improved performance
in a controlled setting.
• Papers independently scored for the
presence of these elements
• 0 = not reported
• 1 = reported
• Global deliberate practice score
• 0 = no use of deliberate practice or
unable to determine
• 1 = possible use of deliberate
practice
• 2 = definite use of deliberate
practice
Ericsson KA. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Academic medicine : journal of the
Association of American Medical Colleges. 2004;79(10 Suppl):S70-81.
McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Medical education featuring mastery learning with deliberate practice can lead to better health for
individuals and populations. Academic medicine : journal of the Association of American Medical Colleges. 2011;86(11):e8-9.
McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Does simulation-based medical education with deliberate practice yield better results than traditional
clinical education? A meta-analytic comparative review of the evidence. Academic medicine : journal of the Association of American Medical Colleges.
2011;86(6):706-11.
Duvivier RJ, van Dalen J, Muijtjens AM, Moulaert VR, Van der Vleuten CP, Scherpbier AJ. The role of deliberate practice in the acquisition of clinical skills. BMC
medical education. 2011;11(1):101.
Methods
Results
Comment
Discussion
Educational Outcomes

Kirkpatrick Level
◦
◦
◦
◦
◦

Level 0 = no assessment of impact
Level 1 = assessment of reaction to the intervention
Level 2a = assessment of attitudes or perceptions
Level 2b = assessment of knowledge or skills
Level 3 = assessment of changes in behavior
Classification
◦ “X” = not measured, or not compared to a control group
◦ “0” = not better than control group
◦ “1” = beneficial (intervention group with significantly better
outcome than control)
Background
Methods
Results
Comment
Discussion
Statistics and Analysis

Inter-rater reliability
◦ Individual elements of the MERSQI scores
◦ Individual elements of deliberate practice
◦ Global deliberate practice score

Study quality
◦ Average and median MERSQI scores with
standard deviations using consensus scores

Narrative synthesis
◦ Group review of tabulated summaries of
studies
Background
Methods
Results
Comment
Discussion
Search and Selection of Articles
7250 Articles Identified and Screened
1543 "physical examination"[MESH] AND "Education, Medical"[MESH]
1943 physical examination AND "Education, Medical"[MESH]
2038 physical examination AND (resident OR intern OR graduate OR residents OR interns OR graduates)
AND (teaching OR learning OR education OR teach OR learn)
1695 "physical examination"[MESH] AND (resident OR intern OR graduate OR residents OR interns OR
graduates) AND (teaching OR learning OR education OR teach OR learn)
31 Hand search and expert review
7095 Rejected Based on Initial Screening Criteria (is the study
about PE education in GME?)
155 Citations Meeting Initial Screening Criteria
141 Citations Removed
109 Duplicates
32 Not Meeting Full Inclusion Criteria
14 Articles Included in Systematic Review
Background
Methods
Results
Comment
Discussion
Study characteristics
Characteristic, N=14
Nation
 USA
 Denmark
Type of examination
 Cardiac
 Thyroid
 Pelvic
 Breast
 General
 Musculoskeletal
Randomized
Background
Methods
#
Characteristic
Studies
13
1
6
1
3
1
1
2
4
Results
#
Studies
GME (a)
 Pediatrics
 Internal Medicine
 Family Medicine
 Danish “house officer”
Outcomes assessed in
comparison to controls (a, b)
 Kirkpatrick level 1
 Kirkpatrick level 2a
 Kirkpatrick level 2b
 Kirkpatrick level 3
3
8
2
1
3
4
12
2
(a) not mutually exclusive
(b) comparison to control not applicable for level 1 outcomes
Comment
Discussion
Study Quality - MERSQI Score
18
17
• Perfect inter-rater
agreement [kappa = 1.0
(95% CI = 1.0, 1.0)] for all
but two items.
o “Sampling” [kappa =
0.44 (95% CI = -0.16,
1.0)]
o “Content validity”
[kappa = 0 (95% CI = 0.52, 0.52)].
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
Reed DA, Cook DA, Beckman TJ, Levine RB, Kern DE, Wright SM.
0 Association between funding and quality of
published medical education research. JAMA. 2007;298(9):1002-1009.
Background
Methods
Results
Comment
Discussion
Interventions
14 Studies
No resident
interaction w/ human
examinee, n = 7
Resident interaction
w/ a human
examinee, n = 7
Interaction w/ patients
in a clinical context
• Leder 2005: Pelvic exam
during clinical rotation
• Smith 2006: Cardiac exam,
bedside rounds
• Freund 1998: Breast exam,
breast care clinic
• Rabinovitz 1987: Pelvic
exam, adolescent medicine
rotation
Background
Methods
• Criley 2008: Web-based cardiac tutorial
• Horiszny 2001: Multimedia cardiac
lecture
• Iversen 2006: Multimedia cardiac
lecture
• Mangione 1994: PE elective, lectures
• Keren 2005: Brief cardiac audio tutorial
• Oddone 1993: Cardiology simulator
• Smith 2005: MSK small group session
Teaching associates
Patient volunteers
• Branch 1999: Arthritis patient
educator.
• Herbers 2003: Pelvic exam
patient educator
• Houck 2002: Thyroid exam on
volunteer patient in workshop
Single session
Results
Comment
Discussion
Assessments
14 Studies
No
interaction
w/ human
examinee,
n=6
Resident interaction
w/ a human
examinee, n = 8
Objective Structured
Clinical Examinations
• Houck 2002: Thyroid exam
OSCE
• Mangione 1994: Multiple PE
type OSCEs
• Smith 2006: Cardiac exam
OSCE
• Smith 2005: MSK exam OSCE
Background
Methods
Multimedia
• Criley 2008: Computer program to
assess cardiac exam
• Horiszny 2002: Recorded heart sounds
• Keren 2005: Recorded heart sounds
Chart
review
• Freund 1998: Chart review for
frequency of breast exam
• Leder 2005: Chart review for genital
exam
Survey
• Rabinovitz 1987: Survey for pelvic
exam confidence
Teaching associates
Patient Volunteers
• Branch 1999: MSK exam,
patient educators
• Herbers 2003: Pelvic exam,
patient educators
• Oddone 1993: Cardiac exam
testing, also used simulator
• Iversen 2006: Cardiac exam
testing, patient volunteers
Previously published assessment tool
Results
Comment
Discussion
Outcomes
What teaching methods are used?
 What assessment methods are used?
 What teaching methods are effective?

Ineffective
interventions
• Summary
• Summary
Kirkpatrick
Level
Effective
interventions
Background
Methods
Results
Comment
Discussion
Outcomes – Level 3: Behavior
Outcomes – KL 3
Background
Effective interventions
Ineffective interventions
Freund 1998. Breast. Weekly session at
Breast Health Center with observed
examination of patients. Improved
documentation of clinical breast exam on
chart review. (13)
Leder 2005. Pelvic. Precepted exams in clinic. No
difference in completeness of documentation in
chart review of suspected sexual abuse cases. (4)
1. No evident pattern for the superiority
of one educational setting over
another
o Bedside teaching, simulator, lecture,
workshop
Methods
Results
Comment
Discussion
Outcomes – KL 2b
Outcomes – Level 2b: Knowledge / Skills
Effective interventions
Background
Ineffective interventions
•
Houck 2002. Thyroid. Workshop using patient
•
volunteer. Better scores in OSCE describing findings. (2)
•
Herbers 2003. Pelvic. Workshop with gynecological
teaching associates. Improvement in observed
technique and communication. (3)
•
•
Keren 2005. Cardiac. Three minute teaching session
without any practice or feedback. No improvement in
accuracy. (9)
•
Iversen 2006. Cardiac. Workshop with recorded heart
sounds +/- advanced stethoscope. No improvement
in accuracy in diagnosing abnormal heart murmurs.
(10)
•
Houck 2002. Thyroid. Workshop using patient
volunteer. No difference in OSCE in observed
technique. (2)
Branch 1999. MSK. Patients with arthritis trained as
educators. Better with checklist assessment of
examination. (8)
•
Leder 2005. Pelvic. Precepted exams in clinic. No
difference in knowledge score between intervention
and control. (4)
Criley 2008. Cardiac. Web-based multimedia tutorial
with human support. Better performance in cardiac
exam testing. (1)
•
Smith 2006. Cardiac. Two types of bedside teaching
rounds. Intervention groups had better technique, and
one of the intervention groups had mildly better key
findings than control. (5)
2. No evident pattern that spending more
time results in better outcomes
•
•
•
Oddone 1999. Cardiac. High-fidelity simulator
mannequin (Harvey). No better at detecting findings
or making the diagnosis. (6)
o Elective rotation, multiple
series,
•
Mangionelecture
1994. General. Lecture
series. No
difference in PE technique or knowledge scores. (7)
Smith 2005.
MSK. Small group
case-based sessions.single lecture
single
workshop,
Better on OSCE PE checklists and diagnosis. (11)
Horiszny 2001. Cardiac. Multi-media lecture. Better at
identifying heart sounds. (12)
Methods
Results
Comment
Discussion
Outcomes – KL 2b
Outcomes – Level 2b: Knowledge / Skills
Effective interventions
Background
Ineffective interventions
•
Houck 2002. Thyroid. Workshop using patient
•
volunteer. Better scores in OSCE describing findings. (2)
•
Herbers 2003. Pelvic. Workshop with gynecological
teaching associates. Improvement in observed
technique and communication. (3)
•
•
Smith 2006. Cardiac. Two types of bedside teaching
rounds. Intervention groups had better technique, and
one of the intervention groups had mildly better key
findings than control. (5)
Keren 2005. Cardiac. Three minute teaching session
without any practice or feedback. No improvement in
accuracy. (9)
•
Iversen 2006. Cardiac. Workshop with recorded heart
sounds +/- advanced stethoscope. No improvement
in accuracy in diagnosing abnormal heart murmurs.
(10)
•
Houck 2002. Thyroid. Workshop using patient
volunteer. No difference in OSCE in observed
technique. (2)
Branch 1999. MSK. Patients with arthritis trained as
educators. Better with checklist assessment of
examination technique. (8)
•
Leder 2005. Pelvic. Precepted exams in clinic. No
difference in knowledge score between intervention
and control. (4)
Criley 2008. Cardiac. Web-based multimedia tutorial
with human support. Better performance in cardiac
exam testing. (1)
•
Oddone 1999. Cardiac. High-fidelity simulator
mannequin (Harvey). No better at detecting findings
or making the diagnosis. (6)
3. Little clarity or consistency in what PE
competence entails and how to measure
it
•
•
•
o Technique vs. accuracy
•
Mangione 1994. General. Lecture series. No
difference in PE technique or knowledge scores. (7)
Smith 2005.
MSK. Detection
Small group case-based vs.
sessions.
o
diagnosis
Better on OSCE PE checklists and diagnosis. (11)
Horiszny 2001. Cardiac. Multi-media lecture. Better at
identifying heart sounds. (12)
Methods
Results
Comment
Discussion
Deliberate Practice and Outcome
•
•
Inter-rater reliability of
components of DP score
Repetitive performance of skills
by the learner
o
•
[kappa=0.81 (95% CI=0.46,
1.0)]
Global deliberate practice score
o
Background
Possible or definite use of
deliberate practice (Global
DP score = 1 or 2)
Branch 1999 (54)
Criley 2008 (55)
Freund 1998 (56)*
Houck 2002 (59)*
Rabinovitz 1987 (64)
Smith 2006 (65)*
No use of deliberate
practice or unable to
determine (Global DP
score = 0)
Herbers 2003 (57)
Horiszny 2001 (58)
Smith 2005 (66)
[kappa=0.71 (95% CI=0.33,
1.0)]
Observation of improved
performance in a controlled
setting
o
•
[kappa=0.72 (95% CI=0.38,
1.0)]
Specific feedback to the learner
by the teacher
o
•
Intervention group with
better educational outcomes
than control at highest
Kirkpatrick level assessed
[kappa=0.86 (95% CI=0.59,
1.0)]
Assessment of skills by the
teacher
o
•
Categorization of Studies by Use of Deliberate Outcome,
Use of Learner Interaction with Human Examinees and
Efficacy of Educational Intervention
[kappa=0.76 (95% CI=0.46,
1.0)]
Methods
Intervention group and
control group with same
educational outcomes at
highest Kirkpatrick level
assessed
Leder 2005 (61)
Iversen 2006 (67)
Keren 2005 (60)
Mangione 1994 (62)
Oddone 1993 (63)
* Studies scored as “definite use of deliberate practice”
Studies that used learner interaction with human examinees as part of
the educational intervention in bold
Results
Comment
Discussion
Limitations
1.
2.
3.
Background
Deliberate practice assessment nonvalidated
Effect sizes neither calculated nor metaanalyzed
Small number of studies with
heterogeneous outcome measurements
Methods
Results
Comment
Discussion
Findings
1.
2.
Background
No convincing
evidence for one
setting: “going to the
bed-side” vs. “going to
the simulator-side”
Not just the “time
spent,” but the “time
well spent”
Methods
Results
+ DP
- DP
Effective
Branch 1999 (54)
Criley 2008 (55)
Freund 1998 (56)*
Houck 2002 (59)*
Rabinovitz 1987 (64)
Smith 2006 (65)*
Herbers 2003 (57)
Horiszny 2001 (58)
Smith 2005 (66)
Comment
Ineffective
Leder 2005 (61)
Iversen 2006 (67)
Keren 2005 (60)
Mangione 1994 (62)
Oddone 1993 (63)
Discussion
Findings
3.
4.
Background
Deliberate practice is
well-suited to teach
PE in GME
Interaction with
human examinees
may be beneficial
Methods
Results
+ DP
- DP
Effective
Branch 1999 (54)
Criley 2008 (55)
Freund 1998 (56)*
Houck 2002 (59)*
Rabinovitz 1987 (64)
Smith 2006 (65)*
Herbers 2003 (57)
Horiszny 2001 (58)
Smith 2005 (66)
Comment
Ineffective
Leder 2005 (61)
Iversen 2006 (67)
Keren 2005 (60)
Mangione 1994 (62)
Oddone 1993 (63)
Discussion
Recommendations
1.
2.
3.
UME PE education systematic review
underway
Evaluate the use of deliberate practice
and human examinees to teach PE
Develop a GME PE blueprint
A. Specific PE skills graduating residents should
have
B. What competence in these skills entails
C. How these skills are best taught and evaluated
Background
Methods
Results
Comment
Discussion
Acknowledgements
Society of Directors
of Research in
Medical Education
 UCSF Division Of
Hospital Medicine
 UCSF – OME –
Teaching Scholars
Program

Background
Methods
Discussion
Results
Comment
Discussion
Outcomes – KL 1
Outcomes – Level 1: Participation / Reaction
Effective interventions
•
Houck 2002. Thyroid. Workshop using
patient volunteer. Residents felt it was
helpful. (2)
•
Mangione 1994. General. Lecture
series. Lectures helpful. (7)
•
Smith 2005. MSK. Small group sessions.
Participants liked it and thought it was
useful. (11)
Ineffective interventions
Outcomes – Level 2a: Attitudes / Perceptions
Outcomes – KL 2a
Background
Effective interventions
Ineffective interventions
•
Leder 2005. Pelvic. Precepted exams in
clinic. Intervention group had higher
confidence and comfort. (4)
•
•
Rabinovitz 1987. Pelvic. Adolescent
•
medicine rotation. Higher confidence in
pelvic exam. (26)
Methods
Results
Leder 2005. Pelvic. Precepted exams in clinic.
No difference in self assessed competence. (4)
Criley 2008. Cardiac. Web-based multimedia
tutorial with human support. No difference in
confidence. (1)
Comment
Discussion