Transcript Neurosurgical Training

The Effectiveness of
Competency-Based Education
Stephen R. Smith, M.D.
Associate Dean for Medical Education
Brown Medical School
Courtesy of Ronald Harden
Courtesy of Ronald Harden
Courtesy of Ronald Harden
Courtesy of Ronald Harden
Calls for Reform
•
•
•
•
•
AMA Future Directions in Med Ed (1982)
GPEP Report (1984)
Macy Conference Report (1989)
Bloom (1988)
Robert Wood Johnson Foundation Program
in Medical Education (1989)
Flexnerian Model

Define
“Fundamental
Knowledge”
Teach the
Fundamentals

Test for
Knowledge of
Fundamentals

Hope for
the Best
?
Competency-Based Model
Develop
Learning
Experiences
Design
Measures and
Standards of
Performance
Define the
Successful
Graduate
Brown’s 9 Abilities
Effective Communication
Basic Clinical Skills
Using Basic Science in the Practice of Medicine
Diagnosis, Management, and Prevention
Lifelong Learning
Professional Development and Personal Growth
Social and Community Context of Medicine
Moral Reasoning and Clinical Ethics
Problem Solving
ACGME Outcome Project
General Competencies for Residents
•
•
•
•
•
•
Patient Care
Medical Knowledge
Practice-based learning and improvement
Interpersonal and communication skills
Professionalism
Systems-based practice
The Scottish Doctor
• What the doctor is able to do
–
–
–
–
–
–
–
Clinical skills
Practical procedures
Patient investigations
Patient management
Health promotion and disease prevention
Communication
Medical informatics
The Scottish Doctor (2)
• How the doctor approaches practice
– Attitudes, ethical understanding, and legal
responsibilities
– Decision-making skills and clinical reasoning
and judgement
– Basic, social and clinical sciences and
underlying principles
The Scottish Doctor (3)
• The doctor as a professional
– Role of the doctor within the health care system
– Personal development
Definition
Lifelong Learning
The competent graduate is aware of the limits of
his/her personal knowledge and experience
and has an intellectual interest in general
education and medical science. The graduate
actively sets clear learning goals, pursues
them, and applies the knowledge gained to the
practice of his/her profession.
Criteria
1.Recognizes personal limits in knowledge
and experience
• initiates steps to rectify gaps in knowledge
• researches, reviews, and extracts data from
multiple sources
• confers with knowledgeable colleagues,
advisors, or experts
Levels
• Beginner: demonstrate proficiency in a
structured setting (e.g., medical school
class)
• Intermediate: demonstrate proficiency in a
clinical or research setting
• Advanced: submission for publication or
acceptance of abstract, poster, or oral
presentation at an academic meeting
Performance-based Assessment
•
•
•
•
Authentic
Increasing level of challenge
Formative feedback
P=KxA
QuickTime™ and a
DV/DVCPRO - NTSC decompressor
are needed to see this picture.
Evidence of Effectiveness
• Carraccio et al. (2002) reviewed 469 articles
• “revealed little scientific evidence
evaluating the outcomes of competencybased education”
• “What evidence that does exist clearly
favors competency-based education over
the current structure- and process-based
model”
Clinical Skills
A competency-based curriculum
introduced into three medical schools in
China rapidly improved the clinical skills
of students compared to the traditional
curriculum.
From Stillman PL. Teaching and assessing clinical skills: a competency-based programme in
China. Med. Educ. 1997; 31:33–40.
Neurosurgical Training
40
35
30
25
20
Traditional
CBE
15
10
5
0
Laminectomy
Spinal fusion
Pterional
craniotomy
Suboccipital
craniotomy
From Long DM. Competency-based Residency Training: The Next Advance in
Graduate Medical Education. Acad. Med. 2000; 75:1178–1183.
Epidemiological Proof
•
•
•
•
•
•
Strength of the Association
Dose–Response Relationship
Consistency of the Association
Temporally Correct Association
Specificity of the Association
Coherence with Existing Information
(Biological Plausibility)
Educational Proof
• Theory and Principle (Educational
Plausibility)
• Presence of Desired Effects
• Absence of Adverse Effects
• Qualitative Assessment
• Near-term Quantitative Assessment
“The only way to get somewhere, you know, is
to figure out where you’re going before you go
there.”
——John Updike, Rabbit Run, 1960
“It is a highly questionable practice to label
someone as having achieved a goal when you
don’t even know what you would take as
evidence of achievement.”
——R. F. Mager, Preparing Educational
Objectives, 1962
Desired Effects
• Changes in learning behavior in desired
directions (e.g., clinical skills)
• More explicit guidance for students
• More authentic and relevant teaching and
assessment
Avoid Adverse Effects
• No diminution in USMLE/NBME scores
• Theoretical criticisms not realized
–
–
–
–
Competencies not representative of tasks
Reductionistic, trivializing, decontextualized
Devalues knowledge
Constrains teaching and creativity
• Not too expensive or resource intensive
Qualitative Assessment
• Concept valued by students
• Initial skepticism by faculty, but willingness
to go along
• Credibility gap created when only lip
service paid to competencies
• Resistance to jumping through more hoops
• Submerged into the culture
Ability VII Fulfillment
80
Involved in
service learning
70
Percent
60
50
Earned >2
credits
40
30
20
Participating in
>1
service–learning
organization
10
0
2001
2002
2003
Year of Graduation
2004
Independent Studies
80
72.5
70
60
50
40
30
20
10
0
33.7
Brown
U.S.
Evaluation Approaches
• Near Term
– Qualitative assessments
– Intermediate goals
• Long Range
– Comprehensive studies (à la Tamblyn)
– Long-term goals