Transcript Knows

Measuring Outcomes for Residency
Graduates
Steven L. Frick, MD
Chairman, Dept. of Orthopaedic Surgery
Director of Medical Education, Nemours Children’s Hospital
Professor and Assistant Dean
University of Central Florida College of Medicine
Orlando, Florida
No financial disclosures.
AAOS Program Committee
POSNA Curriculum Committee
POSNA Residents Review
POSNA Treasurer, Board of Directors
JRGOS Board of Directors
ABOS QWTF
ACGME Milestones Project Workgroup
No competency specifically
addressing surgical skills
Macy Foundation Report 2011
The Content and Format of
Graduate Medical Education
Recommendation III-A: The length of GME
should be determined by an individual’s
readiness for independent practicedemonstrated by fulfillment of nationally
endorsed, specialty-specific standards- rather
than tied to a GME program of fixed duration.
“nationally endorsed, specialtyspecific standards”
Do we have any of those?
Role for CORD
 Optimistic versus Pessimistic
 “an opportunity in every difficulty” versus “ a
difficulty in every opportunity”
Evidence Based Medicine
Integrating individual clinical expertise
with the best external clinical evidence
Outcome
A final product or end result
A. Flexner - 1910
Medicine can be learned but not taught
Active participation required
Need dedicated educators and students
“get comfortable with
uncertainty”
Role of professional education
 Provide practitioners the intellectual tools to
assess information critically, stay abreast of
changing knowledge, adapt to continuous
change, and reflect on the larger role and
responsibilities of the profession in society.
 From Time to Heal by Kenneth Ludmerer
“Is there a core body of
knowledge and skills that the
finishing resident should
possess prior to starting
practice or fellowship?”
- Richard Gross, MD
Need curriculum
and
competency assessment
Residency fundamentally =
Master-Apprentice
William Halsted:
Residency Training system
Introduced in 1889 at Johns Hopkins
based on:
 a fixed period of time for training,
 structured educational content,
 actual experience with patients,
 escalating responsibility for patient
care during training, and a period of
supervised practice after formal
training.
 Remains the cornerstone of surgical
training in North America more than a
century later
Competency Based Education
 Defined by the outcome of the educational
process, not the content
 Develop weighted curriculum to teach and
assess (Farmer, Gross, Wadey)
 Assessing competence focuses on what the
learner is able to do
How do you assess competency?
"the state or quality of being
capable or competent; skill;
ability."
Professional authenticity
Miller’s model of competence
Does
Shows how
Knows how
Knows
Miller GE. The assessment of clinical skills/competence/performance.
Academic Medicine (Supplement) 1990; 65: S63-S7.
Performance
or “hands on”
Live Demo;
Multimedia
Read, Listen
OCAP
Dreyfus Model of Skill Acquisition
Mastery Learning Model
-Bloom 1968
Becoming Expert
“The 10,000 Hour Rule”
 About 10 yearsdedicated practice
 Bill Joy- UNIX, Sun
Microsystems;
Mozart; The Beatles;
Bobby Fischer;Bill
Gates
Model of complete clinical care
Opening
Engage
Empathize
Find It
Educate
Enlist
Closing
Fix It
Culturally Competent Care
 The ability to understand and work
with patients whose beliefs, values,
and histories are significantly
different from our own.
Cultural Competence in Health Care
CCC Education
Medical/
Surgical
Team
Concerns
•Quality of Outcomes
•Patient-Physician
Relationship
•Malpractice Claims
Error Prevention
Courtesy of A. White, III, MD
Team
Harmony
& Quality
of Work
Life
Defining / Teaching/ Modeling
Professionalism most
important
Drs. Cruess body of work
Hidden curriculum
Social Contract
Individual Awareness
Teach and Model
Professionalism
Surgery- tripartite body of
knowledge
Frank Wilson, MD
 Preoperative - evaluation, indications, planning
 Intraoperative - technical execution
 Postoperative - immobilization, weight-bearing, PT
 All 3 necessary for success
Ortho Surgical
Education
 Interns - pre and post operative care,
framework of ortho fundamentals, closed
management of fxs
 PGY 2/3 - basic decision-making and
psychomotor skills
 PGY 4/5 - independent decision-making,
subspecialty skills, integrate knowledge
Our Educational philosophy at CMC
 Not training
 Stimulus - Reaction vs
Stimulus - Thought - Reaction
 Create one-on-one master-apprentice situations
 Graduated responsibility
 ALWAYS supervised in highest risk activities (OR)
 Have to spend enough time with them to know
How do you assess competency?
I DON’T KNOW
Charlotte Competency Stages
 Stage I - do not know anything cannot do
anything, and know it
 Stage II – know and can do a lot, but do not
recognize what you do not know and cannot
do DANGER
 Stage III – know and can do a great deal,
but realize there is much you do not know
“The beginning of a mountaineer’s
career, when energy and enthusiasm
outpace experience and judgment, is said
to be the most dangerous part.”
Photo by Guillaume Dargaud
Setting Standards
 Job of Chair and RPD to set standards of
excellence
 Graduates of program should meet these
standards in all core competencies
Assessing competency
 Complete 5 years of orthopaedic surgery
program under watchful eye of PD
 12 months PGY1 / internship, 48 months
orthopaedic surgery
 Evaluations and comment by faculty, peer
evaluations, portfolio (presentations,
courses, outcomes instruments), OITE,
operative experience log
 Consensus of PD and faculty
ABOS
I believe this individual is
capable of the competent
independent practice of
orthopaedic surgery.
Steven L. Frick, MD
Residency Program Director
An Expert- Knows
 Knows WHAT to do
 Knows HOW to do it
 Knows WHEN to do it (and when not to)
 Knows WHY to do it
 Knows WHEN to ask for help
 Knows WHAT we don’t know
Is it possible/desirable to
define and measure
competency
and then
graduate a resident
before 5 years?
A Competency-Based Curriculum
in Orthopaedic Surgery:
From Idea to Implementation
Markku Nousiainen, MS, MD, MEd, FRCS(C)
Sunnybrook Health Sciences Centre
University of Toronto
Current challenges
in residency training

reduction in work hours

reduced time spent in OR
teaching surgical skills

reduction in wait times

improvement in patient safety
}
reduced training
opportunities for
residents
Competency-based education
“Training process that results in proven competency in
the acquisition & application of skills & knowledge to
medical practice that is not simply dependent on the
student’s length of training & clinical experiences”
“Much of what is counted does
not count, and much of what
counts cannot be counted.”
Problems
 Toronto experience- 5 years, now all in for
first time
 Still no defined “curriculum”
 More resource intensive than traditional
pathway = costs more
 Current environment of GME= very
dependent on Medicare funding
 Some predict reduction in Medicare GME
funding under PPACA 2010
How much of residency education is
experiential?
Can we list / define everything you need to
learn?
Can we transfer knowledge gained from
experience without making residents have the
experience?
Duty hours 2003
 First ortho class with 80 hour work weekdouble failure rate on part I ABOS
certification exam
 Similar result 2011 exam takers
 Why?
 Does this exam measure competency?
 Who do you want – 90%ile or 30%ile?
GME-Decade of Accountability
 To patients by residents, faculty
 Patient safety, Resident safety- RPD
 To residents by faculty, institution
 Societal demands for assurance of
competency
 Safe, Effective, Patient centered, Timely,
Efficient, Equitable (IOM)
 Increased requirements by oversight
organizations – RPD time
 Professional, ethical behavior demanded
NASNext Accreditation System
 Coming to Ortho July 2013
 No more site visits, PIFs every 5 years
 Annual “Biopsy” of 4 things
–
–
–
–
Institutional report
Annual survey of residents and faculty
Case logs
Milestones (q 6 mos reports from
Competency Committee)
 Self report every 10 years
Ortho Milestones- 18
cover PC and MK
All have 5 levels
By graduation resident should be level 4
(competent) in all
For peds- septic arthritis and SC humerus
fracture
Surrogates for knowledge in other areas
Milestone- Peds SCH Fracture
NAS
www.acgme-nas.org
 Institutional reviews (q 18 mos)
 Milestones reports (q 6 mos)- form a
competency committee
 Operative experience database
 Resident annual survey
 Faculty annual survey (new)
 ORTHO JULY 1, 2013
Future of
Orthopaedic Residency Education
 Change is coming
 Need to protect experience, in addition to
more rigorous evaluation / oversight
 More evaluations / structured experiences
 Remember importance of graduated
independence
 Milestones will be modified as we go
 NAS is on the way- BE AN OPTIMIST!
Thank You