Transcript Development and Evaluation of an Objective and Simulation-Based Core Curriculum for Surgery Residents Orlando C.

Development and Evaluation of an
Objective and Simulation-Based
Core Curriculum
for Surgery Residents
Orlando C. Kirton, M.D., F.A.C.S., F.C.C.M., F.C.C.P.
Ludwig J. Pyrtek, M.D. Chair in Surgery
Director of Surgery, Chief Division of General Surgery
Hartford Hospital
Professor of Surgery
Program Director, Integrated General Surgery Residency Program
Vice Chair, Department of Surgery
University of Connecticut School of Medicine
Essential ACGME Requirement
• Create a knowledge-based and clinical
curriculum based on six educational
competencies:
- Medical Knowledge
- Patient Care
- Interpersonal and Communication Skills
- Professionalism
- Practice-Based Learning and Improvement
- Systems-Based Practice
The Perfect Storm faced by
Program Directors
• Restricted resident duty hours
– I.e.;80 hours; PGY 1- no more than 16hrs
• Falling reimbursements for hospitals and
faculty
• Less non-clinical time available to the
Teaching Faculty
• An increasing # of educational requisites
– The ACGME milestone project, robust
simulation curriculum, patient safety and
quality objectives
The University of Connecticut
Integrated General Surgery Residency
Program
• 1 of 8 general surgery residency training
programs in Connecticut
• Integration of 3 separate training programs in
1990
• 46 Residents ; 6 chiefs
• 5 Hospital integrated program
• 110 teaching Faculty
– 60% private practice
The Residents’ Concerns
with the Curriculum
•
•
•
•
•
•
•
•
Program Director (October 2009)
Diminished available time for residents
Less non-clinical time available to teaching faculty
Resident dissatisfaction with the didactic curriculum
Regurgitation from textbooks
Lack of audience interaction
Session taught by resident lacked depth and scope
Inadequate simulation exposure
Inadequate preparation time (learner & presenter)
Methods
• A core curriculum steering committee of residents and
faculty:
– Program Director
– Incoming 2 Administrative Chief Residents
– A peer–elected categorical resident representative
(PGY 1-3)
– A faculty member from each teaching hospital
• A simulation curriculum steering committee also created
– Similar composition + simulation coordinator and
education specialist
• Starts meeting in February; Curriculum completed by
May
The Core Curriculum
• Focuses on specific goals and objectives
– 2 year bundled curriculum of 85 standing (annual) and 25 biannual learning topics
• Utilizes online teaching materials
– SCORE Modules, ACS Fundamentals of SurgeryTM Curriculum
– Various on-line texts : e.g.; Schwartz, Sabiston, etc
• Interactive lecture format
– An audience response system
• Compulsory simulation curriculum
• SCORE system-based practice modules
• Professionalism in Surgery: Challenges and ChoicesTM, ACS
Division of Education (2008)
Chief Resident – Teaching Faculty
Moderator Team (Monthly Curriculum)
• Define content and format
• Identify and confirm presenters
• Utilize the ABSITE program report
• Moderate interactive sessions
– Vignettes, case scenarios, question/answer
– Focus on critical knowledge; decision making
• Exhaustive literature review prohibited
The Monthly Curriculum
Fridays; 8:30 AM – 11:00 AM
Week 1 Interactive presentations
Week 2 Simulation/ skills sessions
Week 3 Interactive presentations; competency lecture, journal club
Week 4 Interactive presentation, resident research (11:00 AM)
Week 5 Dedicated to a 2nd simulation/ skills or competency
presentation
Intern Boot Camp
July / August
• Fluids and
electrolytes
• Cardiac
• Pulmonary
• GI
• MS
• Wounds
•
•
•
•
•
•
•
Nutrition
Pain management
Post-op emergencies
Fever work -up
Hypotension
Pre-op evaluation
Safety
Wednesday 2:00 – 4:30 PM; Friday 8:30 – 11:00 AM
- Fundamental of Surgery Curriculum – 25 essential modules
- Clinical Skills Lab
Simulation / Skills Modules
PGY I
• Asepsis and instrument
identification
• Knot tying; tissue
handling
• Latex allergy; anaphylaxis
• Chest tube and
thoracentesis
• Basic laparoscopy
• CVP and foley placement
• Patient hand-off
• FLS
PGY II-V
• Sim man
• Cholecystectomy/
Advanced laparoscopy
• EGD; colonoscopy
• Vascular anastomosis
• Biolab/fresh tissue lab
• Robotics
ACS e-learning site
Evaluation
• Anonymous Survey Questionnaire
– Baseline (June, 2010); 6 months (December, 2010); 1 year
(June, 2012)
– Fifteen, 5-point Likert-type items
• Conferences in relation to ACGME competencies
– PBI, SBP, IC, Professionalism
• The learning objectives of the curriculum
• Quality of teaching presentations
– Faculty and Residents
• Quality of the supplementary teaching materials
– Chi-square tests of proportions; Kruskal Wallis to
compare full distribution of responses
Survey Questions
I.
Conferences
Please rate the following aspects of the Friday morning core curriculum for the entire year using the scale
below.
The core curriculum conferences taught me the value of practice based learning.
The core curriculum conferences positively impacted my ability to effectively communicate with patients,
families and other health-care professionals.
The core curriculum conferences positively impacted my ability to act in a professional and ethical manner.
The core curriculum conferences provided knowledge of the healthcare system, beyond the confines of the
hospital setting, which enables me to feel comfortable calling on other resources for assistance.
II.
Learning Objectives & Goals
There is the opportunity to suggest curriculum changes with the core curriculum for the Integrated Surgical
Residency Program.
The ACGME core competencies were adequately addressed during the academic year.
.
Survey Questions (con’t)
III.
Supervision & Training – Attending Presentations
Preparedness of the faculty speakers.
Quality of the presentations by faculty speakers.
Do faculty regularly assigns specific reading at least 2 weeks prior to the presentations?
Do faculty presenters ask residents in the audience about key content?
IV.
Supervision & Training – Resident Presentations
Preparedness of the resident speakers?
Quality of the presentations by resident speakers.
Were the weekly assigned reading and video materials relevant to the topics presented?
Do the assigned reading and video materials prepare you sufficiently for the weekly topics?
Survey Questions (con’t)
V.
Materials
Weekly reading and videos relevant.
Weekly reading and videos sufficient preparation.
The Following Likert-Scales Were Used for
Survey Questions
I.
Conferences
0 – Not Applicable
1 – Strongly Disagree
2 – Disagree
II.
3 – Neutral
4 – Agree
5 – Strongly Agree
Learning Objectives & Goals
0 – No Interaction
3 – Sometimes
1 – Never
4 – Most of the time
2 – Infrequently
5 – Always
The Following Likert-Scales Were Used for
Survey Questions (con’t)
III.
Supervision & Training (Attending & Resident
Presentations)
0 – No Interaction
3 – Usually
1 – Not at All
4 – Most Always
2 – Not Usually
5 – Always
IV.
Material
0 – Not Applicable
3 – Valuable
1 – Not Valuable
4 – Very Valuable
2 – A Little Valuable
5 – Extremely Valuable
Results
Percent of Residents Responding:
Two Most Positive Categories
June 2010
1 Year
December
2011
6 months
June
2011
1 Year
X2(p)
Taught value of practice based learning
52.5
78.8
93.3
.005
Positively impacted ability to communicate
37.5
72.7
86.7
.001
Positively impacted ability to act professionally/ethically
45.0
60.6
73.3
n.s.
Provided knowledge of health care system
42.5
60.6
80.0
.034
Opportunity to suggest curriculum changes
70.0
78.8
87.5
n.s.
ACGME core competencies were adequately addressed
72.5
87.9
93.3
n.s.
Faculty speakers prepared
87.5
93.9
100
n.s.
Quality of the faculty presentations
50.0
78.8
73.3
.028
Sufficient lead time for reading assignments
12.5
66.7
73.3
.001
Faculty identifies key required content
50.0
75.8
86.7
.012
Faculty ask residents about key content
37.5
72.7
73.3
.004
Conferences
Learning Objectives and Goals
Supervision and Training - Attendings
6/10 – 42 respondents; 12/10 – 38 respondents; 6/11 – 22 respondents
Percent of Residents Responding: Two Most
Positive Categories (con’t)
June 2010
1 Year
December
2011
6 months
June
2011
1 Year
X2(p)
Resident speakers prepared
80.0
84.8
100
n.s.
Quality resident presentations
25.0
51.5
40.0
n.s.
Weekly reading and videos relevant
34.6
63.6
66.7
.046
Weekly reading and videos sufficient preparation
32.0
54.5
53.3
n.s
Conferences
Supervision and Training – Resident Moderators
Materials
6/10 – 42 respondents; 12/10 – 38 respondents; 6/11 – 22 respondents
American Board of Training Inservice
Examination [ABSITE] Percentage Scores
(Lower Quartiles; Top Quartile)
Exam
Level
Junior
ABSITE
Year
(PGY I, II)
Senior
ABSITE
(PGY III, IV, V)
Year
Lower Quartiles
(0-75%)
Top Quartiles
(76-100%)
2009
89.5%
10.5%
2010
89.5%
10.5%
2011
81.0%
19.0%
2012
57.1%
42.9%
2009
95.0%
5.0%
2010
90.9%
9.1%
2011
88.9%
11.1%
2012
84.2%
15.8%
American Board Of Surgery IntrainingExamamination
Proportion of Residents in Top Quartile (76-100%)
All residents
Junior Level
Senior Level
50
40
Introduction of new
curriculum
30
20
10
0
2009
2010
2011
2012
Study Limitations
• Unequal # of respondents at the 3 times of
the survey
• Unable to separate the survey data by PGY
years or categoricals vs. preliminaries
• ? Factor of survey fatigue
The Residents’ Concerns
prior to the new approach to Core
Curriculum Development
•
•
•
•
•
•
•
•
Diminished available time for residents
Less non-clinical time available to teaching faculty
Resident dissatisfaction with the didactic curriculum
Regurgitation from textbooks
Lack of audience interaction
Session taught by residents lacked depth and scope
Inadequate simulation exposure
Inadequate preparation time (learner & presenter)
Residents’ comments post
implementation
• All lectures more structured and beneficial
– Presenters and learners better prepared
• Presentations much more interactive and engaging
• The best lectures were those that invite audience participation
• The intern boot camp is an extremely valuable component of the
core curriculum
– Taught us from Day 1 important concepts
• Audience response system engaging and great prep for ABSITE
• Avoid resident presentations all together
• I enjoyed the practical portions that have been included in the
curriculum
– Pig lab, lap trainer
– Great chance to practice new skills in an environment not quite
so high stress as the OR
Conclusions
• The systematic collaborative approach (faculty and
residents) to curriculum development with interactive,
objective competency-based presentations, robust
simulation, use of online teaching tools, engaged
teaching faculty resulted in:
–  Resident satisfaction with the curriculum and their
self-reported clinical and academic abilities
–  Increase in the number of residents scoring in the
top quartile in the ABSITE
–  Effectively addresses the ACGME competencies
Thank you!