Transcript Trimming the Fat: Optimizing Overall Educational Value by Defining Factors Associated with Overall Educational Value and Service to Education Ratio Caroline E.

Trimming the Fat:
Optimizing Overall Educational Value by Defining
Factors Associated with Overall Educational Value
and Service to Education Ratio
Caroline E. Reinke, MD MSPH, Rachel R. Kelz, MD MSCE,
Lori Pray, BA, Noel Williams, MD, Joshua Bleier, MD,
Kenric Murayama, MD, Jon B. Morris, MD
Division of Surgery Education
Hospital of the University of Pennsylvania
Philadelphia, PA
Optimizing Overall Educational Value
Nothing To Disclose
Optimizing Overall Educational Value
Background
• Accreditation Council for Graduate Medical Education
(ACGME) responsible for assessing and advancing the
quality of resident physicians’ education
– Founded in 1981
– Requires programs to achieve a balance between education and
service
ACGME Resident Survey:
• With decreasing work hours, defining which
In characteristics
your opinion, howare
often
do your rotations
and other
major
educational
is critical
to effective
assignments
educationprovide an appropriate balance between your
residency education and other clinical demands?
•How
Little
is has
known
modifiable
rotation by
often
your about
clinical which
education
been compromised
characteristics
impact perceived service and education
excessive
service obligations?
Optimizing Overall Educational Value
Specific Aims
AIM 1: To identify rotation characteristics associated with service to
education ratio and overall educational value
AIM 2: To understand the relationship between the perceived overall
educational value and the service to education ratio
AIM 3: To understand how these relationships differ between junior
(CY1-2) and senior residents (CY3-5)
Optimizing Overall Educational Value
Data Source
• Responses to an 11-item Web-based survey during
FY2011
– Distributed to all residents at the end of each rotation
• Exclusion criteria:
– Non-clinical rotations (simulation month)
– Subspeciality-specific rotations (ie. plastic surgery)
• Survey responses were linked to additional rotation data:
– Length of the rotation in number of weeks
– Number of attending surgeons on each service
– Presence or absence of fellows
Optimizing Overall Educational Value
Rotation Survey
Rotation Characteristics
1
2
3
4
5
Poor
Fair
Adequate
Very Good
Complexity of Operative Experience
Too Simple
Basic
Appropriate
Challenging
Exceptional
Beyond my
Capacity
Autonomy of Operative Experience
Minimal
Occasional
Commesurate
with ability
Frequent
Excessive
Patient Care Participation
Minimal
Occasional
Adequate
Frequent
Excessive
Autonomy in Patient Care
Minimal
Occasional
Commensurate
with ability
Frequent
Excessive
Minimal
Would
prefer more
Sufficient
Would
prefer less
Excessive
Detrimental
Inefficient
Irrelevant
Marginal
Exceptional
50% Educ,
50% Service
80% Educ,
20% Service
100% Educ,
0% Service
Adequate
Very Good
Outstanding
Just right
1 month too
long
2 months
too long
Occasionally
Frequently
Always
Volume of Operative Experience
Time for Reading/Cognitive
Development
Educational Value of Fellows and
Speciality Residents
Ratio of Educational to Service
Activity
Overall Educational Value
Duration of Rotation
Teaching Faculty are supportive of
ACGME Duty Hour Rules
0% Educ,
20% Educ,
100% Service 80% Service
None
Fair
2 months too 1 month too
long
short
Never
Rarely
Optimizing Overall Educational Value
Study Population
57 general surgery training program rotations were
evaluated by 85 residents
Excluded rotations in which less than three residents in a
given CY evaluated the rotation (n=9)
Overall survey response rate = 87%
Final study population = 48 general surgery rotations
with 578 resident evaluations (range 3-36)
Optimizing Overall Educational Value
Statistical Analyses
• Linear regression used to assess the relationship between
rotation characteristics (RC), rotation data (RD), and
– Overall Educational Value (OEV)
– Service to Education Ratio (SER)
• Linear regression used to assess the relationship between
OEV and SER
• Multivariate linear regression was used to examine the
relationship between SER and OEV after controlling for
significant RC and RD
• Subgroup analyses performed for junior (PGY1-2) and senior
(PGY3-5) residents
Optimizing Overall Educational Value
Relationship between OEV and SER
1
2
3
4
5
Operative Autonomy vs. Overall Educational Value
1
2
3
4
Autonomy of Operative Experience
5
Optimizing Overall Educational Value
Relationship between OEV and SER
1
2
3
4
5
Operative Autonomy vs. Overall Educational Value
1
2
3
4
Autonomy of Operative Experience
5
Optimizing Overall Educational Value
RC, RD and the relationship with OEV
Rotation Characteristics (RC) of interest
Overall educational value (OEV)
Additional Rotation Data (RD)
Rotation Length
Number of attending surgeons
Fellow
Other Rotation Characteristics (RC)
Volume of operative experience
Complexity of operative experience
Autonomy of operative experience
Patient care participation
Autonomy in patient care
Time for reading and cognitive development
Educational value of fellows on the rotation
Duration of the rotation
Teaching faculty are supportive of ACGME Duty Hour Rules
β Coefficient
--
p-value
--
0.03
-0.11
-0.02
0.71
0.02
0.93
0.37
0.69
0.58
-0.36
0.58
0.85
0.35
-1.63
1.02
<0.001
<0.001
<0.001
0.32
0.005
<0.001
0.002
<0.001
<0.001
Optimizing Overall Educational Value
RC, RD and the relationship with OEV
Rotation Characteristics (RC) of interest
Overall educational value (OEV)
Additional Rotation Data (RD)
β Coefficient
--
p-value
--
Rotation Length
0.03
0.71
Number of attending surgeons
-0.11
0.02
Fellow
-0.02
0.93
Other Rotation Characteristics (RC)
Volume of operative experience
Complexity of operative experience
Autonomy of operative experience
0.37
0.69
0.58
<0.001
<0.001
<0.001
Patient care participation
-0.36
0.32
Autonomy in patient care
Time for reading and cognitive development
Educational value of fellows on the rotation
Duration of the rotation
Teaching faculty are supportive of ACGME Duty Hour Rules
0.58
0.85
0.35
-1.63
1.02
0.005
<0.001
0.002
<0.001
<0.001
Optimizing Overall Educational Value
RC, RD and the relationship with OEV
Rotation Characteristics (RC) of interest
Overall educational value (OEV)
Additional Rotation Data (RD)
Rotation Length
Number of attending surgeons
Fellow
Other Rotation Characteristics (RC)
β Coefficient
--
p-value
--
0.03
-0.11
-0.02
0.71
0.02
0.93
Volume of operative experience
Complexity of operative experience
Autonomy of operative experience
0.37
0.69
0.58
<0.001
<0.001
<0.001
Patient care participation
-0.36
0.32
Autonomy in patient care
Time for reading and cognitive development
Educational value of fellows on the rotation
0.58
0.85
0.35
0.005
<0.001
0.002
Duration of the rotation
Teaching faculty are supportive of ACGME Duty Hour Rules
-1.63
1.02
<0.001
<0.001
Optimizing Overall Educational Value
RC, RD and the relationship with OEV
Rotation Characteristics (RC) of interest
Overall educational value (OEV)
Additional Rotation Data (RD)
Rotation Length
β Coefficient
--
p-value
--
0.03
0.71
Number of attending surgeons
-0.11
0.02
Fellow
Other Rotation Characteristics (RC)
Volume of operative experience
Complexity of operative experience
Autonomy of operative experience
Patient care participation
Autonomy in patient care
Time for reading and cognitive development
Educational value of fellows on the rotation
-0.02
0.93
0.37
0.69
0.58
-0.36
0.58
0.85
0.35
<0.001
<0.001
<0.001
0.32
0.005
<0.001
0.002
Duration of the rotation
-1.63
<0.001
Teaching faculty are supportive of ACGME Duty Hour Rules
1.02
<0.001
Optimizing Overall Educational Value
RC, RD and the relationship with SER
Rotation Characteristics (RC) of interest
Service to Education Ratio (SER)
Additional Rotation Data (RD)
Rotation Length
Number of attending surgeons*
Fellow
Other Rotation Characteristics (RC)
Volume of operative experience
Complexity of operative experience
Autonomy of operative experience
Patient care participation*
Autonomy in patient care*
Time for reading and cognitive development
Educational value of fellows on the rotation*
Duration of the rotation
Teaching faculty are supportive of ACGME Duty Hour Rules
β Coefficient
--
p-value
--
-0.7
1.29
0.13
0.87
0.06
0.45
-1.62
-0.63
-0.81
0.19
-0.29
-0.74
-0.51
0.25
-0.53
<0.001
<0.001
<0.001
0.03
0.06
<0.001
0.06
0.001
<0.001
Optimizing Overall Educational Value
RC, RD and the relationship with SER
Rotation Characteristics (RC) of interest
Service to Education Ratio (SER)
Additional Rotation Data (RD)
Rotation Length
β Coefficient
--
p-value
--
-0.7
0.87
Number of attending surgeons*
1.29
0.06
Fellow
Other Rotation Characteristics (RC)
Volume of operative experience
Complexity of operative experience
Autonomy of operative experience
Patient care participation*
Autonomy in patient care*
Time for reading and cognitive development
0.13
0.45
-1.62
-0.63
-0.81
0.19
-0.29
-0.74
<0.001
<0.001
<0.001
0.03
0.06
<0.001
Educational value of fellows on the rotation*
-0.51
0.06
Duration of the rotation
Teaching faculty are supportive of ACGME Duty Hour Rules
0.25
-0.53
0.001
<0.001
Optimizing Overall Educational Value
RC, RD and the relationship with SER
Rotation Characteristics (RC) of interest
Service to Education Ratio (SER)
Additional Rotation Data (RD)
Rotation Length
Number of attending surgeons*
Fellow
Other Rotation Characteristics (RC)
Volume of operative experience
Complexity of operative experience
Autonomy of operative experience
Patient care participation*
Autonomy in patient care*
Time for reading and cognitive development
Educational value of fellows on the rotation*
β Coefficient
--
p-value
--
-0.7
1.29
0.13
0.87
0.06
0.45
-1.62
-0.63
-0.81
0.19
-0.29
-0.74
-0.51
<0.001
<0.001
<0.001
0.03
0.06
<0.001
0.06
Duration of the rotation
0.25
0.001
Teaching faculty are supportive of ACGME Duty Hour Rules
-0.53
<0.001
Optimizing Overall Educational Value
Relationship between OEV and SER
1
2
3
4
5
Service to Education Ratio (SER) vs. Overall Educational Value (OEV)
1
2
3
4
Service to Education Ratio (SER, mean)
Senior (CY 3-5)
Junior (CY 1-2)
5
Optimizing Overall Educational Value
Relationship between OEV and SER
1
2
3
4
5
Service to Education Ratio (SER) vs. Overall Educational Value (OEV)
1
2
3
4
5
Service to Educaiton Ratio (SER, mean)
Senior (CY 3-5)
Junior (CY 1-2)
Fitted values
Optimizing Overall Educational Value
The relationship between SER and OEV
Rotation Characteristics (RC)
of interest
Overall Educational Value
β Coefficient
p-value
Adj. Rsquared
Overall educational value (OEV)
--
--
--
Service to education ratio (SER)
-1.23
<0.001
0.68
Optimizing Overall Educational Value
The relationship between SER and OEV
Rotation Characteristics (RC)
of interest
Overall Educational Value
β Coefficient
p-value
Adj. Rsquared
Overall educational value (OEV)
--
--
--
Service to education ratio (SER)
-1.23
<0.001
0.68
-1.08
<0.001
0.85
After adjustment for
other RC & RD
Service to education ratio (SER)
Optimizing Overall Educational Value
Subgroup Analysis of Junior and Senior Rotations
Overall Educational Value
Junior Rotations (n=28)
β Coefficient
p-value
Adjusted
R-squared
--
--
--
Service to education ratio (SER), unadjusted
-1.27
<0.001
0.71
Service to education ratio (SER), adjusted
-1.27
<0.001
0.85
--
--
--
Service to education ratio (SER), unadjusted
-1.28
<0.001
0.65
Service to education ratio (SER), adjusted
-0.46
0.15
0.90
Overall educational value (OEV)
Senior Rotations (n=20)
Overall educational value (OEV)
Optimizing Overall Educational Value
Summary
• We identified an inverse relationship between the
service to education ratio and overall educational value,
which was not fully explained by other rotation
characteristics
• This relationship differed significantly for junior and
senior rotations
• Individual rotation characteristics varied in their
relationship with overall educational value
– The value of fellows varied, but their simple presence or
absence was not predictive of OEV
– Number of attending surgeons was negatively correlated
with overall educational value
Optimizing Overall Educational Value
Thank You
Optimizing Overall Educational Value
RC, RD and the relationship with OEV – Junior
Rotation Characteristics (RC) of interest
Overall educational value (OEV)
Additional Rotation Data (RD)
Rotation Length
Number of attending surgeons
Fellow
Other Rotation Characteristics (RC)
Volume of operative experience
Complexity of operative experience
Autonomy of operative experience
Patient care participation
Autonomy in patient care
Time for reading and cognitive development
Educational value of fellows on the rotation
Duration of the rotation
Teaching faculty are supportive of ACGME Duty Hour Rules
β Coefficient
--
p-value
--
-0.11
-0.09
0.01
0.29
0.10
0.96
0.31
0.55
0.48
-0.55
0.78
0.86
0.76
-1.80
1.04
0.006
0.02
0.02
0.22
0.003
<0.001
0.001
0.001
0.001
Optimizing Overall Educational Value
RC, RD and the relationship with OEV – Senior
Rotation Characteristics (RC) of interest
Overall educational value (OEV)
Additional Rotation Data (RD)
Rotation Length
Number of attending surgeons
Fellow
Other Rotation Characteristics (RC)
Volume of operative experience
Complexity of operative experience
Autonomy of operative experience
Patient care participation
Autonomy in patient care
Time for reading and cognitive development
Educational value of fellows on the rotation
Duration of the rotation
Teaching faculty are supportive of ACGME Duty Hour Rules
β Coefficient
--
p-value
--
0.22
-0.15
0.01
0.08
0.11
0.90
0.60
1.44
1.18
0.10
0.28
1.04
0.32
-1.52
1.56
<0.001
<0.001
<0.001
0.89
0.43
0.04
0.03
0.003
0.03
Optimizing Overall Educational Value
Background – ACGME Resident Survey
• In your opinion, how often do your rotations and other
major assignments provide an appropriate balance
between your residency education and other clinical
demands?
• How often has your clinical education been
compromised by excessive service obligations?