Transcript 2013_06_05_EPA_for_UCSF

Defining and Assessing Entrustable
Professional Activities
Karen E. Hauer, MD
H. Carrie Chen, MD, MSEd
John Q. Young, MD, MPP
Patricia S. O’Sullivan, EdD
With thanks to Olle ten Cate, PhD
UCSF
June 5, 2013
Agenda










1-1:15
1:15
1:45
2:05
2:35
Break
3:15
3:45
4:00
4:30
Welcome
Introduction to EPAs
individual writing of an EPA
overview of EPA development
small group activity, design an EPA (steps 1-4)
small group reporting
overview of steps 5-7
second activity, evidence for an EPA (steps 5-7)
small group reporting and wrap up (all)
Workshop objectives
 By the end of this workshop, participants
will be able to:
 Define an entrustable professional activity
(EPA)
 Develop an entrustable professional activity
that can be used for assessment of medical
trainees
What do you hope to
learn today?
Current Assessment System
 ACGME core competencies:






Patient Care
Medical Knowledge
Practice-based Learning and Improvement
Interpersonal and Communication Skills
Professionalism
Systems-based Practice
Current Assessment System
Practice-based Learning and Improvement
(Housestaff) (Question 5 of 9 - Mandatory)
Assimilates evidence from scientific studies or consultants related to
their patients' health problems. Participates in organized transitions of
care curriculum as specified at each site. Actively seeks out and acts
on performance feedback from the supervising attending and other
team members.
Insufficient
Contact to
Judge
0
1 & 2 = Unsatisfactory
3 = Marginal
1
2
3
Satisfactory/Meets
Expectations
4
5
6
7 & 8 = Excellent
9 = Outstanding
7
8
>> 9 <<
Current Assessment System
What are some limitations of current
assessment systems?
 Core competencies and sub-competencies: long
checklists of behavioral descriptors
 Relies on traditional but limited assessment methods –
knowledge exams, ward evaluations
 Not a holistic summative view of the trainee
Competency
An observable ability of a health professional,
integrating multiple components such as
knowledge, skills, values and attitudes.
The International CBME Collaborators, 2009
Competent
Possesses the required abilities in all domains
in a certain context at a defined stage of
medical education or practice.
The International CBME Collaborators, 2009
Competence
Competence entails more than the possession
of knowledge, skills and attitudes; it requires
you … to apply these [abilities] in the clinical
environment to achieve optimal results.
ten Cate, Med Teach, 2010
does the job
Can you trust the learner to
function independently?
How to Drive a Car
 Competency
 Can accelerate and brake smoothly
 Can approach an intersection and can turn left
 Competent
 Passes driver’s education classes
 Passes driver’s exam to get the license
 Competence
 Drives safely on interstate or during bad weather,
avoids accidents, no traffic tickets
 Parents hand over the keys and walk away
What is the goal with
assessment?
 Integrated, longitudinal, learner-centered
assessment system
 Promote skill acquisition in multiple domains
concurrently
 Assess what learners actually do in practice
 Be able to conclude: this is a trustworthy
trainee
Features of genuine competency
based medical education
 Outcome-based, not process-based: what is
attained is key, not just what is done or taught
 Integrates knowledge, skill, attitude
 Time-independent: length of training for defined
outcomes is not pre-set
 Individualized: trainees and contexts vary
 Workplace-learning based
 Lifelong learning oriented
Competence.
What criteria would you use to
select your doctor?
 Passed all tests and exams?
 Grades and scores?
 Years of training?
 Follows protocols and guidelines?
 Trust that s/he will manage a case in the best
possible way?
Entrustable Professional
Activities (EPAs)
Entrustable Professional Activity (EPA)
 Define important clinical activities
 Link to competencies / milestones
 Include professional judgment of competence by
clinicians
 Make “decisions of entrustment” for “entrustable”
activities
 Something changes for the learner….
 Build collection of mastered EPAs (portfolio) to
document full competence
Definition of an Entrustable
Professional Activity
A core unit of work, reflecting a responsibility that should only
be entrusted upon someone with adequate competencies
More specific:
 Part of essential professional work in a given context
 Independently executable, within a time frame
 Leads to recognized output of professional labor
 Observable and measurable in process and outcome, leading to a
conclusion (“well done” or “not well done”)
 Must require sufficient, specific knowledge, skill and attitude,
generally acquired through training
 Should reflect competencies, important to be acquired
 Usually confined to qualified personnel only
Competencies versus EPAs
•
•
•
•
•
•
Competencies
person-descriptors
EPAs
work-descriptors
knowledge, skills,
attitudes, values
essential parts of
professional practice
content expertise
collaboration ability
communication ability
management ability
professional attitude
scholarly approach
•
•
•
•
•
•
discharge patient
counsel patient
lead family meeting
design treatment plan
perform paracentesis
resuscitate if needed
Competencies and EPAs combined
xx
xxx
x
xx
Lead a family meeting to
discuss serious news with
patient/family and other
health providers
xxx
x
x
xxx
xx
x
System-Based
x
Practice
Professionalism
xxx
PBLI
Communication
Medical
Knowledge
Patient Care
Develop and implement a
safe discharge plan for a
patient from the acute care
setting
Exercise
 Complete Worksheet A on your own
 List good examples of essential EPAs your trainees
should be able to do independently at the end of training.
 List bad examples of EPAs
 Pair Share: share your examples with one person next
to you.
Recommended full EPA description
1.
EPA Title (max 20 words, avoid skill and avoid adjectives)
2.
Description of the activity (to serve universal clarity, include
limitations)
3.
Expected KSA (to serve trainee)
4.
Link with competencies and predefined milestones (to embed
within the existing framework)
5.
Sources of information to determine progress (to serve
observation and assessment)
6.
Basis for formal entrustment decision (who will have a say in
the decision -- signatures if formal and documented)
7.
Post level-4 of entrustment (“unsupervised”) (what difference
does it make for the trainee?)
Example from Pediatrics
One of the 16 draft EPAs developed by AAMC and ABP
team for pediatric residency training:
1. EPA Title: Manage patients with acute, common
single system diagnoses in an ambulatory,
emergency, or inpatient setting
2. Description: Merged with expected KSA and clarified
in descriptive vignettes
Example EPA cont’d
3. Expected KSA:
 Gathering info thru hx, PE, and initial labs
 Sound clinical reasoning driving development of DDx to




allow proper diagnostic testing and initial therapy
Knowledge of evidence related to primary problem
Application of evidence to management plan
Patient and family-centered care with bidirectional
communications
Documentation of plan and reasoning that is
transparent to other members of health care team
Example EPA cont’d
4. Link with competencies/milestones:
 Patient care




Perform complete and accurate PE
Make informed diagnostic and therapeutic decisions that
result in optimal clinical judgment
Develop and carry out management plan

Medical knowledge
Interpersonal and communication skills
 Communicate effectively with patients, families, and the

public as appropriate, across a broad range of
socioeconomic and cultural backgrounds
Maintain comprehensive, timely, and legible medical
records
PE Sub-competency
Performs essentially the
same rote head-to-toe
physical examination of the
patient regardless of
presenting complaint; does
not use diagnostic
hypotheses from the
history to anticipate or look
for specific positive or
negative findings on
physical examination.
With a broad list of
diagnostic hypotheses after
the history, uses a head-totoe approach to the
physical examination to
anticipate and look for a
myriad of potential positive
and negative physical
examination findings for
multiple diagnostic
considerations. This
approach can lead to failure
to identify pertinent and
important physical findings
that are present,
misinterpretation of
physical findings, and
attribution of importance
and meaning to irrelevant
findings.
Uses a narrow list of
diagnostic hypotheses
generated through the
history to anticipate and
look for specific positive or
negative physical
examination findings of
only the most relevant
diagnostic considerations;
open to new diagnostic
possibilities in the process
of performing a survey
physical examination to
elicit unexpected
abnormalities but may
dismiss these as
unimportant when it is
difficult to integrate these
findings into the working
differential diagnosis.
Uses a narrow list of
diagnostic hypotheses
generated through the
history as well as through
extensive clinical
experience to anticipate
and look for key specific
physical examination
findings that will
discriminate between
competing similar
diagnoses; uses surprises
that result from a survey
physical examination to
rethink and retest
diagnostic hypotheses;
actively looks for physical
exam findings that
disconfirm the working
diagnosis or rule in or out
rare but high-risk
alternative diagnoses.
Description
 Behaviors of early, more advanced, competent,
proficient, and expert learner
 Expected elements from milestones
 Vignette – 2 yo with wheezing, resp distress
 Reassurance provided during hx
 Has FB aspiration on differential so focuses on differential
BS in addition to wheezing and WOB
 Presents focused hx/PE with reasoned assess/plan
consistent with family’s wishes and health literacy,
including SW referral for loss of insurance
Small Group Activity:
Design an EPA
 Work in small groups
 Use resources provided
 Internal medicine milestones
 SOM milestones
 Develop an EPA for
 Internal medicine
OR
 Medical students
Small Group Debrief
When is “competence” reached?
When you trust the trainee
When a professional activity is mastered
• on a threshold level
• that permits unsupervised practice
• and full entrustment
 It happens all the time: when trainees work
without direct supervision
Level of supervision
 Level 1: not allowed to practice the EPA
 Level 2: practice with full supervision
 Level 3: practice with supervision on demand
 Level 4: “unsupervised” practice allowed
 Level 5: supervision task may be given
Recommended full EPA description
1.
EPA Title (max 20 words, avoid skill and avoid adjectives)
2.
Description of the activity (to serve universal clarity, include limitations)
3.
Expected KSA (to serve trainee)
4.
Link with competencies and predefined milestones (to embed within the existing framework)
5. Sources of information to determine progress (to
serve observation and assessment)
6. Basis for formal entrustment decision (who will have
a say in the decision -- signatures if formal and
documented)
7. Post level-4 of entrustment (“unsupervised”) (what
difference does it make for the trainee?)
Dreyfus and Dreyfus Model
• Novice – Don’t know what they don’t know
• Advanced Beginner – Know what they don’t know
• Competent – Able to perform the tasks and roles of the
discipline – restricted breath and depth
• Proficient – In depth knowledge concerning the
discipline – often rule based – know what they know
• Expert – Expert thrives with situations that break the
rules – who the proficient practitioners go to for help
Development of Competence
expert
proficient
competent
advanced
novice
training
deliberate professional practice
Dreyfus & Dreyfus, 1986
Sample Competency Curve
EPA2
EPA4
EPA3
EPA1
Competence
EPA5
Threshold
Justified entrustment decisions
training
deliberate professional practice
Small Group Activity:
Choosing Evidence
 What evidence will inform the EPA that you developed
Small Group Reporting
Wrap Up