Transcript Competency Based Education and Assessment

Competency Based Medical Education
and Assessment
Maria Lucarelli, MD
Associate Program Director
Internal Medicine Residency
January 13, 2015
Conflict of Interest
 None
2
Describe the characteristics and
benefits of competency assessments in
medical education
Objectives
Define types of competency
assessments that may be applicable to
your medical education area
Identify existing competency
assessment tool in your medical
education area
3
Competency Based Education
 Competency-based education (CBE) is an approach
to preparing physicians for practice that is
fundamentally oriented to graduate outcome abilities
and organized around competencies derived from
analysis of societal and patient needs. It
deemphasizes time-based training and promises
greater accountability, flexibility, and learner
centeredness
 J. R. Frank et al. 2010; 32: 631–637 Medical Teacher
4
Competency Based Education





5
Outcomes Based
Evaluation integrates knowledge, skills and attitudes
Time-independent
Learning is Individualized
Learning occurs in the workplace
Process versus Competency based
Programs
Carraccio, C, Wolfsthal, SD, Englander, R, Ferentz, K, Martin C “Shifting Paradigms: From Flexner to Competencies” Acad Med
(2002) 77:361-367.
6
Competency Based Education
 Requires continuous, comprehensive assessment
 Majority of assessment should happen in clinical
environment
 Requires ongoing, highly effective feedback
7
Milestones
 A significant point in development and helps define
the appropriate trajectory of a trainee
 Identify discrete knowledge, skills and attitudes
expected of learners as they progress through
training
8
Milestones
 Can demonstrate individual trajectory of competency
acquisition
 Provides clear path of progress
 Can help focus assessment
 Allows for richer feedback
9
Competency
 Observable
 Measurable
 Integrating multiple components such as knowledge,
skills, values, and attitudes
 Can be assembled for progressive development
 Descriptors of physicians
11
Competence
 Abilities across multiple domains or aspects of
performance in a certain context
 Multi-dimensional and dynamic. It changes with time,
experience, and setting.
 Requires qualifiers
 Relevant abilities
 Context
 Stage of training
12
Competent
 Possessing the required abilities in all domains in a
certain context at a defined stage of medical
education or practice.
13
Dreyfus Model
Expert
Proficient
Competent
Advanced
Beginner
Novice
14
Statement of Awarded Responsibility
Expert
Proficient
Providing supervision to others
Acting unsupervised
Competent
Advanced
Beginner
Acting with supervision
available within minutes
Acting with direct
supervision
present in the
room
Novice
Observing
the Activity
A case for competency-based anaesthesiology training
with entrustable professional activities: An agenda for
development and research.
Jonker, Gersten; Hoff, Reinier; ten Cate, Olle
15
European Journal of Anaesthesiology. 32(2):71-76,
February 2015.
DOI: 10.1097/EJA.0000000000000109
16
Entrustable Professional Activities (EPA)
 Specific knowledge,
skills and attitudes
acquired over the
course of training
critical to performing as
a physician
Ten Cate Academic Medicine, Vol. 82, No. 6 / June
2007
17
EPA
Competency
Subcompetencies
Milestones
18
End-ofTraining EPA
1.
Manage
the care of
patients
with acute
common
diseases
across
multiple
care
settings
Step 1
Description and Tasks
Internal medicine physicians entering
into unsupervised practice are able to
diagnose and manage common acute
medical symptoms (e.g., joint pain,
chest pain, and headache) and
conditions (e.g., uncontrolled HTN,
decompensated HF, and COPD
exacerbation) in community,
ambulatory, and hospital settings.
The tasks required:

Obtain accurate and complete
information sufficient to develop
differential diagnosis and inform
care plan;

Knowledge of diseases common
to internal medicine;

Communicate plans of care to
patients, families and care givers

Adapt care plans to changing
clinical information
Step 2
Related Curricular Milestones
(Abbreviations on AAIM Website)
Patient Care (PC)
Medical Knowledge (MK)
Practice-Based Learning &
Improvement (PBLI)
Interpersonal &
Communication Skills (ICS)
Professionalism (P)
Systems-Based Practice
(SBP)
A2, A3, B1, B2, C2, C3, D1,
E1, F8
A2, A3, B1, B3
B1, B2, B3, D4, E1, E2
A3, A4, A5, A7, B1, B3, D3,
F1
B1, B3, D2, E1, F1, F2, F3
G2, H1, I1, I2, K3
A3, B2, D2, D4, E1, E3
Building Assessments for an EPA in Three Steps. AAIM Conect
19
Step 3
Assessment
Methods/Tools
Multisource
feedback
Chart stimulated
recall
Chart audits
Direct observations
Standardized
patient/OSCE
In-training
examination
Fig. 1
A case for competency-based anaesthesiology training
with entrustable professional activities: An agenda for
development and research.
Jonker, Gersten; Hoff, Reinier; ten Cate, Olle
European Journal of Anaesthesiology. 32(2):71-76,
February 2015.
DOI: 10.1097/EJA.0000000000000109
Fig. 1 . Acquisition of competence. (a) Acquisition of
competence, showing the competence threshold
(corresponding to proficiency level IV) and continuing
growth of expertise after delegation of a clinical activity.
(b) Acquisition of competence for 5 different EPAs. At
reaching the competence threshold, an informed and
justified delegation decision can be made. Adapted with
permission from 20.
© 2015 European Society of Anaesthesiology. Published by Lippincott Williams & Wilkins.
2
Role of Assessment
• Earlier identification
• Customized training
• More rapid
advancement
Deficiencies
• Earlier identification
• Customized
remediation plans
• Earlier and fair exit
from training when
necessary
21
Advanced
Holmboe et al The role of assessment in
competency based medical education.
2010; 32:676-82. Medical Teacher
Assessments
 Needs to be continuous and frequent
 Formative > summative
 Allows for deliberate practice
 Needs to be coupled with effective feedback and
ongoing mentoring/coaching
22
Assessments
 Criterion Based
 Allows for appropriate expectations
 Developmental (milestones)
 Blueprint for assessment and informs appropriate
methods/tools for assessment
 Take place in the clinical environment
 Does not eliminate the role of simulation
 Heavily dependent upon faculty
observations/assessment
23
Assessment
 Requires the use of high quality assessment tools
 One single evaluation may not be perfect
 Consider the use of multiple tools
 Qualitative approaches to assessment can be
valuable
24
Narrative descriptions should replace grades and numerical ratings for clinical performance in medical
education in the United States
Janice L. Hanson*, Adam A. Rosenberg and J. Lindsey Lane Front. Psychol., 21 November 2013
25
Knowledge assessments
360 evaluations
Patient surveys
26
Mapping Milestones
A Tool for Mapping the ACGME Milestones to Evaluation Sources
Linda Myerholtz Ph.D.
27
Resources
 PM&R Milestones Central
 www.physiatry.org
 Society of Teachers of Family Medicine
 www.fmdrl.org
 Alliance For Academic Internal Medicine
 www.im.org
28