Transcript Survey Results - American Society of Hematology

Program Directors Survey
How are we evaluating our fellows?
Ann Zimrin, MD
University of Maryland Greenebaum
Cancer Center
Disclosures for Ann Zimrin, MD
In compliance with ACCME policy, ASH requires the following disclosures to the session audience:
Consultancy
Equity Ownership
Research Funding
Honoraria
Patents & Royalties
Speakers Bureau
Membership on Board of
Directors/Advisory Committee
Other
Presentation includes a description
of the following off-label use of a
drug or medical device
none
none
none
none
none
none
none
none
none
none
52nd ASH Annual Meeting ♦ Orlando, FL
Employment
Why do the survey?
Original survey
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
SHARING BEST PRACTICES SURVEY
General Information
Name of program:
Name of program director:
Number of fellows per year:
Number of months clinical work on average (out of 24 or 36) during fellowship:
Strategy for assessing competencies
CompetencyMethods Used
Patient Care
Medical Knowledge
Practice-based Learning
Interpersonal and Communication Skills
Professionalism
Systems-based Practice
How do you assess a fellow’s capacity for analytic thinking and its application to patient problems?
Modified survey
•
•
•
•
•
•
•
•
•
•
•
•
•
•
What evaluation methods are you currently using?
Mini-CEXs
Faculty global ratings
360 degree evaluations
Faculty feedback on presentations (such as journal club, grand
rounds, research presentations)
Portfolios
Procedure logs
In-service exams (ASH, ASCO)
QOPI
Simulation center (patient interviews, physical exams)
Simulation center (procedures)
Case review conferences
Anything else?
How do you assess a fellow’s capacity for analytic thinking (optional)?
Who responded?
• Adult programs
– Hematology/Oncology combined 69/126 (55%)
– Hematology 3/7 (43%)
– Oncology 4/16 (25%)
• Pediatric Hematology/Oncology29/65 (45%)
Adult programs
Pediatric programs
Responses by Region
Evaluation Methods
Evaluation Methods
Evaluation Methods
Other Methods (Adults)
• A set of cases for which the fellows provide
“consultation” report with critical analysis of
the issues and relevant literature (q 6 mos)
• Written test and demonstration of manual
competency in administrating chemotherapy
• Chart reviews by the faculty of the fellows’
notes; review of fellows’ overnight triage log
• Chart stimulated recall oral exams
Other Methods (Peds)
• Individualized Learning Plans
• Scholastic Oversight Committee presentations
• “Pretending” month as third year
Wish List
• On-line teaching modules concerning ethics,
delivery of bad news, cultural sensitivity
• Better ways to mentor and assess research
aspect of fellowship
• Simulation models for bone marrow biopsies
New ACGME Requirement
• “Fellows must have access to training using
simulation.”
• “The committee does NOT expect each program
to use a simulator or have a simulation center.
Simulation means that learning about patient
care occurs in a setting that does not include
actual patients. This could include OSCEs,
standardized patients, patient simulators, or
electronic simulation of codes, procedures, and
other clinical scenarios.”
Simulation – OSCE
• Objective Structured Clinical Examination
– Students rotate from station to station
– Interview patients, interpret lab findings and
imaging
– Observed by examiners and performance scored
• Useful way to test students’ grasp of common
topics (“Approach to the Patient with Chest Pain”,
for example)
OSCE - Application to Heme/Onc
• Possible topics
–
–
–
–
Approach to the patient with thrombocytopenia
Approach to the patient with anemia
Approach to the patient with a mediastinal mass
Gerontology issues
• Disadvantages
– Expensive
– Time-consuming to set up
– Questionable utility
Computer-Based Simulation
• Schick and Burke (The Internet Journal of Hematology
2010 : Volume 7 Number 1)
– Computer-based tutorial on the topic of bleeding
disorders
– After reading history, residents and fellows were
asked to order tests from a menu and make
recommendations for treatment
– Authors noted that selection of diagnostic tests
was the weakest area
Lab-based Simulation
• Insertion of CVP catheters
• Administration of intrathecal chemotherapy
• Bone marrow biopsy simulation
– Sternal aspirate
• More on simulation in upcoming breakout
session
Tools: Where to Look for Help
• ASH Website
– http://www.hematology.org/Training
•
•
•
•
Presentations from previous TPDWs
Grants Clearinghouse
Hematology curriculum
Case studies for fellows (board-type questions with
answers and references)
• Image Bank (slides to download or print)
• Teaching Cases for Medical Students
• Medical Student Hematology Course Learning
Objectives
Tools: Where to Look for Help
• ASCO Website www.asco.org
– Training Program Director sessions from previous
annual meetings
– Oncology Training Program Directors Retreat –
presentations and extra material submitted by PDs
– Medical Oncology Core Curriculum
– Communication Skills
– “Works for Me”
– ASCO Tumor Boards
Tools: Where to Look for Help
• The American Society of Pediatric
Hematology/Oncology
– Site for training program directors (for sharing
information, documents, links and files in an
organized, searchable library) – password
protected
Tools: Where to Look for Help
• European Hematology Organization
(www.ehaweb.org)
– Passport (a hematology curriculum)
– On-line cases
• ACGME
– Outcomes Project
• Toolbox
• Educating Physicians for the 21st Century
• RSVP (Recognize Success Via imPlementation)
My Conclusions
• Although many evaluation methods are used,
faculty feedback is the cornerstone of the
evaluation in essentially all programs surveyed
• Faculty development to make faculty better
observers would be useful
• Simulation not yet incorporated into most
programs
• Resources are available on-line