Transcript Novel Teaching Methods and Innovative Fellow Evaluation Tools

Novel Teaching Methods and
Innovative Fellow Evaluation
Tools
Joanne Filicko-O'Hara, MD;
Thomas Jefferson University
Disclosures for Joanne Filicko-O’Hara, MD
Employment
Kimmel Cancer Center
Thomas Jefferson University
Consultancy
None
Equity Ownership
None
Research Funding
NIH
Honoraria
None
Patents & Royalties
None
Speakers Bureau
None
Membership on Board of
Directors/Advisory Committee
None
Other
Presentation includes a description
of the following off-label use of a
drug or medical device
Investigational Drug Supplied by Millenium
Pharmaceuticals on current trial
None
52nd ASH Annual Meeting ♦ Orlando, FL
In compliance with ACCME policy, ASH requires the following disclosures to the session audience:
Objectives


Discuss ways to make evaluations of
trainees meaningful in real time.
Discuss ways to make faculty
participation in the evaluation process
and other fellowship activities
meaningful.
The Evaluations

Of Trainees








By faculty on each rotation
By faculty in continuity
experience
By faculty – Other?
By peers
By patients and families
By other staff (RN, Social
Work, Pharm D)
By program director
By others (students?
residents?)

By trainees

Of rotations



Of faculty






faculty
total experience
Rotations
Continuity clinics
Other
Of peers
Of program
Of other staff
The Evaluations






Medical Knowledge (faculty, peers?)
Patient Care (faculty, patients/ families, other staff,
peers)
Practice Based Learning and Improvement (faculty,
other staff, peers)
Systems Based Practice (faculty, patients/ families,
other staff, peers)
Interpersonal Skills (faculty, patients/ families, other
staff, peers)
Professionalism (faculty, patients/ families, other
staff, peers)
The Evaluations






Medical Knowledge (faculty, peers?)
Patient Care (faculty, patients/ families, other staff,
peers)
Practice Based Learning and Improvement (faculty,
other staff, peers)
Systems Based Practice (faculty, patients/ families,
other staff, peers)
Interpersonal Skills (faculty, patients/ families, other
staff, peers)
Professionalism (faculty, patients/ families, other
staff, peers)
The Evaluation Process
Evaluation of Fellow Performance During Rotation
Fellow
provides
patient care
under
supervision of
Dr. Jones
Dr. Jones
provides
ongoing
feedback
about fellow
performance
Dr. Jones provides a final
evaluation of fellow
performance at the end of
rotation that is:
“in sync” with ongoing
feedback
and
reviewed with the fellow
The Evaluation Process
Fellow
provides
patient care
under
supervision of
Dr. Jones
Dr. Jones
provides
ongoing
feedback
about fellow
performance
Dr. Jones provides a final
evaluation of fellow
performance at the end of
rotation that is:
“in sync” with ongoing
feedback
and
reviewed with the fellow
Why isn’t it done “right”?


Too busy
No one wants to be critical
Why isn’t it done “right”?



Too busy
No one wants to be seen as critical
On line evaluation systems are great for
record keeping, but you can’t carry the
system in your pocket
One solution





Self assessments mirror the faculty end-ofrotation evaluation
Fellows do self-assessment on paper version
Fellows review with attending during the final
few days of rotation and sign
Fellows hand in signed version to program
coordinator
Faculty often keep a copy for reference when
doing the online version
Why isn’t it done “right”?



Too busy (forces the issue)
No one wants to be seen as critical
On line evaluation systems are great for
record keeping, but you can’t carry the
system in your pocket
The problem of too few
evaluations




It’s easy to match evaluations to
individual evaluators
One good/bad evaluation can skew data
Many evaluators grade up
Many evaluators don’t want to be the
one who’s “not nice”
Hematology/ Medical Oncology Fellowship
Faculty Quarterly Evaluation of Fellows
During the last three months, I have worked with this
fellow (check all that apply):
 In the outpatient setting
 In the inpatient setting
 During night and weekend call
 I have not worked with this fellow during the past
three months.
Scale: 1 – Severe Deficits, 2 – Marginal
Competence, 3 – Expected Competence, 4 –
Above Expected Competence, 5 - Outstanding
1
Arrives on time for office hours/ rounds
Completes dictations in a timely fashion
Demonstrates professionalism in dealing with
patients; Communicates effectively with patients
and families ; shows compassion and empathy
Demonstrates professionalism in dealing with
nurses/physician extenders; relates well with
other health care professionals
Responds to requests for consults in a timely
fashion
Writes an appropriate letter/ progress note – all
necessary components, grammar/ spelling
Remains focused on tasks at hand (not having
constant interruptions/ phone calls, etc)
Is competent at performing history and physical
exam
2
3
4
5
NA
1
Has an appropriate knowledge of basic science
related to hematology/ oncology
Shows appropriate clinical judgment in
developing differential diagnosis and treatment
plan
Is sensitive to patient’s age, gender, culture
and disabilities
Is knowledgeable about indications/
contraindications of procedures
Is skilled in performing procedures
Adheres to professional ethical standards
Teaches medical students and residents
effectively
Is willing to admit errors and accept feedback
2
3
4
5
NA
1
Considers costs of diagnosis and management
Assesses own strengths and weaknesses; Has
good self directed learning skills
Adapts to new technology
Is committed to quality assurance and
improvement
Is effective at managing health care resources
Is an effective team member
Is an effective consultant
2
3
4
5
NA
Fellow Quarterly Evaluation of
Faculty
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Discusses patient care plans w/me at the time of ADMISSION
Treats me with RESPECT as a colleague
TEACHES about oncologic or hematologic issues and directs me
to appropriate reading
Involves me in patient care decisions
Interacts well with the other faculty – shows them respect and
is collegial
Provides me with appropriate support/back-up
Responds to calls at night – in a reasonable time, is helpful and
appreciative
Is accessible
Provides helpful input at conferences
discusses end-of-life issues with patients, families and me
Please rate the attending physicians on the following aspects of patient care and interaction with
you: In each case, please feel free to provide comments as needed to explain answers or to
provide examples showing why interacting with any individual faculty member is a good or bad
experience. This is completely anonymous.
…..discusses patient care plans with me at the time of admission
Faculty Name
Dr. Axelrod
Dr. Besa
Dr. Carabasi
Dr. Carr
Dr. Filicko-O’Hara
Dr. Flomenberg
NA
Never
Rarely
Sometimes
Often
Usually
Always
0
1
2
3
4
5

12 fellows; 10 questions – 120 data points
Twice each year – 240 data points
Anonymous and untraceable

Reviewed with chiefs



Faculty evaluation of fellow – 23 questions;
on line; more faculty harder to enforce
Does it help?


Data presented at department meeting
Incentives for bonus


Faculty need to score at the median or
better
Target participation in educational activities
Why isn’t it done “right”?




Too busy
No one wants to be seen as critical
On line evaluation systems are great for
record keeping, but you can’t carry the
system in your pocket
Incentives for faculty help get the job
done
Other solutions?

For discussion