Transcript Surveys: One More Outcomes Measure

Surveys: One More Outcomes
Measure
Jay Shapiro, MD
Program Director
Anesthesiology
Program Evaluation and Improvement
1. The program must document formal, systematic evaluation of
the curriculum at least annually. The program must monitor
and track each of the following areas:
a) resident performance;
b) faculty development;
c) graduate performance, including performance of program
graduates on the certification examination; and,
d) program quality. Specifically:
(1) Residents and faculty must have the opportunity to
evaluate the program confidentially and in writing at least
annually, and
(2) The program must use the results of residents’
assessments of the program together with other program
evaluation results to improve the program.
2. If deficiencies are found, the program should prepare a
written plan of action to document initiatives to improve
performance in the areas listed in section V.C.1. The action
plan should be reviewed and approved by the teaching faculty
and documented in meeting minutes.
Graduate performance: Results of performance on
board certification examinations is one measure of
graduate performance. Data can also be collected by
annual surveys of graduates. Typically, such surveys
target physicians one year and five years after
graduation. Forms used may be provided by the
institution, developed locally or adapted from the
published literature (or unpublished but available
online). Survey questions may inquire about such items
as current professional activities of graduates and
perceptions on how well prepared they are as a result of
the program.
REMEMBER:
Can, Should, Must…For ACGME all these MEAN:
Thou Shalt Do These Things or Be Cited
Surveys in Common Use
• ACGME Annual Resident Surveys
– Common Survey; Specialty Specific Survey
• Exit Surveys-currently done by GME at
Graduation
• Post-NRMP applicant surveys
• Alumni Surveys
Alumni Surveys
• When
– 1 year, 5 year, after fellowships (1-3 yrs)
• Who
– Graduate or Employer
• How
– Traditional Mail
– Email
– Web-based Survey tools (eg. Survey Monkey)
Other uses of Alumni Surveys
• Assess CME Needs
• Tie Alumni closer to program
– Maintain accurate contact info
– Provide database to solicit funds for education
– Provide job contact network for current
Residents
Structure of a Survey
• General Demographics
– Name, Office/Home addresses, email
• Type of Practice
• General Questions
– Open ended-comments
– Likert Scale (strongly agree/disagree)
– True/False (probably least information)
• Competency Based Questions?
Competency Based Survey
Open Ended Survey
Objective Specific Survey
Pathology Trainee Post-Graduation Survey
Anatomic and Clinical Pathology Resident Training Program - VCU Department of Pathology
As part of our effort to continuously improve the quality of training we provide, we ask you to please respond to the following
questions about your residency experience.
1. How do you rate the training you received as a resident/fellow in each of the following areas?
1=poor
2=fair
3=good
4=excellent
Autopsy
1 2 3 4 N/A
Hematopathology
1 2 3 4 N/A
Immunohistochemistry
1 2 3 4 N/A
Transfusion Medicine
1 2 3 4 N/A
Surgical Pathology
1 2 3 4 N/A
Microbiology
1 2 3 4 N/A
Neuropathology
1 2 3 4 N/A
Molecular Pathology
1 2 3 4 N/A
Cytopathology
1 2 3 4 N/A
Cytogenetics
1 2 3 4 N/A
Clinical Chemistry
1 2 3 4 N/A
Forensic Pathology
1 2 3 4 N/A
Toxicology
1 2 3 4 N/A
Informatics
1 2 3 4 N/A
Laboratory Management
1 2 3 4 N/A
2. How well prepared were you for the practice of pathology upon completion of your residency?
Not well-prepared_____ Somewhat prepared_____ Very well-prepared_____
3. In what area(s) did you feel your preparation was weakest?
4. In what area(s) did you feel your preparation was strongest?
5. What specific recommendations do you have to improve the quality of training in this program?
6. Year completed residency at VCU_____
7. Year completed fellowship at VCU_____
Send Me Money
Survey Points
• Ask open ended questions or incorporate comment boxes
• Don’t make your survey so long that responders give up
• Consider a short “preamble” to explain the importance of a
response
• Thank the responder for taking the time to complete the
survey
• Follow up, when appropriate with direct communications
Input from the Coordinators
Name_____________________
Grad Yr. ______
Address_________________________ Home___Offi ce__
City/State/Zip____________________
Email __________________________
Current Practice (Type, Locatio n, Yrs in practice)
Business Contact Info
“Did your career path foll ow your original plan or
what you had discussed with your mentor?”
Do you work full time or part time (FTE)
Are you in an a cademic practice?
What are your primary responsibiliti es?
W/In your institu tio n to you teach: med students,
resid ents, post-grads
Board Certifi catio n: did you take the certif yin g exam y /n
Did you pass on 1st attempt ___ 2nd ___
Date of init ial Certifi catio n ________
Did the Program ad equately prepare you for your exams?
Did the Program ad equately prepare you for your
practice?
How us eful were these activi ties to your success in
the program:
Journal Clubs
Lectures
Grand Rounds
M & M conferences
Workshops
If you work in the region, are you will ing to be a
preceptor/mentor to fut ure Residents?
Is your group lookin g to hire? Would you consider hiri ng
current or fut ure residents from this Program?
WHAT ELSE???
What to do with data?
• Obtain and Assess Data
• Formulate an action plan
• Compare any new data against previous action
plans
• Develop new programs to address deficiencies
delineated in the data…
• And DOCUMENT that you did this
– Who was present in meeting
– When did you meet
– Outcome products of the meeting
• Minutes, Report to Faculty, New Action Plans