Transcript Template - UHN Presentation

Who’s Listening?
Quality Assurance and Phone Interpretation
Elizabeth Abraham, M.A., M.Sc., C.Tran.
Manager, Interpretation and Translation Services
Grace Eagan, M.A., Ph.D. (ABD)
Project Coordinator, Interpretation and Translation Services
University Health Network
© UHN Interpretation and Translation Services
Outline
• Project Background
• Development of RFP Requirements
• Performance Testing
– Purpose, Methods, Results, Conclusions
• Recommendations for integrating QA
into RFP process
Background
• Regional Health Authority of downtown
Toronto: 18 hospitals, home care services,
170+ community agencies
• Bulk purchase of over-the-phone
interpretation (OPI) services
• UHN operational lead, working with
independent procurement services firm
(Plexxus)
• Creation of HSP “Consortium”
Development of RFP Requirements
OPI Bible
OPI Guru
• Feedback from Consortium members
• OPI / F2F data from previous fiscal year
RFP Requirements (cont.)
• Expandable contract
• Per-minute rate reduces as volume
increases
• Trained and tested interpreters
• Specific language requirements:
– European / Azorean Portuguese
– Canadian French
– Aboriginal languages spoken in Ontario
Performance Testing: Purpose
• To determine whether proponents could
deliver services as set out in proposal
• Quantitative and qualitative data
• Fair and transparent process
• Controlled and consistent conditions for
each test call
• All calls recorded
Performance Testing: Methods
1. Mock Calls
• 10 scripts, vetted by clinicians
• 10 Volunteer Clinicians
• 22 interpreters playing “patient”
• Feedback forms completed after each
call (137)
2. Connection time calls (51)
3. Language identification calls (6)
Performance Testing: Mock Calls
1. Mini Mental Status Examination
2. PT/OT Assessment
3. Discharge Planning
4. Medication Reconciliation
5. In-patient Assessment upon Transfer from ED
6. ED Nurse Triage Assessment
7. ED Medical History Interview
8. Consent and Screening for MRI
9. ESAS Assessment
10.Pre-op Patient Verification Assessment
Performance Testing: Data
•
•
•
•
Feedback from clinician and “patient”
Number of rings to pick up
Intake duration
Connection time (end of intake to
interpreter)
• Ability to identify language
• Interference/Sound quality noted
Performance Testing Database
Performance Testing Database
Performance Testing Database
Interpreter’s Feedback
Performance Testing Schedule
Performance Testing Schedule
Time of day
Sunday
May 6
Monday
May 7
Tuesday
May 8
00:00 to
08:00
14:00 – 16:00
Mock Scenario:
Discharge Planning
Italian
Spanish
Punjabi
Hindi
Vietnamese
08:00 to
17:00
09:30 - 11:30
Mock Scenario: PT
Assessment
Cantonese
Mandarin
Polish
Hungarian
Portuguese
14:00 – 16:00
Mock Scenario:
ESAS Assessment
Portuguese
Punjabi
Vietnamese
Urdu
Hindi
16:00 – 18:00
Connection Times
All
languages
14:00 – 16:00
Mock Scenario:
Discharge Planning
Italian
Spanish
Punjabi
Hindi
Vietnamese
16:30 – 18:30
Mock Scenario: IP
Transfer
Cantonese
Portuguese
Punjabi
Hindi
Mandarin
17:00 to
24:00
22:00 – 24:00
(moved to May 9)
20:00 – 22:00
(moved to May 9)
Wednesday
May 9
06:00 – 08:00
Mock Scenario:
Medication
Reconciliation
French
Farsi
Tamil
Cantonese
Mandarin
08:30 – 10:30
Mock Scenario: Pre-Op
Patient Verification
Farsi
Tamil
French
Portuguese
Spanish
10:30-12:30
Connection Times
All languages
12:00-14:00
Mock Scenario: ED Triage
Nurse
Italian
Spanish
Portuguese
Cantonese
Mandarin
13:00 – 15:00
Mock Scenario: MiniMental Status Exam
Hebrew
Spanish
Tigrinya
Amharic
Tagalog
14:45-16:45
Mock Scenario: MRI
Consent and Screen
Spanish
Italian
Cantonese
Mandarin
Portuguese
15:30 – 17:30
Mock Scenario: ED
Medical History
Tagalog
Hungarian
Polish
Vietnamese
Greek
Thursday
May 10
11:00 – 12:00
Mock Scenario: ED
Medical History
Tamil
Performance Testing: Results
• Based on Performance Testing, one
vendor stood out for quality of
interpretation and quality of service
• Errors of potential clinical consequence
were noted
• Unable to fill: 6% for each vendor
• Many interpreters were untrained and/or
failed to adhere to standards of practice
and professional ethics
Conclusions & Recommendations
• Not all OPI suppliers are monitoring
interpreters
• May be a significant difference between
proposal and performance
• Testing the system(s) and service is
necessary to identify best supplier
• Testing process must be well
documented
Questions?
Elizabeth Abraham
[email protected]
Grace Eagan
[email protected]