Targeted Methods for Obtaining Feedback on Your EH&S Program

Robert Emery, DrPH, CHP, CIH, CSP, RBP, CHMM, CPP, ARM Vice President for Safety, Health, Environment & Risk Management Associate Professor of Occupational Health

Change in Focus

 EH&S programs have changed in recent years – Originally “command and control”, regulatory driven – Now service oriented, with goal to support the organizational goals

Soliciting Feedback

 As part of this service orientation, reliance on feedback is crucial  Feedback is typically obtained in four ways: – – Unsolicited: usually complaints, rarely compliments Training courses: smile sheets – – A few generic questions as part of a larger organizational survey Passive link on website: “let us know how we did”

Soliciting Client Feedback

Soliciting Client Feedback

Soliciting Client Feedback

Previous Efforts

     Previous client satisfaction work at UTHSCH focused on routine safety surveillance program Intended to evaluate staff performance in 5 persistently problematic areas – Interruptions, discourteous, – unknowledgeable, not technically proficient, – and waste not picked up Results overwhelmingly positive >90% approval ratings Unanticipated results – written comments: “thanks for asking!” Powerful tool for demonstrating program goodwill value to upper management

Major Challenge

 Feedback from surveys can be skewed or misleading if client expectations are not understood first  The trick is to first understand what client expectation are, and then to conduct operations accordingly

Two Types of Client Expectations



Realistic

expectations that are perceived as not being achieved – Solution: recalibrate operations to meet expectations 

Unrealistic

expectations that can never be met – Solution: educate client so that expectations can be adjusted

Measuring Expectations and Perceptions

 SERVQUAL tool  Developed by Parasuraman et al. under the auspices of the Marketing Science Institute  Research shows that customers evaluate firms by comparing service performance (perceptions) with service expectations

Five Dimensions of Service Quality



Tangibles

– appearance of staff, facilities 

Reliability

reliably – ability to perform promised service dependably and 

Responsiveness

service – willingness to help clients and provide prompt 

Assurance

– knowledge and courtesy of staff which instills trust and confidence 

Empathy

– caring, individualized attention

SERVQUAL Tool

 22 paired statements split into two sections – – Expectations Perceptions  Example: “When excellent cable TV companies promise to do something by a certain time, they will do it”  Each statement evaluated on a 7 point Leikert scale  Data summarized and graphically displayed, comparing expectations versus perceptions

Methods

 Modified questionnaire developed, consisting of 7 paired statements about EH&S program services – Areas of concentration: reliability, responsiveness, assurance, empathy  Distributed to 280 principle investigators in paper form with a personalized, signed cover memo  Survey form pre-labeled for return via campus mail

Results

 By the end of a 3 week period, 32% return rate  Data entered into a spreadsheet and displayed graphically  Overtly display to various stakeholders!

Figure 1. U THSCH laboratory personnel rating of the importance of certain service characteristics of "excellent" environmental health & safety programs, compared to their rating of the level of service currently being provided by UTHSCH EH&S. 7.0

6.0

Excellent EH&S Programs 5.0

4.0

3.0

2.0

1.0

0.0

P ro m ise to d o S in ce re P ro m pt C ou rte ou s K no w le dg ea ble C on ve nie nt H ou rs U nd er st an d N ee ds

Figure 1. U THSCH laboratory personnel rating of the importance of certain service characteristics of "excellent" environmental health & safety programs, compared to their rating of the level of service currently being provided by UTHSCH EH&S. 7.0

6.0

5.0

Excellent EH&S Programs UTHSCH EH&S 4.0

3.0

2.0

1.0

0.0

P ro m ise to d o S in ce re P ro m pt C ou rte ou s K no w le dg ea ble C on ve nie nt H ou rs U nd er st an d N ee ds

Benefits

 Results provide tangible evidence of program meeting the expectations of an excellent EH&S program  Powerful leverage tool to gain needed resources  Written comments identified other areas of concern  Great for staff morale – feedback for work rarely acknowledged

Other Examples

• Clients of Radiation Safety Program • Clients of Chemical Safety Program • Clients who interact with Administrative Support Staff • Employees and Supervisors Reporting Injuries • Clients of Environmental Protection Program Services • Determining the Level of Informed Risk

Administrative Support Staff Survey Results Summary

distributed to 90 targeted faculty and staff clients across UTHSCH, with 54 responses in 30 days (60% response rate) Questions Phone answered within 3 rings?

Timely response to inquiries?

Courteous response?

If couldn’t answer, offer suggestions or alternatives?

Yes 78% 93% 94% 66% No 2% 2% 3% 2% N/A 20% 8% 3% 34%

Administrative Support Program Client Satisfaction Survey (distributed to 90 targeted faculty and staff clients across UTHSCH, with 54 responses in 30 days (60% response rate)

“ 7) Compared to other administrative personnel you interact with across UTHSCH, please indicate your impression of the level of proficiency of the EH&S Administrative Support Staff member demonstrates during your interaction with them”

50 45 40 35 30 25 20 15 10 5 0 Very low prof iciency Low prof iciency Averge prof iciency Somew hat prof icient Very prof icient

Survey of Employees and Supervisors Filing UTHSC-H First Reports of Injury in 2007 (Email based Zoomerang survey for period February 2007 to August 31, 2007)

Employee Population (not reporting any injuries, n = 4,181) Injured Employees Requiring Care and Lost Time (n = 39): Not Included in survey, as each injured worker that accrues lost time is assigned a case manager to personally assist in the rehabilitation process.

Employees requiring care, but no lost time (n = 28) Employees not requiring care, no lost time (n = 179) Was this the first time you have reported an injury or exposure at UTHSC-H?

Prior to the recent reported injury event were you aware of your obligation to report any injury or exposure?

Did you receive a copy of the completed first report of injury form?

To your knowledge has the source of your injury or exposure been addressed?

Did you encounter any issues with the reporting process that you didn’t know or anticipate?

Our records indicate that you did not receive any health care in response to your injury or exposure. Who made the determination that health care was not needed?

Have you experienced any residual affects from your injury or exposure?

Where did you access health care?

Please indicate your impression of the level of service provided by the health care provider who addressed your injury or exposure?

Were you able to easily access the necessary Supervisor's First Report of Injury form?

If any assistance was needed in order to complete and submit the Supervisor's First Report of Injury form, was this assistance readily available?

Were you provided with the information needed for you to effectively manage the affected employee?

No Care or Lost Time (18% response rate) 67%(Y) 33%(N) 88%(Y) 12%(N) 70%(Y) 30%(N) 81%(Y) 19%(N) 12%(Y) 88%(N) 72% Yourself 9% Supervisor 19% Other 9%(Y) 91%(N) Care But No Lost Time (57% response rate) 62%(Y) 38%(N) 88%(Y) 12%(N) 62%(Y) 38%(N) 88%(Y) 12%(N) 38%(Y) 62%(N) 12%(Y) 88%(N) 53% Employee Health 20% Student Health 27% Other 38% Very Good 44% Good 6% Average 0% Poor 12% Very Poor Supervisors (13% response rate) 37%(Y) 63%(N) 96%(Y) 4%(N) 96%(Y) 4%(N) 88%(Y) 12%(N) 27%(Y) 73%(N) 92%(Y) 8%(N) 46% (Y) 8% (N) 46% (none needed) 100%(Y) 0%(N)

Survey of Principal Investigators Utilizing EH&S Biological Safety Program Services

Email based survey distributed from 4/29/2010 to 6/2/2010 to 210 Principal Investigators identified as utilizing biological safety services in FY 2010. Survey response rate: 47 out of 210 (22%) Survey Question Responses Yes No No Opinion 1.

2.

3.

4.

5.

Do you feel the Biological Safety Program understands your needs and requirements as a faculty member or researcher? Do you feel you have adequate access to the Biological Safety Program via 44 (94%) 3 (6%) 0 (0%) 47 (100%) 0 (0%) 0 (0%) phone and/or email? Do you feel the Biological Safety Program responds to your requests in an 46 (98%) 1 (2%) 0 (0%) acceptable time frame? Do you feel the Biological Safety Program has adequate professional knowledge 43 (93%) 2 (4%) 1 (2%) to address your needs related to biological safety? (n= 46 responses) Do you feel the Biological Safety Program provides helpful and courteous service? 46 (98%) 1 (2%) 0 (0%) Are you able to obtain assistance if you are having issues submitting an Institutional 34 (72%) 1 (2%) 12 (26%) 6.

7.

8.

Biosafety Committee protocol, renewal, or update? In your opinion, do you feel that accessing the Institutional Biosafety Committee 31 (66%) 4 (9%) 12 (26%) protocol submission forms online is convenient? Does the online Institutional Biosafety Committee protocol submission process 22 (49%) 7 (16%) 16 (36%) provide adequate instructions for completion of the forms? (n=45 responses) Do you feel the online protocol submission system allows for easier initial 26 (55%) 5 (11%) 16 (34%) 9.

submissions, updates, and renewals of Institutional Biosafety Committee protocols as compared to the previous paper-based process? Better Same Worse No Previous Experience 10.

If you have been involved with Biological Safety Programs 21 (45%) 6 (13%) 1 (2%) 19 (40%) at other institutions, please rate how the service provided at UTHealth compares?

Summary

 Institutional EH&S programs are service intensive operations  Important to understand client expectations before measuring satisfaction  Formal surveys quantify intangibles   Other possible applications surely exist Great way to capture and display program’s goodwill value!

References

 Emery, R.J., Sawyer, R.L., Sprau, D.D., "Assessing the Service Provided by an Institutional Radiation Safety Survey Program"

Health Physics

, 70(5): 741-743, 1996.

 Emery, R.J., Savely, S., "The Benefits of Actively Soliciting Worker Concerns During Routine Safety Inspections"

Professional Safety

, 42(7): 36-38, 1997.

 Emery, R.J., "Adding Value to Your Radiation Protection Program", Chapter in Roessler, C.E.

Management and Administration of Radiation Safety Programs

, Medical Physics Publishing, Madison, WI. 1998.

 Parasuraman, A., Berry, L.L., Zeithaml, V.A., Guidelines for measuring service industry quality.

Marketing Research

, American Marketing Association., December 1990

UTH EHS