The Safety Health of Maintenance Engineering (SHoMe) Tool

Steve Mason Principal Human Factors Consultant HSEC Ltd

Error Types & Action Strategies



Cannot physically achieve goal

- improved ergonomics



Slips & Lapses + Distractions

- design & interlocks



Mistakes

- training, job aids



Violations



safety culture + organisational factors

Existing Guidelines/Tools

 Maintainability Design Handbook  Bretby Maintainability Index  HFRG Guide to reducing violations  HFRG Guide to reducing maintenance error  HSEC Human Factors Solutions  SHoMe

Reactive, Proactive, or Both?

INCIDENT REVIEWS Strengths:  Information available from incident investigations  Based on factual evidence  Cost of accidents/incidents can be determined Weaknesses:  Root issues not always well documented  Often insufficient incidents for analysis  Data relates to the past – may not be relevant to the future QUESTIONNAIRES Strengths:  Quick to administer  Findings reflect current attitudes  Involves staff identifying areas of concern  Can promote ‘ownership’ Weaknesses:  Findings based on perceptions may be influenced by range of factors unrelated to maintenance  No factual information to gauge ‘spend’ justified on expected improvements

SHoMe Basic Structure

Safety Health of Maintenance Engineering Evidence of Non-Compliance •E.g: risk taking, errors through tiredness, supervisors condoning unapproved actions, etc Potential Root Issues affecting Compliance •Factors known to have caused problems •Factors that could cause problems Part(s) of Job Causing Most Difficulties

SHoMe Staff Groups

Technical Certifying Staff Technical Non-Certifying Staff Management & Technical Support Staff (e.g. planners, )

Questionnaire Administration

Staff Group Tech Cert Generic Q  version C Job Difficulty  Org Q  Tech Non-Cert Support Staff  version NC  version NT  N/A  N/A

Questionnaire Administration

•Avoid issuing questionnaires & collecting them in at the end of a shift – response by committee!

•Preferably – organise small groups, lead with introduction my management (& unions if possible) •Stress anonymity and purpose of the surveys •Emphasise that feedback will be given & acted upon (if appropriate) •Please: ensure correct question set is issued

Example Generic Questions

Please indicate the extent to which you agree or disagree with the following statements in relation to your work over the last 6 months of so

Management and staff communicate well with each other Managers always let us know of important safety findings I am always properly briefed by those giving me a job Before I start a job I am always given the necessary information I know exactly what I am expected to do and my responsibilities SA A ?

D S D

Extract Job Difficulty Questionnaire PLANNING PREPARATION INSPECTION ROUTINE WORK A Was this part of your jobs?

B No Problems Y/N?

 C Some Problems D Major Problems Y/N?

Y/N?

  Y/N?



Extract from Organisational Questionnaire The ease with which general written procedures can be understood The amount of jargon and ambiguity contained in procedures Ambiguity as a result of different layouts of different types of forms you use Systems for implementing temporary revisions to maintenance manuals Previously encountered problems not highlighted The ability of planners to minimise jobs running over other shifts The amount of work your immediate boss expects you to do in a shift The way management treat individuals who report mistakes or safety concerns

Prioritising Results

Ranked Evidence of Non-Compliance Ranked Major Root Issues [19] Ranked Human Factors Issues – Generic Q Ranked Issues – Organisational Q

19 MAJOR H.F. ISSUES

 Design & Maintenance Interface  Provision of Resources  Training  Fatigue  Complacency  Planning  Communications  Commercial Pressures  Maintenance Procedures  Roles & Responsibility  Management Attitudes  Safety Commitment of Staff  Job Pressure  Working Conditions  Fair-Blame Culture  Management of Change  Supervisor Effectiveness  Supervisor Attitudes  Competence

Results Presentation

Level 1 Non-Compliance: Generic Q Summary of Major Root Issues only Level 2 Non-Compliance: Generic Q Expanded Major Root Issues : Generic Q Level 3 Expanded Non-Compliance: Generic & Org Q Expanded Major Root Issues : Generic & Org Q MEDA Results presented against relevant MEDA structure

HSEC SHoMe Tool Version 1.0

September 2003  This tool was developed by HSEC Ltd, on behalf of the UK CAA  The tool may be freely copied, distributed and used for flight safety purposes, but not resold for commercial purposes  The SHoMe tool must be used in conjunction with the user manual  The software is not supported

Scoring Criteria

GENERIC QUESTIONNAIRE: Good Worthy of Attention Concern < 35 > 55 > 65 ORGANISATIONAL QUESTIONNAIRE: Probably Acceptable Concern < 15 > 30

EVIDENCE OF NON-COMPLIANCE

Mean Score Completion of job despite the non-availability of appropriate tools and equipment Regular non-compliance by the workforce Aircraft released with work not done due to parts shortages Direct pressure to deviate from procedures Reports of colleagues making errors due to tiredness with last month Direct pressure from the supervisor to deviate from procedures Pride on getting aircraft back on time, even if this needs some non-compliance Supervisors taking risks in releasing aircraft when maintenance incomplete Supervisors condoning unapproved actions to get an aircraft away 71 36 34 26 22 61 50 43 41

POTENTIAL ROOT ISSUES Design & Maintenance Interface Planning Fatigue Provision of Resources Commercial Pressures Working Conditions Training Roles & Responsibility Score 57 51 51 50 49 49 47 41

POSITIVE FEATURES Supervisor Attitudes Competence Supervisor Effectiveness No-Blame / Fair-Blame Culture Management of Change Complacency Safety Commitment of the Engineers / Staff Procedures – Accuracy, Relevance & Practicality Scores 36 38 39 39 29 30 31 34

Job Difficulty

Using SHoMe data

Show Differences between Cert, Non-Cert, Support Staff ‘Big Picture’ – are there any major concerns?

Typical Questions

Are there any significant differences between those who have worked in the Industry for > 20 years and those new to the Industry?

Are the problems with contract staff significantly different from those with permanent staff?

Differences within one company with staff at two sites.

Differences between those working long hours.

SUMMARY

Human Factors Issues are important & need to be managed SHoMe does not rely on incidents/near misses SHoMe requires investment in time by all staff Hopefully – we agree it complements existing tools Freely available + option for Industry benchmarking