Developments & Issues in the Production of the Summary Hospital-level Mortality Indicator (SHMI) Health and Social Care Information Centre (HSCIC)

Overview

Overview of SHMI How SHMI was developed and is governed How issues are captured and handled How changes to the methodology are reflected and communicated

What is SHMI

Single hospital-level mortality indicator to be used across the NHS in England • reports mortality at trust level across the NHS in England using standard and transparent methodology • Excludes specialist, community and mental health hospitals Uses 2 linked administrative datasets • Hospital Episode Statistics (HES) • Office for National Statistics (ONS) death registrations Produced and published quarterly by the HSCIC First publication as an experimental Official Statistic in October 2011

Details of SHMI

Covers all deaths reported of patients admitted to acute, non-specialist trusts who either die while in hospital or within 30 days of discharge Indicates whether a trust’s mortality ratio is as expected, higher than expected or lower than expected Not suitable as a direct measure of quality of care on its own, it is a trigger for further investigation Recommended to be used as part of a range of more detailed indicators Contextual indicators published alongside to add some context to the interpretation of the SHMI • Other contextual indicator only available to trusts via previewer

Why we are producing SHMI

Review commissioned because of concerns about • different indicators in use, • the lack of consistency and • lack of clarity about the way some were being calculated Review looked at both technical & audience/use issues Following the recommendations from this review, the Department of Health committed to implementing the SHMI as the single hospital-level mortality indicator which could be adopted across the NHS National review of hospital summary mortality ratios (HSMR)

How SHMI was developed and is governed

A steering group to define the high-level requirements A detailed independent statistical modeling and analysis exercise carried out by ScHARR, University of Sheffield A technical group to agree on the specifications HSCIC commissioned to lead the continued development and improvement of the SHMI, working with a range of stakeholders as well as publishing on a quarterly basis Annual review through the Indicator Assurance Pipeline Process

Details of SHMI

SHMI



ObservedDe aths ExpectedDe aths

Observed Deaths • the number of patients who die following treatment at the trust Expected Deaths • the number of patients who would be expected to die on the basis of average figures for England, given the characteristics of the patients treated there Risk adjusted for age, admission method, sex, co morbidity and diagnosis group • calculated from a full logistic regression model in SAS using MLE

Using Statistical Process Control

Using statistical modelling (logistic regression and other generalised linear models), we develop a national baseline model which provides us with an risk estimate of patient outcome Providers who do not conform to the national baseline (with associated control limits) are indicating special cause variation This variation has not been explained by the baseline model, many possibilities for reasons why, but this warrants a follow-up

SPC Chart – Funnel Plot

Upper Control Limits England Baseline Lower Control Limits

Issues Capturing and Handling

Issues Log • • • Feedback captured from users Issues added to Issues Log Review and updates at every publication • Updates to issues announced through Methodology Changes Methodology Changes Publication • Changes reflected on publication • Documentation • Outputs • …

Issue 1: Over-dispersion

Issue Description • 2 distinct views from technical working group • 2 bandings published for October 2011 publication • Confusing & undermine the value of the SHMI Methodology Changes • 1 banding published from January 2012 publication although both control limits still published

Issue 2: Model convergence

Issue Description • 95 models converged out of 140 models • Complete/quasi-complete separation and ridging errors in dataset • Over and under estimating of events Methodology Changes • Use 3 years data for model build • Inclusion of year index for case-mix adjustment • Applied SAS in-built logistic regression options • Ongoing issue to be reviewed further

Issue 3: Palliative Care

Issue Description • A small number of acute non-specialist trusts have hospices within their organisation • Current guidance on coding for palliative care difficult to follow and subject to a variety of interpretations Methodology Changes • Contextual indicator • Ongoing issue to be reviewed further

Future Developments

Use of co-morbidity(s) and/or prior-morbidity(s) Additional contextual indicators • Weekend Mortality • Social Deprivation • … Use of more/all episodes • Identification of diagnosis from R-codes • Allocation of event to relevant diagnosis group • Calculation of co-morbidity(s) or prior-morbidity(s)

Summary

Overview of SHMI How SHMI was developed and is governed How issues are captured and handled How changes to the methodology are reflected and communicated Further details available on http://www.ic.nhs.uk/services/summary-hospital-level mortality-indicator-shmi

Questions & Answers

Clinical Indicators Health and Social Care Information Centre TEL: 0845 300 6016 email: [email protected]