Transcript No Slide Title

Evaluating complex interventions
(or the tribulations of trials)
Prof. Rowan Harwood
Health Care of Older People
Nottingham University Hospitals
What we may be witnessing is
not just ... the passing of a
particular period of post-war
history, but the end of history as
such...
That is, the end point of
mankind's ideological evolution
and the universalization of
Western liberal democracy as
the final form of human
government.
The end of Medical History
John R Hampton
Examples of complex services for older people
• Rehabilitation
(stroke, community hospitals, intermediate care)
• Most psychiatry
• Falls services
• Continence services
• Care home and long term care interventions
• Education, training, guidelines, care pathways, service
improvement
Complex interventions
intervention … comprising a number of separate elements
which seem essential to its proper functioning;
components … may act independently and inter-dependently
usually include:
• behaviours
• parameters of behaviours (frequency, timing)
• methods of organising and delivering those behaviours
(types of practitioner, setting and location).
MRC 2000
Plan
• Briefly describe 4 trials
• Highlight design, implementation, analysis and
interpretation issues
• Consider design compromises in a new trial of a
complex intervention constrained by service demands
Stroke Unit Evaluation 1991-3
• Operational since 1982, 15 beds
• Restrictive inclusion criteria
• 315/1760 (18%) stroke patients included
• SU median 76 days, standard care 54 days
• Follow up: 275 (87%) at 3 mo; 241 (77%) at 12 mo
• At 12mo:
• Died 14% vs 19%
• Home 65% vs 62%
• Barthel 17 vs 16
• Rivermead ADL 12 vs 9
• NEADL 7 vs 4
• GHQ 17 vs 20
• psychological adjustment better at 3 & 6 months
Stroke Unit Evaluation issues
• Keep unit full (changed to 60:40 randomisation)
• Blinded assessment
• Loss to follow up (23% at 12 months)
• Contamination
• Power
• Multiple outcome measures
• … but what is a stroke unit?
• … why the difference in outcome?
SUE observation of process
• Non participant observations of 76 patients
• 3x8h shifts over 3 days, 10min intervals
• location
• rehab 6.4% vs 3%; bedside 61% vs 65%; social 5% vs 2%
• activity
• isolated disengagement 15% vs 24%
• positioning -
poor 13% vs 30%
• contacts
• more time with nurses, therapists; less time with doctors, visitors
• rehab time (on ward) 46 min/d vs 23min/d
Community Stroke team
• Established multidisciplinary service
• 428 referrals randomised (50:50 then 40:60) over 2y
• Group A: routine care including day hospital or OP (n=189)
• Group B: CST (n= 232). Median 18 sessions (range 0-115, IQR 1042). Mean 4.8 h PT; 3.8h OT; 2h SLT, 1.9h MH nurse; 0.5h Rehab
Support
• Six months later, contacted by letter, requested consent to
follow up, postal outcome questionnaire
• Consent procedure:
• avoid perception of ‘missing out’ by control group
• reduce team loyalty bias
• logistically easier
• Offer of assistance or visit to complete questionnaire
Community Stroke Service trial: issues
Outcomes at 6 months
Patient
Barthel/20
NEADL/44
GHQ/24
EQ5D/100
Satisfaction
emotional support/4
overall/4
CST vs standard
16 vs 16
24 vs 26
13 vs 15
52 vs 55
3 vs 2 (CST better)
3 vs 2 (CST better)
Carer
GHQ/24
Carer strain/12
EQ5D
satisfaction
knowledge/4
overall/4
13 vs 15
8 vs 10 (CST better)
73 vs 75
3 vs 2 (CST better)
3 vs 2 (CST better)
Community Stroke Team: problems
• Low and differential follow up
(47% vs 43%)
• No baseline data
• No record of routine services
• Time but not content of CST intervention recorded
• Was the service successful or not?
• If this didn’t work what could …?
EDRS: service structure
A new EDRS was established in 1998.
9-month development phase.
Staffing: 2 OTs, 2 PTs, 3 nurses, 1 SW, 7 rehab assistants,
secretarial support. No doctors.
Visits for assessment, monitoring, rehabilitation, or assistance
and care.
Up to four visits per day, up to 7 days per week, 8am to 10pm.
Duration up to 4 weeks.
EDRS: randomised trial
• 370 elderly medical, stroke and surgical patients
• Randomised between EDRS and conventional care
• Mean 22 visits per patient over 4 week
• Mean 7 days shorter hospital length of stay
EDRS: outcomes
Qualitative evaluations
• Exploratory, in depth
• Subject to ‘rules’
• Identify:
•
•
•
•
what an intervention comprises in practice
how an intervention is useful
how it achieves its effect
the conditions necessary for its effect
• Introduce rigour into otherwise potentially anecdotal
evidence
EDRS: Qualitative study
• Patients interviewed:
• before hospital discharge
• after 4 weeks
• after 3 months.
• Staff responsible interviewed 6–8 weeks later
• 64 interviews of 20 patients
• 15 interviews of 11 staff
Qualitative study
• Before discharge
• Participants anxious about return home, but keen to do so
• Felt vulnerable, feared not being able to cope, or being a burden
on families.
• After discharge
• Patients felt they had improved
• … but described pain, loneliness, fatigue, reduced mobility,
problems climbing stairs or getting outdoors.
• Lack of confidence and anxiety about activities
Qualitative study
• EDRS delivered: skilled assessment, negotiated treatment goals
meaningful to the patient, and demonstrated team working.
• Interventions: functional rehabilitation, teaching skills,
information giving, advice, overcoming emotional barriers, provision
of aids and appliances and provision of personal and domestic care.
• Emotional support: improved confidence and morale, and led to
greater task performance and psychological well-being.
Delirium prevention trial
• 1 intervention ward, 2 standard care wards
• Multi-component delirium prevention regimen
• hydration
• vision and hearing
• orientation
• mobilisation
• non-pharmacological sleep promotion.
• RCT pilot failed - ‘120% bed occupancy’
• Prospective individual matching design
Prospective individual matching design
• Prospectively assessed:
2424 patients admitted to 3 wards
• Research staff
‘not involved in care’, ‘unaware of nature of
study, or patient group assignment’.
• Over 70, at intermediate or high risk of delirium
• Recruited all patients on intervention ward
• Matched these with standard care patients
•  5 years
• sex
• baseline risk of delirium
• U&E, visual and cognitive impairment, APACHE-II.
Exclusions
• Assessed 2424 patients
• Excluded 1265
• severe dementia 154
• aphasia 38
• no English 92
• coma or terminal illness 69
• LOS<48h 219
• other incl. no interviewer or patient unavailable 355
• Declined 250
• No match 67
(24 intervention, 43 standard)
• Recruited 852
(426 intervention, 426 standard)
Results
Issues
• Able to do research interviews on all at admission
• Demonstrated power equivalence of RCT
• Recruitment
• Demonstrated good matching at baseline on 32
variables
• Contamination
• Bias and blinding
MRC 2000 framework
MRC 2008
NIHR Medical and mental health unit
controlled clinical trial
Medical and mental health unit: development
• Initial 250 patient register/cohort study.
• Multi-professional operational group
• Additional staff: RMNs and MH specialist OT/PT
• Programme of staff training
• Advice from MH service
• Environmental changes
• New approach to families
• Develop policies and new working practices
Medical and mental health unit
‘sufficiently different from standard care to have
a reasonable chance of demonstrating different
outcomes in a trial’
Medical and mental health unit: controlled clinical trial
• Modelled on stroke unit trials
• Compared with ‘standard care’
• Starting May 2010 (pilot); July 2010 (main trial)
• up to 2 year recruiting, 3 months follow up
• Target n=240 per group
• Effectiveness, economic and qualitative studies:
• not just does it work, but how does it work?
• is it value for money?
• Service funding ceases April 2013
Medical and mental health unit: controlled trial
Outcomes at 3 months:
• number of days spent at home:
• length of stay, readmissions, deaths, care home placements
• number of days in care home for those starting there
• scales:
• DemQoL, NPI (behaviour), Barthel, EQ5D, London Handicap
• Carer strain and psychological wellbeing
• resource use and costs
• non-participant observer study
Medical and mental health unit: recruitment and randomisation
Constraints
• No waiting on AMU (for assessment/recruitment/consent/MMHU bed)
• No empty beds on MMHU
• Keep MMHU full with suitable patients
• Minimise ward moves for confused patients
• Likely to lose trial beds to non-randomised patients
Medical and mental health unit: Recruitment and randomisation
Randomisation prior to consent, recruit to follow up later.
• ‘Confused, over 65’
• Web based stratification (care home or not)
• Transferred to MMHU or standard ward
• Once on ward recruited to trial (consecutively or randomly)
Other issues
• Need to persuade AMU to do something of no benefit to them
• Majority lack capacity to consent: effort and delay
• 10% lack a consultee
• 40% of cohort declined
CLRN/MHRN to the rescue
• Initial plan was limited by availability of researchers to
recruit and follow up
• Current plan to recruit all randomised patients who agree to
take part
• Option to collect primary outcome on those declining full study
Summary and lessons for the triallist
• The RCT can be adapted to evaluate complex interventions
• Logistics can be tricky and compromises needed
• Interpretation is more difficult
• multiple outcome measures
• what is the outcome?
• does the outcome match the intervention?
• condition specific vs generic outcomes
• qualitative evaluation
• Look at the content of the intervention
• what is the intervention?
• how different it is from the comparator
Complex interventions
Watch this space!
Thank you.