Transcript Autism SIG 7th March 2007

NAS Conference 2008
Greg Pasco & Kate Gordon
THE EFFECTIVENESS OF PECS STUDY:
FINDINGS AND IMPLICATIONS FOR
FUTURE RESEARCH
1
The Effectiveness of PECS
Study
2
PECS: The research context
3
Evaluations of PECS: What
next?
1
The Effectiveness of
PECS Study
• The largest published randomised trial of any
psychosocial intervention for children with
autism
• 84 children in 17 classes
• Classes randomly assigned to three treatment
groups
• All children assessed at baseline and two followup points
The Effectiveness of PECS Study
Assessment timetable
T1
T2
T3
Immediate Treatment
2-day PECS workshop
Monthly PECS consultation visits
Delayed Treatment
Approx 2 terms
No Treatment
The Effectiveness of PECS Study
Participant characteristics (at T1)
6:10
3:11 – 10:02
Age (y:m)
Mean
Sex
11 girls
Mullen Nonverbal
Age Equiv (m)
Median
23.5
Interquartile Range
ADOS Diag score
Median
16
Interquartile Range
ADOS Diagnosis
75 Autism
BPVS (Raw)
Median
1
Interquartile Range
0 – 11
EOWPVT (Raw)
Median
0
Interquartile Range
0 – 8.5
Range
73 boys
19.0 – 29.5
7 – 22
9 ASD
The Effectiveness of PECS Study
Outcome measures
Rates of:
Initiations
PECS use
Speech
observed in children’s snack times
ADOS Communication & Reciprocal
Social Interaction Domain scores
BPVS and EOWPVT raw scores
The Effectiveness of PECS Study
COSMIC
Classroom Observation Schedule to Measure
Intentional Communication
(COSMIC: Pasco et al, 2008)
All children videoed for 15 minutes during snack
All videos rated by GP and KG using COSMIC
Each interaction coded according to Form,
Function, Spontaneity, Communication Partner
The Effectiveness of PECS Study
COSMIC Outcomes
PECS use:
Distribution very positively skewed, so data
transformed into ordinal categories
120
100
N = 250
N = 250
100
80
No. of children
No. of children
80
60
60
40
40
20
20
0
0
0
1
2
3
4
rate of PECS use (per min)
5
0
1
2
PECS use ordinal categories
3
The Effectiveness of PECS Study
Analysis
Hierarchical Ordinal Regression:
Accounted for the fact that data were clustered by
individual and by class, and in a time sequence
Controlled for a number of baseline variables – Age,
Mullen NVDQ, ADOS Language Level
Tested for immediate treatment effects and
maintenance (for ITG)
The Effectiveness of PECS Study
Hierarchical ordinal regression
T1
T2
Immediate Treatment
Maintenance of treatment effects
Delayed Treatment
Immediate treatment effects
No Treatment
T3
The Effectiveness of PECS Study
Results
COSMIC DVs:
Initiations
Immediate: Odds Ratio 2.73 (p<.05)
Not maintained at 10-month follow-up: OR 1.08 (p=.91)
PECS Use
Immediate: OR 3.90 (p<.001)
Not maintained at 10-month follow-up: OR 1.56 (p=.48)
Speech
Immediate: OR 1.10 (p=.83)
The Effectiveness of PECS Study
Results
Other DVs:
ADOS Communication
Immediate: OR 0.52 (p=.10)
ADOS RSI
Immediate: OR 0.55 (p=.13)
But, delayed treatment effect (?) OR 0.28 (p<.05)
EOWPVT
BPVS
Immediate: OR 1.01 (p=.87)
Immediate: OR 1.54 (p=.44)
The Effectiveness of PECS Study
Results
100
N = 250
What does this mean?
No. of children
80
60
40
20
Immediately following the intervention
period, children in the treatment groups
were more than 2½ times more likely to be in a
higher initiation ordinal category, and nearly 4
times more likely to be in a higher PECS use
ordinal category, than they were at other time
points, and in comparison to the NTG at all time
points. These effects were not maintained 10
months after the intervention period ceased
0
0
1
2
PECS use ordinal categories
3
The Effectiveness of PECS Study
Results
And…
There were no benefits for children’s rates of
speech
Immediate treatment effects did not generalise to
communication behaviours observed in a
structured context, nor in relation to scores on
standardised tests of receptive and expressive
vocabulary
There may be a longer-term benefit for children’s
reciprocal social interaction skills
The Effectiveness of PECS Study
Discussion
Clinically meaningful?
Pre- to post-treatment: from ~15 to ~26
initiations per hour and from ~12 to ~40
interactions involving PECS per hour
These reflect changes in children’s everyday
use of communication in the classroom
Does lack of change in speech reflect
duration or intensity of intervention or
participants’ skills?
The Effectiveness of PECS Study
Discussion
Strengths of study
Randomisation
Ecologically valid measurement of
communication
‘Real World’ context
Number of participants
Analysis
Intention to treat protocol
The Effectiveness of PECS Study
Discussion
Limitations of study
Observations restricted to snack times
No measure of treatment fidelity
Raters not blinded to treatment status,
time points, etc.
The Effectiveness of PECS Study
Paper
Howlin, Gordon, Pasco, Wade & Charman (2007)
The effectiveness of Picture Exchange
Communication System (PECS) training for
teachers of children with autism: A pragmatic, group
randomised controlled trial
Journal of Child Psychology & Psychiatry, 48(5),
473-481.
2
PECS: The research
context
Evaluations of PECS
The 4- (or 5-) step/phase model for establishing
the effectiveness of an intervention, based on
established medical research model
e.g. Robey & Schultz (1998)
Chambless & Hollen (1998)
Flay et al (2005)
Smith et al (2007)
Evaluations of PECS
The 4-Phase Model
1. Novel intervention applied to a small
sample, usually without controls. Practice
is refined. Does the intervention help, or
lead to adverse outcomes?
2. Formulation and standardisation of
intervention (i.e. developing a manual),
validating measurement instruments,
assessing factors predicting treatment
response, optimal ‘dosage’
Evaluations of PECS
The 4-Phase Model
3. Efficacy studies. Large samples, RCTs,
treatment fidelity measurement,
independent replication. Intervention
under ‘ideal’ conditions, often
implemented by experts
4. Effectiveness studies. Large samples,
controls and randomisation (but not
necessarily). Intervention under Real
World conditions, implemented by ‘typical’
professional – teacher, SLT, etc.
Evaluations of PECS
The 4-Phase Model
1
4
2
3
Evaluations of PECS
Phase 1
Bondy & Frost (1993)
Bondy & Frost (1994)
Relatively large samples. Descriptive
accounts of change
Evaluations of PECS
Phase 2
Manuals
Frost & Bondy (1994) – First manual
Frost & Bondy (2002) – Second manual
Evaluations of PECS
Phase 3
Single case/Case series
Webb (2000): N = 6 (5 with ASD). Increases
in use of PECS and speech for all children,
plus increases on Derbyshire DTC
Charlop-Christie et al (2002): N = 3. Multiple
baseline design. All children learned to use
PECS to Phase 6 and showed increases in
speech
Evaluations of PECS
Phase 3
Service evaluations
Schwartz, Garfinkle & Bauer (1998): N =
31 (16 with autism or PDD-NOS). Children
used PECS to communicate with adults
and peers by 14 months. N = 18 (10 with
autism). 8 developed spontaneous speech
over 1 year (No breakdown by diagnosis)
Evaluations of PECS
Phase 3
Controlled studies
Carr & Felce (2007a): N = 41. Researchers
delivered 15 hours of PECS training to
children, up to Phase 3. Observation of
child-adult interactions. Increases in childadult initiations and adult-child initiations
with opportunity for child response relative
to controls. Some evidence of increase in
speech (Carr & Felce, 2006b)
Evaluations of PECS
Phase 3
RCTs
Yoder & Stone (2006 a & b): N = 36.
Comparison of PECS and RPMT
(Responsive Education and Prelinguistic
Milieu Teaching). RPMT better at
facilitating turn-taking and joint attention,
PECS better for requests and spontaneous
speech (& spoken vocab). Rigorous
methodology, generalisation measures,
predictors of response to treatment
(conditional treatment effects)
Evaluations of PECS
Phase 4
Uncontrolled study
Magiati & Howlin (2003): N = 34. Pre/Post-intervention. Rapid acquisition of
PECS, increases in PECS vocabulary,
and frequency of PECS use. Relied on
report data
RCT
Howlin et al (2007)
PECS: The research context
“Does PECS work?”
Criteria: (see Rogers & Vismara (2008) re. early
interventions)
Chambless et al (1996 & 1998)
Well-established:
Manual
PLUS
2 independent well-designed studies
showing treatment > placebo or = a
proven treatment
OR
9+ single-subject studies with strong
designs and comparison to an
alternative treatment
PECS: The research context
“Does PECS work?”
Probably efficacious:
Clearly specified participant groups
(manual preferable but not required)
PLUS 2 studies showing better outcomes than
a no treatment control group
OR
2 strong group studies by the same
investigator treatment > placebo or = a
proven treatment
OR
3+ single-subject studies with strong
designs and comparison to an
alternative treatment
PECS: The research context
“Does PECS work?”
According to the Chambless criteria, PECS can
probably be described as “well-established”,
although note that studies have incorporated
varied outcome measures (acquisition of
PECS, use of speech, spontaneity…), different
settings and ages of participants
“very strong, positive
evidence”
3
Evaluations of PECS:
What next?
PECS is currently the only autism-specific
psychosocial intervention to have been tested to
the effectiveness phase by RCT
Not surprisingly, PECS appears to facilitate
spontaneous requesting, and may also have
additional benefits for developing speech
Evaluations of PECS
What next?
Clearly PECS is an appropriate intervention for
nonverbal children with autism, although (as with
other interventions) there is limited evidence to
say which children will benefit most
Further well-conducted independent studies
(small & large scale) are required to establish
efficacy and effectiveness in relation to different
settings, ages and diagnoses
Evaluations of PECS
What next?
More evidence is required as to what are
the most effective ways of providing
training & consultancy to develop best
practice in everyday settings and ensure
maintenance of benefits for children
Evaluations of PECS
What are the prerequisites to
becoming a successful PECS user?
Christina Tohill, SLT: MSc Project. 23 initially
nonverbal children who received intensive PECS
training in special school setting. 16 made
significant progress, whilst 7 did not move
beyond Phase 3 (i.e. unable to discriminate). All
children had PEP-R assessments around time of
introduction to PECS
What distinguished the successful from the
unsuccessful children?
Evaluations of PECS
What are the prerequisites to
becoming a successful PECS user?
Nonverbal skills?
Mann-Whitney U = 53.0
(p=.82)
Evaluations of PECS
What are the prerequisites to
becoming a successful PECS user?
Perception skills?
Mann-Whitney U = 7.0
(p=.001)
Evaluations of PECS
What are the prerequisites to
becoming a successful PECS user?
Overall cognitive skills?
Mann-Whitney U = 0.5
(p<.001)
We don’t know what this
score on PEP-R equates to
in terms of “actual”
cognitive ability
Need to do a prospective
study to test this out further
Acknowledgements
We would like to thank all the children who
participated in the Effectiveness of PECS
Study, their parents and teachers
Sue Baker and Teresa Webb and
colleagues at Pyramid
The study was funded by the Three
Guineas Trust
Thank you