Transcript SGS - Centre for Evidence Based Early Intervention

Initial validation of the Schedule
of Growing Skills (SGS)
Margiad Elen Williams
Bangor University
Content
•
•
•
•
•
Background
Validation process
Step one
Step two
Conclusions
Background
• Screening tools are used to identify children
with possible developmental delay to enable
subsequent more rigorous assessment.
• Quick, inexpensive, and easy to use.
• Should be as accurate as possible.
American Academy for Pediatrics
(2006)
• Published recommended psychometric criteria
that all screening tools should meet.
• Sensitivity – proportion of correctly identified
children in need of further assessment.
• Specificity – proportion of correctly identified
children who are developing typically.
• Both need to be at least 70%.
The SGS in Wales
• Welsh Assembly Government introduced
Flying Start (FS) Initiative.
• SGS chosen as the developmental screening
tool to evaluate FS Initiative.
• Problems with scoring identified during IY
Toddler trial (Hutchings et al., 2011).
Problems with SGS scoring
• Windows of assessment vary.
• Score highest item within scale regardless of
performance on other items.
• Cannot compare between groups or across time.
• Problems can be solved by developing way of
scoring to yield a Developmental Quotient (DQ)
score.
SGS Profile Form
Aims
• To validate both the original and new DQ way
of scoring the SGS.
• Two step validation process.
• Use of two data sets, the RCT of the IY Toddler
programme and MRes project comparing the
SGS and GMDS.
Step one
Aim 1: Estimate
appropriate cutoff for new SGS
scoring method
Aim 2: Determine
concurrent validity
of both SGS
scoring methods
against GMDS
Step two
Aim 1: Determine
concurrent validity of
both SGS scoring
methods against Ages
and Stages
Questionnaire (ASQ)
Step one: Sample & Measures
Participants
• 39 children
• Mean age 31 months
• 61% male
Measures
• Griffiths Mental
Development Scales
(GMDS)
• Schedule of Growing
Skills (SGS)
Step two: Sample & Measures
Participants
• 94 children
• Mean age 22 months
• 61% male
Measures
• Ages and Stages
Questionnaire (ASQ)
• Schedule of Growing
Skills (SGS)
Subscale comparisons
Griffiths Mental
Development Scales
(GMDS)
Schedule of Growing Skills
(SGS)
Ages and Stages
Questionnaire (ASQ)
Locomotor
Gross motor
Gross motor
Performance & Eye-Hand
Coordination (fine motor)
Manipulative & Visual (fine
motor)
Fine motor
Language
Hearing, Speech, &
Language
Communication
Personal-Social
Interactive & Self-care
Results: Step one
Aim 1: Establishing cut-off point
• Receiver Operating Characteristic (ROC) Curves
• Explored three potential cut-off points:
- DQ < 90
- DQ < 85
- DQ < 80
Results: Step one
Aim 1: Establishing cut-off point
SGS cut-off
AUC
Sensitivity
Specificity
DQ < 90
.794
90.83
67.88
DQ < 85
.779
74.18
81.55
DQ < 80
.789
65.83
91.90
• Most accurate cut-off is DQ < 85.
• Both sensitivity and specificity levels > 70%
Results: Step one
Aim 2: Concurrent validity with GMDS
• Calculated:
- Sensitivity
- Specificity
- Over-referral rates
- Under-referral rates
Results: Step one
Aim 2: Concurrent validity with GMDS
Development
area
SGS scoring
Sensitivity
(%)
Specificity
(%)
Overreferrals (%)
Underreferrals (%)
Locomotor
Original
16.67
100
0
12.80
New (DQ < 85)
83.33
51.52
41.02
2.56
Original
0
100
0
7.69
New (DQ < 85)
66.67
100
0
2.56
Original
20
100
0
10.26
New (DQ < 85)
80
94.12
5.13
2.56
Original
0
100
0
7.69
New (DQ < 85)
66.67
80.56
17.95
2.56
Original
9.17
100
0
9.61
New (DQ < 85)
74.17
81.55
16.03
2.56
Personal-Social
Language
Fine motor
Overall
Results: Step two
Aim 1: Concurrent validity with ASQ
• Calculated:
- Sensitivity
- Specificity
- Over-referral rates
- Under-referral rates
Results: Step two
Aim 1: Concurrent validity with ASQ
Development SGS scoring
area
Sensitivity
(%)
Specificity OverUnder(%)
referrals (%) referrals (%)
Locomotor
10
98.81
1.06
9.57
New (DQ < 85) 70
92.86
6.38
3.19
Original
50
93.02
6.38
4.26
New (DQ < 85) 75
83.72
14.89
2.13
Original
26.67
92.41
6.38
11.70
New (DQ < 85) 66.67
77.22
19.15
5.32
Original
28.89
94.75
4.61
8.51
New (DQ < 85) 70.56
84.60
13.47
3.55
Language
Fine motor
Overall
Original
Discussion 1
• New SGS scoring method shows increased
concurrent validity.
• Better sensitivity, comparable specificity,
higher over-referrals, lower under-referrals.
Discussion 2
Limitations
• Small sample sizes
• GMDS training
Implications
• Increased detection rates
• Greater use in clinical practice and research
Thank you for listening
Diolch am wrando
Any questions??