How to spot a quality screening measure

Frances Page Glascoe

www.pedstest.com

www.forepath.org

STANDARDIZATION

Large, nationally representative Proportionate Demographics: minorities, parents’ level of education, broad geographic locations, Spanish speakers, etc.

Children with disabilities (



18% of total) Naturalistic sample 16% -

RELIABILITY

Inter-rater--at least 85% Test-retest--at least 85% Internal consistency > .80

How well does a screen actually measure what the authors claim?

SCREEN Good DX TEST Good Poor Poor

Good SCREEN ?

DX TEST Poor ?

VALIDITY

Concurrent: battery should sample same domains as screen Discriminant: information on how well various kinds of problems are detected Predictive: optional but helpful for showing that items sample enduring aspects of development

SCREEN Criterion Measure/Gold Standard

CRITERION-RELATED VALIDITY =

ACCURACY

Screening sorts those who probably have problems from those who probably don’t

ACCURACY

SPECIFICITY = % of children without problems correctly detected (e.g., by passing, above cutoffs scores) SENSITIVITY = % of children with problems correctly detected 70% to 80% is the standard

SPECIFICITY

SENSITIVITY

Accuracy of the Denver-II Developmental DX NO YES Denver-II PASS FAIL 69 17 8 10 86 27 86 18 Sensitivity = 10/18 = 56% Specificity = 69/86 = 80%

AGE

ACCURACY: ACROSS AGE RANGES

SENSITIVITY N

0 - 18 mos 3/4 18 mos - 3 yrs 27/34 3 - 4 1/2 yrs 4 1/2 - 8 yrs 26/35 42/57 TOTAL 98/130

%

75 79 74 74 75

SPECIFICITY N

66/82 117/149 118/165 172/245 473/641

%

80 79 72 70 74

Rigorously Peer-reviewed

Because of the enormous expense involved in standardization, reliabilty and validity research and accuracy studies, tests are expensive to create, keep current, provide translations, and to ensure that users and researchers have support and guidance. For these reasons, tests costs money and it is why few measures can be photocopied. The money, however small, you spend when purchasing measures goes to their continued development.

Detection rates WITH Screening Tests 70% to 80% of children with developmental disabilities correctly identified Squires et al

,

JDBP. 1996;17:420 - 427 80% to 90% of children with mental health problems correctly identified Sturner, JDBP . 1991

;

12

:

51-64 Most over-referrals on standardized screens are children with below average development and psychosocial risk factors

-

Glascoe, APAM. 2001; 155:54-59.