Screening for Disability

Washington Group on Disability Statistics April 20 1

Screening for Disability on Censuses

• The main purpose of a screening instrument is to be as inclusive as possible in order to identify all eligible people for further study. April 20 2

Screening Questions

• • To be inclusive, questions should cover all three of the main dimensions described in the ICF: body structure & function, activity (basic actions), and participation (complex activities). Once the disabled population is identified, the follow-up survey could focus obtaining more information on a wide range of aspects of disability related topics.

April 20 3

Screening Questions

• • • • Screening reduces total respondent burden and survey costs.

However, the length of the ideal screener could be prohibitive, and The more inclusive the screened population – the larger the sample to be followed and the higher the costs Disability Supplement to the US NHIS screener took 25 minutes to administer and identified over 25% of the population April 20 4

Canadian Experience

• • The Census is used to develop a frame for a disability survey The Participation and Activity Limitation Survey (PALS) determines disability status April 20 5

Census Filter Question - 2006

1. Does this person have any difficulty hearing, seeing, communicating, walking, climbing stairs, bending, learning or doing any similar activities?

2. Does a physical condition or mental condition or health problem reduce the amount or the kind of activity this person can do

At home? At work or at school? In other activities, for example, transportation?

Response categories: Yes, sometimes; Yes, often; No

April 20 6

• •

2006 PALS

2006 Census 2006 PALS collection (46,000 persons) • Post collection follow-up • • • Face to face interviews with 50 false positives Face to face interviews with 100 false negatives Face to face interviews with 50 mild disabilities April 20 7

results

 False + false April 20 8

•

False Negatives

A person with a disability that does not get picked up by the filter question – • Proxy reporting • • • • Timing of the Census – cyclical or episodic disability Mild to medium pain limitations Psychological limitations Perception of activity limitation • Related to age, cultural identity, language April 20 9

False Negative - Results

• • • • • • False negatives: persons with disabilities who say No to the original screening questions but who WOULD have reported a limitation if they were administered the PALS survey.

Following the 2006 PALS a small scale exercise for adults was conducted.

Persons in Toronto and Ottawa who said No to the Census filters were administered the PALS survey.

Some reported limitations and this cast some doubt on the effectiveness of the filter questions.

But on analysis it was found that almost all respondents were coming in on the pain module middle aged persons with sore knees or shoulders.

For the adult population, the filter questions were not seen as missing large numbers of persons with disabilities.

April 20 10

False Positives

• A person without a disability who answers ‘Yes’ to a filter question – • Proxy responding • Short term disability or injury • • Cyclical or episodic disability High false positive rates for children April 20 11

• •

False Positive - Results

For adults, about 25% of PALS respondents are false positives, i.e. they said Yes on the census but report no limitation on the PALS survey.

The most common reasons are: 1.

2.

3.

short term injury proxy effect as the census is completed by a single household member and mistake, often language based.

• For children, the false positive rate is about 50% here, the proxy effect is less clear as both the Census and PALS are proxy for children.

April 20 12

• • •

Irish Experience

2004 - Government decision to conduct post-censal National Disability Survey (NDS) Purpose • • Establish prevalence, severity and impact of disability in Ireland Identify improvements needed in policy and service provision Targeted survey - sample based on responses to 2006 Census questions on disability April 20 13

Census questions

April 20 14

9 Disability types in NDS

Seeing Hearing Speech Mobility and dexterity Remembering and concentrating Intellectual and learning Emotional, psychological and mental health Pain Breathing 15

NDS thresholds for disability

Disability type

A. Seeing B. Hearing C. Speech E. Remembering & concentrating F. Intellectual & learning G. Emotional, psychological, & mental health H. Pain I. Breathing

Level of difficulty in every day activities No difficulty Just a little A moderate level

  

A lot of difficulty

  

Cannot do at all

                    16

•

Census and NDS – comparison

NDS private household sample matched to Census data file • • • • Analysis based on unweighted sample data only Comparison of distribution of ‘Yes’ and ‘No’ responses in each - ‘false positive’ / ‘false negative’ responses to Census question Types of disability reported in Census and NDS Consistency of reporting of disabilities between Census and NDS April 20 17

Comparison of Yes and No responses

Census response No Yes Total Census No

1,321 8.4% 1,910 12.4% 3,231 20.8%

NDS Response Yes Total NDS

295 1.9% 12,062 77.4% 12,357 79.3% 1,616 10.3% 13,972 89.7% 15,588 100.0% April 20 18

Census ‘false positives’

• • • • 14% of persons reporting a disability in Census did not report a disability in NDS 25% 65 and over Report lower levels of limitation 53% - Census category “Other including chronic illness” April 20 19

Census ‘false negatives’

• • • 18% of people who did not report disability in the Census did report disability in the NDS 51% aged 65 or over Highest proportion of responses: • Pain disability category - 51% • • Mobility & dexterity disability category 46% Report lower levels of limitation April 20 20

Prevalence rates

• • Census: 9.3% NDS: 18.5% • 11.5% of false negatives are classified as disabled on the NDS; this group has large weights April 20 21

Conclusions - Ireland

• • • • • • • Matching of NDS records to Census at person level increases statistical value of NDS output A high proportion of the Census false positive responses are due to responses in the residual ‘Other’ category >3/4 of sample reported a disability in both surveys The Census questions & methodology resulted in a much higher level of single disability reporting than NDS Learning related difficulties prevalent among children in the sample Mobility and Pain most prevalent disabilities for older people People of working age had a more mixed range of difficulties April 20 22