Transcript Slide 1

Key issues for longitudinal
research
A view from overseas, National
Longitudinal Survey of Children and
Youth (Canada)
Stephanie Lalonde, Statistics Canada
April 20, 2011
Today’s presentation








2
Survey Overview
History of NLSCY
Survey Design
Survey Content
Direct Assessments
School Collection
Research
Lessons Learned
Statistics Canada • Statistique Canada
What is the NLSCY?
 A long-term study of Canadian children that follows their
development and well-being from birth to early
adulthood.
 Conducted by Statistics Canada and funded by Human
Resources and Skills Development Canada (HRSDC).
 Development began in 1992 and data from the final
collection were released in 2010.
3
Statistics Canada • Statistique Canada
Objectives of the NLSCY
 To determine the prevalence of risk and protective
factors for children and youth.
 To understand how these factors, as well as life events,
influence children’s development.
 To make this information available for developing
policies and programs that help children and youth.
 To collect information about a wide variety of topics –
biological, social, economic.
 To collect information about the environment in which a
child is growing up – family, peers, school, community.
4
Statistics Canada • Statistique Canada
History of NLSCY




5
Planning report May 1993
First data collection 1994-95
Data collection every two years
Last data collection 2008-09
Statistics Canada • Statistique Canada
Origins of NLSCY
 May 1992, the Canadian government
announced “Brighter Futures” initiative.
 “What works for children – Information
Development Program” is a component of
initiative
 Mandate to develop the first multi-disciplinary
national longitudinal and cross-sectional
database on children.
6
Statistics Canada • Statistique Canada
Implications on survey design
 Need to provide longitudinal and cross-sectional
information
 The ecological or holistic approach to the measurement
of risk and outcomes
 The need to provide children and family information
 The need to gather and integrate community information
 The need to collect information on selected children from
teachers
7
Statistics Canada • Statistique Canada
SURVEY DESIGN
8
Statistics Canada • Statistique Canada
NLSCY overview
1
0 original cohort
0 1
2
2
13
3
0 1
2 3
4
0 1
2 3
4 5
5
0 1
2 3
4 5
6
0 1
7
8
0 1
01
Cohort 1
9
11
2 3
2 3
23
Cohort 2
4
15
6
17
8
10
4 5
4 5
45
Cohort 3
19
6 7
67
12
8 9
89
Cohort 4
21
14
10 11
Cohort 5
23
Cohort 6
Statistics Canada • Statistique Canada
Cohort 7
25
Cohort 8
Longitudinal sample size and
response rates
Survey
cycle
1
2
3
4
5
6
7
8
10
Age
0-11
2-13
4-15
6-17
8-19
10-21
12-23
14-25
Sample size
hh
child
12,818
19,487
12,863
19,481
12,818
19,435
12,912
19,481
13,134
19,474
13,532
19,474
14,409
19,474
14,690
19,474
Respondents
Response
hh
child
%
11,141
16,903
86.7
10,220
15,403
79.1
9,810
14,796
76.0
8,839
13,168
67.8
8,592
12,300
63.1
8,222
11,210
57.6
8,597
11,016
56.6
8,510
10,268
52.7
Statistics Canada • Statistique Canada
Cycle 8 sample and response rates
Number of sampled children and
response rate, by age at Cycle 8
Age as of
December 31, 2008
Sampled
Years
In-scope
Cycle 8
response
rate
Respondents
Number
%
0-1
5,482
5,463
4,106
75.2
2-3
5,580
5,555
4,372
78.7
4-5
5,404
5,372
4,130
76.9
6-7
4,271
4,256
3,450
81.1
14-15
3,134
3,129
2,501
79.9
16-17
2,238
2,235
1,770
79.2
18-19
2,523
2,515
1,635
65.0
20-21
2,361
2,354
1,366
58.0
22-23
2,418
2,406
1,470
61.1
24-25
2,382
2,368
1,466
61.9
Total
35,793
35,653
26,266
73.7
11
Statistics Canada • Statistique Canada
Collection strategy
 Every 2 years - Mixed mode of collection
• Lasts 9 months (Sept – June)
• Multi level data: Household, Child, Parent (PMK),
Schools
• Direct measures
• Community surveys
12
Statistics Canada • Statistique Canada
SURVEY CONTENT
13
Statistics Canada • Statistique Canada
Context
Resources
Family
Community
Work
School
Social
Outcomes
Transitions:
Illness/Injury
Accidents
Divorce/separation
Death of Family Member
Spell of Poverty
Puberty
School Entry
Graduation
First Job
Marriage
First Child
Physical Health
Emotional
Social
Cognitive/
Learning
Space
14
Public
Programs
Time
Statistics Canada
• Statistique Canada
Language/
Communication
Person most knowledgeable (PMK)
 Person Most Knowledgeable (PMK)
• Person who answers question about the child
• Usually the biological mother, but not always
15
Age
PMK – biological
mother
PMK – biological
father
0-7
88.4%
9.6%
14-17
83.5%
11%
Statistics Canada • Statistique Canada
Questionnaires
PMK
Child/Youth
Teacher/Principal
(cycles 1-5)
Household
Adult
Child
Self-completes
Youth
Kindergarten
Elementary
Direct
Assessments
16
Statistics Canada • Statistique Canada
Survey Overview
17
Statistics Canada • Statistique Canada
Adult component
 List of subjects covered








18
Education
Labour force
Income
Health
Family functioning
Neighbourhood safety
Social support
Socio-demographic characteristics
Statistics Canada • Statistique Canada
Subjects Covered by Child Component










Education
Health
Medical/Biological
Mother’s work after child’s birth
Temperament
Literacy
Communication
Activities
Developmental Milestones
Childcare
19
 Behaviour
 Sleep habits
 Motor/Social/Cognitive
Development
 Relationships
 Parenting
 Custody
 Expectations (Aspirations)
 Socio-demographic
characteristics
Statistics Canada • Statistique Canada
Subjects Covered by Youth Component









Moving out of parental home
Health
Education
Labour Force
Career Aspiration
Income
Suicide
Political engagement
Self-assessment of abilities
20








Criminal behaviour
Relationships
Sexual Health
Activities
Self-esteem
Emotional Quotient
Social Support
Family formation and
fertility
Statistics Canada • Statistique Canada
Self-completed questionnaires
 Family and friends
•
•
•
•
•
•
•
•
•
•
21
School
Self-Esteem
Emotional Intelligence
Puberty
Dating
Smoking, Drinking and Drugs
Activities
Health
Work
Feelings and Behaviours (suicide)
Statistics Canada • Statistique Canada
DIRECT ASSESSMENTS
22
Statistics Canada • Statistique Canada
Direct Assessments
Age
Assessment
Measures
Grades 2 to 10
Math tests
(usually 7 to 15 years)
Achievement of basic
academic skills
16-17
Problem solving
exercise
Reading
comprehension,
problem solving
decision making
18-19
Literacy assessment
Prose literacy
Document literacy
20-21
Numeracy
assessment
Numeracy
23
Statistics Canada • Statistique Canada
Direct Assessments 4-5 year olds
 PPVT-R
• Measure of receptive vocabulary
 Who Am I?
• Measure of level of development
 Number Knowledge
• Measures understanding of numbers
24
Statistics Canada • Statistique Canada
Choosing direct assessments
 Increase focus on early childhood development
 Review of framework
 Selection of direct measures based on literature review and
review of other surveys
 16 measures selected for more detailed review
 11 measures informally tested
 5 measures field tested
25
Statistics Canada • Statistique Canada
Choosing direct assessment - criteria
 Available in English and French (or easily adaptable)
 Appropriate for administration in the child’s home
 Easy to administer by lay interviewers
 Easy to score
26
Statistics Canada • Statistique Canada
Assessments tested
 Who Am I?
 Number Knowledge
 Weschler Preschool and Primary Scales of
Intelligence (WPPSI-R)
 Early Screening Inventory
 Raven’s Coloured Progressive Matrices
27
Statistics Canada • Statistique Canada
Field test of assessments – Who Am I?

Measures developmental level and acquired knowledge and skills

Developed by Dr. Molly de Lemos at the Australian Council of Educational Research
(ACER)

Can be used for children aged 3 to 7

General development assessed using a copying shapes task and drawing picture of onself

Acquired knowledge and skills assessed through writing symbols such as numbers, letters,
words and sentences

Field test found that the assessment was relatively easy to administer and enjoyed by
children

The assessment is inexpensive

Dropped the drawing task to save time

Added to the NLSCY in Cycle 4
28
Statistics Canada • Statistique Canada
Field test of assessments – Number
Knowledge
 Assesses children’s understanding of quantity and the system of
whole numbers
 Developed by Dr. Robbie Case and colleagues at the Institute of
Child Study - University of Toronto
 Four developmental levels (pre-dimensional, uni-dimensional, bidimensional, and integrated bi-dimensional)
 Levels are attained at approximately 4, 6, 8 and 10 years of age.
 Only first three levels used in test
 The assessment included counting to 10, concepts of quantity,
number line, simple additions and subtractions and some problem
solving.
 Added to Cycle 4 of the NLSCY with some modifications.
29
Statistics Canada • Statistique Canada
Field test of assessments – Block Design
 Sub-test of the Wechsler Preschool and Primary Scales of
Intelligence-Revised (WPPSI-R)
 WPPSI-R can be used as an intelligence test. Block design is one
of the performance sub-tests and examines logical reasoning.
 The child must design with blocks a shape copying either one the
interviewer constructs or from a booklet.
 The task proved too difficult for the interviewers to administer
consistently so was not added to the NLSCY.
30
Statistics Canada • Statistique Canada
Field test of assessments – Early
Screening Inventory
 A brief developmental screening instrument that is individually
administered to children from 3 to 6 years of age.
 Developed by Dr. Samuel Meisels and colleagues at the University of
Michigan.
 Designed to identify children who may need special educational services
to participate successfully in school.
 Provides an overview of the child’s development in three major areas:
Visual-Motor/Adaptive (fine motor skills, eye-hand co-ordination and
short-term memory skills), Language (not used in test) and Cognition and
Gross Motor.
 Some difficulties in administration but recommended for inclusion in
NLSCY
 Final decision was not to use the ESI
31
Statistics Canada • Statistique Canada
Field test of assessments - Raven’s
Coloured Progressive Matrices
 Assesses child’s capacity for analogical reasoning as one aspect
of intelligence.
 Child must select the missing element to complete a pattern
 Test is non-verbal
 Well liked by parents, children and interviewers
 Not added to the NLSCY
32
Statistics Canada • Statistique Canada
SCHOOL COLLECTION
33
Statistics Canada • Statistique Canada
School collection
 Changes cycle to cycle
• Dropped from Cycle 6 (2004) on




34
Requires signed parental permission
Requires co-operation from all ten provinces
Relatively low response rates
Logistically complex
Statistics Canada • Statistique Canada
Survey content teachers






35
Child/student’s education
Child/student’s behaviour and attendance
Involvement of parent and guardian
Teaching practices
Teacher’s perceptions of the school
Personal information
Statistics Canada • Statistique Canada
Survey content - Principals





36
Students in the school
Involvement of parent(s) and guardian(s)
Characteristics of school
Principal’s perceptions of the school
Personal information
Statistics Canada • Statistique Canada
COMMUNITY SURVEYS
37
Statistics Canada • Statistique Canada
Community surveys
 Part of Understanding the Early Years initiative
 Designed to give communities information to enhance community
resources and services
 Worked with community groups
 Includes a mapping project to map community resources and services
 Goal is to allow each community to use information to improve early
childhood development
38
Statistics Canada • Statistique Canada
Collection for Communities
Vocabulary Test
PPVT
Community
Resource
Use
Who Am I
Household Contact
Parent Questionnaire
Child Questionnaire
Number Knowledge
Phone interview
Questionnaire
CAI
39
Tests
Paper Questionnaire
Statistics Canada • Statistique Canada
EDI
Teacher completed
Collected in the school
SELECTED FINDINGS FROM
NLSCY
40
Statistics Canada • Statistique Canada
Growing Up in Canada
 Hyperactivity biggest risk factor in slowing math skills
 Aggressive behaviour tended to decrease as children
grew up
 Positive parenting can make a difference in
disadvantaged families
41
Statistics Canada • Statistique Canada
Vulnerable children
 Edited by J. Douglas Willms
 Development of Vulnerability Index
 Childhood vulnerability only weakly linked to income
 Effects of good parenting outweigh effects of income
 Vulnerability varies amongst communities
42
Statistics Canada • Statistique Canada
Parenting style and children's aggressive
behaviour
 Change in parenting environment predicts
change in child's behaviour
 Aggressive behaviour linked to parenting style,
regardless of sex or income
43
Statistics Canada • Statistique Canada
Other examples
 Relationship between youth depression and changes
in relations with parents and peers.
 Description of Child care in Canada
 Description of Readiness to Learn of Five Year Olds
 Canadian Nine Year Olds at School
 Successful Transitions conference
44
Statistics Canada • Statistique Canada
LESSONS LEARNED
45
Statistics Canada • Statistique Canada
Challenges
 Breadth of content
 Large number of age groups
 Mixed requirements
• Longitudinal
• Cross-sectional
 Changes from cycle to cycle
46
Statistics Canada • Statistique Canada
Example of changes
 Age – age at time of collection vs.
reference age
 Most questions asked based on reference
age but norms based on actual age
 Change of PMK over time
47
Statistics Canada • Statistique Canada
Lessons Learned






Keep it simple wherever possible
Have adequate resources from the start to design the best survey vehicle
Think longitudinally
Re-evaluate decisions made at previous cycles to determine if they are still
appropriate
Do research and analysis to develop new methods
Ensure that there is good documentation
•

Have clear long-term objectives
•
48
That explain the ‘why’ and allow others to replicate the work
NLSCY is trying to please everyone: focus on a few things and do them well
Statistics Canada • Statistique Canada
Contact Information
 Stephanie Lalonde
 [email protected]
 General NLSCY inquiries [email protected]
49
Statistics Canada • Statistique Canada