Transcript No Slide Title

Key issues for longitudinal
research design:
Lessons from the Growing
Up in Scotland study
Paul Bradshaw
Definitions of longitudinal research
• In the broadest sense longitudinal research involves
the follow up of any set of entities in which changes
can be observed over time.
•
•
•
•
Individuals
Households
Institutions such as hospitals or schools
Nations
• Most commonly focussed on individuals
• No recognised definition of what period or what
number of follow-ups constitutes ‘longitudinal’
• Aim is usually to determine causes and processes
which lead to change and/or to particular outcomes
The UK experience
• The UK has a long, and highly regarded, history of
(particularly quantitative) longitudinal social enquiry
• A number of ongoing internationally renowned
longitudinal studies providing information on the life
histories of people as they move from birth to old age
• Current QNLR being undertaken in Britain includes:
•
•
•
•
•
Birth cohort studies
Age cohort studies
Family/Household panel studies
Area studies
Census-based studies
The challenges of longitudinal
research
“The success of a longitudinal study depends on stable
leadership from a committed principal investigator and a team of
highly skilled researchers”
(Bynner et al, 2006)
Data collection
Sample
Cost
Duration
(& relevance)
Data analysis
Ethics
The Sample
Two main issues:
• Sample design
• Non-response and attrition
Sample design considerations
• What is the population of interest?
• All vulnerable children?
• Sub-groups of interest? E.g. with particular characteristics –
age, family circumstances, area where they live,
interventions received
• What do you want to be able to say about them?
• Use sample to generalise to population?
• Compare outcomes and experiences between children in
different groups?
• How will you find/recruit them?
• Is there a sampling frame with the info you need?
• Will you need the help of an agency/organisation to recruit?
Sampling precision (1)
• When using a sample, the data produces ‘estimates’ of what the real
value may be in the population. The precision of these estimates is
affected by:
• Sample size
• Sample clustering
• Measuring sample precision:
• Use ‘confidence intervals’ and ‘standard errors’
• Confidence interval (CI)
• Typically 95% meaning 95 times out of 100 this interval will capture the
true population value that we are trying to estimate
• Expressed as e.g. 64% (+/- 6%) suggesting the true value is
somewhere between 58% and 70%
• Differences will usually have to be of the magnitude of the CI for it to be
statistically significant – e.g. in example above, an increase or decrease
of around 6% would be necessary.
• Smaller sample sizes produce larger CIs requiring greater change for
statistically significant differences to be detected
Sampling precision (2)
Sample numbers required in each group to demonstrate
significant differences (with a power of 0.8)
Size of difference between groups
(percentage points)
Lower of the two
percentages:
5
7
10
15
10
687
374
200
101
25
1252
653
330
153
50
1566
797
389
170
75
1095
541
251
101
90
436
195
75
-
GUS response and attrition
rates
No. cases
issued
No. cases
achieved
Response
rate
As % of sw1
achieved
Sweep 1
6583
5217
79%
100%
Sweep 2
5217
4512
86%
86%
Sweep 3
4665
4193
90%
80%
Sweep 4
4394
3994
91%
77%
Sweep 1
3605
2859
79%
100%
Sweep 2
2859
2500
87%
87%
Sweep 3
2599
2332
90%
82%
Sweep 4
2460
2200
89%
77%
Birth cohort
Child cohort
GUS: Sweep 5 response by age
of mother
95
% achieved
90
85
80
75
70
Under 25
25-29
30-34
35-39
Mother's age at child's birth
40+
Non-contact and refusal rates
No. cases
issued
No. noncontact
% noncontact
No.
refused
%
refused
Sweep 2
5217
162
3%
316
6%
Sweep 3
4665
102
2%
268
6%
Sweep 4
4394
93
2%
203
5%
Sweep 2
2859
100
3%
153
5%
Sweep 3
2599
51
2%
142
5%
Sweep 4
2460
51
2%
136
6%
Birth cohort
Child cohort
Sample maintenance strategies
•
•
•
•
Keeping in touch
Tracing
Keeping respondents informed
Valuing and appreciating respondents (including, but
not restricted to, use of incentives)
• Boosts of key sub-groups of interest and on-going
sample refreshment
• Making reasonable demands
Sample maintenance: keeping in
touch
• Potentially willing respondents can be lost by virtue of
moving home.
• Important to establish effective procedures for
obtaining updated contact information.
• Techniques include:
• Collecting as much contact information as possible at first
contact including telephone/mobile numbers and e-mail
• Information about ‘stable contacts’ – someone who knows
the respondent and would know where they had moved to
• Regular mailings with return address – undelivered items act
as address checks ahead of fieldwork
• Updates from administrative and service databases
Sample maintenance: keeping
respondents informed
• In any research project, it is considered important to
provide good information to respondents about the
purpose and nature of research
• Respondents must understand the longitudinal nature
of the study so that they also recognise why they are
repeatedly visited and why they are irreplaceable
• Many studies adopt a ‘branding’ approach – a name
and logo that clearly identifies the study and which is
easily recognisable by respondents
• Websites can be extremely useful in providing more
detailed information
Sample maintenance: valuing and
appreciating respondents
• If respondents are going to continue to take part in a
longitudinal study they need to feel that their time and
effort is valued and is worthwhile.
• This can be demonstrated through:
•
•
•
•
The interviewer
Thank you letters sent after each interview
Providing evidence that survey findings have been used
Illustrating the impact they have had on government policy
• Use of incentives
• Non-financial rewards – gifts such as pens, fridge
magnets, calendars
Being part of NCDS
There’s one thing I’m guaranteed, all of my entire life, is one
birthday card.
I’ve always liked being part of it, I’ve always enjoyed being part
of it because it’s different and I’m quite proud of it really.
I think I could almost put this in as a landmark in my life.
I haven’t kidded on about anything in my life, warts and all,
I’ve been honest and I’ve said it all.
….. I think, you know, it does make you feel special in certain
ways, like really…..
I haven’t ever seen anything negative in it actually, not
anything negative at all…..I’ve always felt comfortable and I
always know that if I don’t want to answer a question I don’t
have to………… I think it’s just been fascinating
Recognising the importance of
NCDS
I can remember becoming part of the NCDS, I didn’t understand
it.... I think I’ve just grown to understand it more as I’ve got
older and the importance of it and how it’s helping everybody
really, that’s what I think.
Well, I think to a certain extent you’ve got to say it’s mainly for
helping others, you know. Like you say it’s no benefit to me to
do it, but then again it’s no skin off my nose not to do it, so. It’s
one of those things like, you know….I don’t see a reason [not
to]……
I think it’s interesting that they’re actually following all these
people, right through their life and ……they find out
comparisons. Aye, I think it’s really…. that’s why I take the
bother to, aye, come. I want to be part of it because I’ve been in it
all my life.
Data collection: what to ask and
when
• Decisions on content of data collection require good
planning – important to ask the right questions at the
right time
• Take a ‘longitudinal’ view:
– What ‘stage’ are your subjects at?
– What stage will they reach?
– What situations, characteristics or contexts might you want to
compare between those two stages?
– What might be significant at stage 1 when you are comparing
outcomes at later stages?
• Decisions on content will usually be theory or
hypothesis-based
• What if you miss something?
Data collection: intervals between
fieldwork
• How often do you follow-up your sample? This is
dependent upon:
• Respondent burden
• Developmental stages, processes or transitions that you are
interested in
• Budget
• There is no recognised nor legitimate pattern – it’s
largely dependent upon the objectives and focus of
the research
Follow-up intervals of selected UK
child cohort studies
Child’s age in years
Study
NCDS (1958)
BCS (1970)
MCS (2000)
GUS BC (2005)
GUS CC (2005)
ALSPAC (1991)
0
1
2
3
4
5
6
7
8
10 11 16
GUS: Sources of data
Sw1
Sw2
Sw3
Sw4
Sw5
Sw6
(2005/6)
(2006/7)
(2007/8)
(2008/9)
(2009/10)
BC only
(2010/11)
BC only
Main carer
Main carer
Main carer
Main carer
Main carer
Main carer
Partner
Child height
& weight
Child height
& weight
Cognitive
assessmts
Health
records
Health
records
Health
records
Child height
& weight
Cognitive
assessmts
Health
records
Health
records
Health
records
School
records
School
records
Data analysis
• Benefits of longitudinal data
• Measuring change over time:
– Flows into and out of ‘states’ (e.g. poverty, unemployment,
being looked after)
– The effects of change, or of state durations, on outcomes
– Impact of interventions
– ‘Individual’ development
•
•
•
•
Temporal ordering of events
Improved control for omitted explanatory variables
Improved control for the effects of previous states
Exploring the effects of ageing and cohort membership
Data analysis
• Drawbacks of longitudinal data
• Data management
– Complex structure/relations
– Complex variable/samples
– Resultant file and variable management requires training and
skills of good practice
• Software issues
– Complexity of methods
– Some methods only available via specific software packages
Longitudinal models
• Two main modelling approaches in social science
research:
• Event history analysis, time to an event
(Also known as: duration analysis; survival analysis; failure time;
duration economics; hazard modelling)
• Panel data analysis
–
–
–
–
–
Regression models suitable for repeated observations
Time generally conceptualised as being discrete
Extension of standard regression models
Closely related to multilevel modelling
(Simple methods can also be used)
Ethical issues (1)
• Confidentiality, data security and data access
• Safeguarding confidentiality through
– Restricting the data that are released
– Controlling the arrangements under which potentially disclosive
data is released
• Data access/release
– Providing data for researchers to analyse on their desktop with
the level of sensitive detail restricted
– Providing more sensitive data under a special licence
– Remote access where raw data is never released and analyses
are run in house on behalf of researchers who receive edited
outputs
– Safe settings where researchers are required to visit a
protected site where access to the data is carefully managed.
Ethical issues (2)
• Informed consent
• What is ‘informed consent’?
– Respondents must understand that the project is longitudinal in
nature and what that means for them
– They are free to withdraw at any time from the project as a
whole or any aspect of it
• One-off or repeated consent?
– No universal practice, but
– Some form of repeated consent would normally be involved
Summary
• Longitudinal research in any methodological
discipline presents a set of core challenges
• The key issue is to consider, in detail, how the
fundamental temporal nature of the project affects the
basic aspects of its design including:
•
•
•
•
The sample
Data collection
Data analysis
Ethics
• Requires a more complex design, but a good design
will produce tremendously valuable data