Transcript Linking administrative data to TALIS and PISA

Linking administrative data to
TALIS and PISA
Dr. John Jerrim
UCL
Structure
Data linkage in TALIS (previous work)
• What data is available
• Why bother to link?
• How was the linking done?
•What additional results were produced?
Data linkage in PISA (future work)
• What data is available?
• What projects might linked PISA data make possible?
What do we mean by ‘admin’ data?
• Typically collected by national authorities.....
• Known for every unit (e.g. Pupil / teacher / school) in the
population....
• Includes very basic information (e.g. School type / location)....
•...but could be a lot more detailed (e.g. Test scores)
Every country will have some kind of administrative data!!
(Though some likely better than others)
Why are admin data so good?
Strengths of admin data....
•Very low amounts of missing data
• Available for both survey and non-survey units
•Typically very well measured
•Provides extra information not collected within the survey
•FREE!!!!
• More salient national definitions than ‘international’ variables (see
next slide)
Private schools in England.....
Table 2.17
OECD report
Netherlands
Flanders
Chile
UAE
England
Australia
Spain
Denmark
Cyprus
Korea
France
Brazil
Mexico
Sweden
United States
Portugal
Israel
Japan
Slovak Republic
Norway
Poland
Alberta (Canada)
Estonia
Finland
Italy
Czech Republic
Iceland
Croatia
Latvia
Bulgaria
Singapore
Malaysia
49% in England!?
We don’t fit
international
definition very well!
Data on school type
(national definition)
linked in from admin
data.....
...about 7% of
population (much
more salient to
readers of our TALIS
report)
0
20
40
60
80
100
What administrative data is
available in England?
School level
• School type (Academy, maintained independent)
• Average achievement of intake (age 11)
• Historical school average performance (age 16 exams)
• Pupils eligible for free school meals (i.e. level of deprivation)
• Ofsted rating (‘quality’ based upon school inspections)
Teacher level
Teacher workforce census
• Teacher pay (and pay progression)
•Qualifications
• Sickness / absences from work
• Demographic information (e.g. Ethnicity)
• Type of contract
• Roles and responsibilities
Pupil level
National pupil database
• Test scores at age 5, 7, 11, 14 and 16.....
• Very detailed ethnicity
•Detailed special educational needs (SEN)
• Deprivation (FSM; local area based measures)
• Absences from schools
•School exclusions
What can be done with
administrative-linked TALIS data?
What can be done with linked TALIS data?
1. Address new and interesting questions of national interest
- E.g. Teacher working hours by OFSTED rating
- E.g. Link between teacher pay and job satisfaction?
2. Exam patterns (and potentially correct for) non-response
- E.g. Compare age distribution of teachers in TALIS to population
3. Look at measurement properties of certain variables
- E.g. How well do teacher reported qualifications match up with
administrative records
A longitudinal TALIS study?
Possible to link TALIS to teacher workforce census
Hence can track the TALIS teachers as they move forward in their
career….
E.g. What are the predictors of teachers leaving the profession?
What are the drivers of teachers moving to another school?
What predicts who becomes a headmaster?
How do you link?
How do you link?
Theory = easy!
• Each unit (e.g. teacher) has a unique record number.....
• ...if this number observed in both TALIS and admin data
• ...then straight forward to match (in theory!)
Alternative
• Unique record number might not be in both files......
•
...(particularly in linking done ex-post = won’t be in TALIS)
•
Match on other characteristics (e.g. Forename, surname, DoB)
•
Will result in some error!
Beware of the pitfalls!! (England example)
•
Every school in England has its own URN
•
We observe this in both TALIS and admin data
•
Should therefore be easy to match....
But it was more complicated than that!!
•
2012/13 some schools changed their ‘status’ from a ‘maintained’
school to an ‘academy’.......
•
Basically still the same school...but given a new URN!
•
Hence URN in TALIS and Admin data didn’t always match!
•
Had to go back and manually find out the new URN!!
What analysis did we do with the
administrative linked dataset?
Analysis by Ofsted rating
Headteachers’ job satisfaction by Ofsted rating
% of teachers reporting lack of support for TPD
Teachers’ views on working conditions by Ofsted rating
Analysis by age 16 test scores
(school average)
England did not do the PISA-TALIS link
Why?
We already know a lot about the TALIS schools:
E.g. Pupils entry scores (age 11)
Pupils exit scores (age 16)
Proxy for family background etc
Less value added for England than other countries.....
So we just linked to the administrative data instead!
Headteacher job satisfaction by historical school
performance
Teachers’ views on working conditions by school average age
16 scores
Appendix on teacher nonresponse
A linear probability model of teacher non-response
Some minor
differences
observed
between
sampled and
participating
teachers.....
Please download our national report!!!!
https://www.gov.uk/government/publications/teachers-insecondary-schools-evidence-from-talis-2013
https://johnjerrim.files.wordpress.com/2013/07/national_report_dfe
.pdf
Linking PISA to administrative
data
PISA
•
International study of 15 year olds skills in reading, maths and
science
•
Conducted by OECD every three years
•
Lots of media, academic, public policy attention
•
Detailed demographic information
•
Also a lot of attitudinal data also
Recently noted by OECD (long-term strategy of PISA)
To continue to seek methodological and analytical means to
strengthen the policy relevance and analytical power of PISA,
including establishing best practice for linking PISA with national
assessments
PISA-NPD linked data
•
In England, PISA can be linked to NPD
•
This means we can observed PISA scores along with children’s
test scores at ages 5, 7, 11, 14 and 16
•
This makes it possible to consider children’s progress towards
PISA scores during secondary school (e.g. ‘value-added’
measures)
•
Can also link to children’s post-16 qualifications and destinations
•
Can therefore look at the link between PISA scores and the
probability of going to university, subject choice etc.
•
In other words – allows longitudinal follow-up of PISA cohorts!
Use 1: Link between PISA scores and
England’s national exams
There is a strong link
between PISA scores
and age 16 exam
performance……
…helps demonstrate
the validity of PISA as
a test score measure
Use 2: Investigate and correct nonresponse bias
England missed response target in PISA 2003
Kicked out of the international report
Linked data allowed us to examine the likely bias / impact upon
results…..
Use 3: Regional estimates in England
(forthcoming)
Several countries now producing regional PISA estimates….
… e.g. results for certain US states
Interest in England too – but there are a lot of local education
authorities (152) meaning a high cost
Also a lot of extra burden upon schools
Forthcoming work……
…..Use administrative data to produce proxy PISA estimates by LEA
……Linked administrative data helps us answer this interesting policy
question
Conclusions
Conclusions
Many advantages to administrative linked data
- Available for population
- Well measured. Low missing data
- New / extra information
- FREE!
Can use to address many interesting questions
- Link to teacher and pupils
- Relationship to national examinations
- Investigate non-response issues
• Other countries can probably do this too!!!