Transcript 1 - Phil Davies - Overseas Development Institute (ODI)

Government Social Research Unit
Impact and Insight Seminar
Overseas Development Institute
17 October 2005
Evidence-Based Policy at the Cabinet Office
Philip Davies PhD
Government Social Research Unit
Cabinet Office
London SW1A 2WH
www.gsr.gov.uk
Government Social Research Unit
Impact and Insight Seminar
Overseas Development Institute
17 October 2005
Making Public Policy More Evidence-Based
Philip Davies PhD
Government Social Research Unit
Cabinet Office
London SW1A 2WH
www.gsr.gov.uk
Outline
• Why evidence-based policy?
• Factors other the evidence
• Different types of evidence
• Some Problems
• Some solutions
www.gsr.gov.uk
Why Make Public Policy More Evidenced-Based?
• Effectiveness - ensure we do more good than harm
• Efficiency - use scarce public resources to maximum effect
• Service Orientation - meet citizen’s needs/expectations
• Accountability - transparency of what is done and why
• Democracy - enhance the democratic process
• Trust - help ensure/restore trust in government and public
services
www.gsr.gov.uk
What is Evidence-Based Policy?
“Evidence-based policy helps people make wellinformed decisions about policies, programmes and
projects by putting the best available evidence from
research at the heart of policy development and
implementation”
(Davies, P.T., 1999a)
www.gsr.gov.uk
Factors Influencing Policy Making
Pragmatics &
Contingencies
Lobbyists &
Pressure Groups
Habits &
Tradition
Experience &
Expertise
Judgement
Evidence
Resources
Values and
Policy
Context
www.gsr.gov.uk
Different Types of Evidence for Policy
Experimental
Quasi-Experimental
Counterfactual
Social Ethics
Public Consultation
Cost-Benefit
Cost-Effectiveness
Cost-Utility
Econometrics
Experimental
Quasi-Experimental
Qualitative
Theories of Change
Implementation
Evidence
Ethical
Evidence
Impact
Evidence
Economic
and
Econometric
Evidence
Statistical
Modelling
Descriptive
Analytical
Evidence
Surveys
Admin Data
Comparative
Qualitative
Surveys
Qualitative
Attitudinal
Evidence
Multivariate www.gsr.gov.uk
Analysis
Evidence-Based Policy
Opinion-Based
Policy
Evidence-Based
Policy
Increasing Pressure (Time)
Adapted from: Muir Gray 1997
www.gsr.gov.uk
Problem 1 - Social Scientific Knowledge
• Not all research is of sufficient quality
• Unclear objectives
• Poor research design
• Methodological weaknesses
• Inadequate data reporting
• Selective use of data
• Unsupported conclusions
• Uncertainty of scientific knowledge
• Different status of different fields of knowledge
www.gsr.gov.uk
Evidence-Based Policy
Opinion-Based
Policy
Evidence-Based
Policy
Increasing Pressure (Time)
Adapted from: Muir Gray 1997
www.gsr.gov.uk
Problem 2 - Different Notions of Evidence
Policy Makers’
Evidence
Researchers’
Evidence
• Colloquial (Contextual)
• ‘Scientific’ (Context free)
• Anything that seems reasonable
• Proven empirically
• Policy relevant
• Theoretically driven
• Timely
• As long as it takes
• Clear Message
• Caveats and qualifications
www.gsr.gov.uk
Problem 2 – UK Policy Makers’ Types of Evidence
• Quantitative and statistical evidence
• Economic evidence
• Qualitative evidence
• Surveys and attitudinal evidence
• Consultation evidence
• Behavioural evidence
• Anecdotal evidence
• Hard and soft evidence
• International evidence
X • Experimental and Quasi-Experimental Evidence
X • Systematic review/meta-analytical evidence
www.gsr.gov.uk
Problem 3 - Getting Research Into the Evidence Chain
Policy or Practice
Special Advisers
'Experts'
Think Tanks/Opinion Formers
Lobbyists and Pressure Groups
Professional Associations
Media
Constituents, Consumers and Users
Academics
www.gsr.gov.uk
Problem 4 – Uncertain Policy Logic/Theory of Change
Theories of Change - A Policy Logic Model
‘Scared Straight’ Programmes
Programme Theory
Visit to a
Prison by
Juveniles
First Hand
Experience of
Prison Life
Exposure to
Prison Life and
Prisoners as
Negative Role
Models
Frightens or
Scares
Juveniles Away
from Crime
Reduces
Crime and
Offending
Exposure to
Prison Life and
Prisoners as
Positive Role
Models
Stimulates or
Attracts
Juveniles
Towards Crime
Increases
Crime and
Offending
Programme Evidence
Visit to a
Prison by
Juveniles
First Hand
Experience of
Prison Life
www.gsr.gov.uk
Problem 5 – Inaccessibility of Evidence
For Policymakers Research Evidence Is Too:
• Long
• Verbose
• Detailed
• Dense
• Impenetrable
• Jargonesque
• Methodological
• Untimely
• Non-relevant/irrelevant
www.gsr.gov.uk
Some Solutions
• Integrate and plan research into policy strategically (CRAG)
• Establish incentives to use evidence (PSG initiative)
• Establish ownership of the evidence
• Clarify the policy/practice issues with users ex ante
• Establish users’ theory of change/logic model
• Establish answerable questions
• Establish the policy/practice timetable
• Knowledge translation (establish key messages clearly)
• Use appropriate formats for presentation (e.g. 1:3:25)
• Persistence and Opportunism (Matthew Taylor, 2005)
www.gsr.gov.uk
Contact
[email protected]
Government Social Research Unit
Cabinet Office
Admiralty Arch
The Mall
London SW1A 2WH
England
Tel: +44 (0)20 7276 1862
Fax: +44 (0)20 7276 1450
http://www.policyhub.gov.uk
www.gsr.gov.uk