Transcript Module 2 - Louise Shaxson - Overseas Development Institute

Evidence and the
Policymaking Process
CFHSS Congress 2006
York University
Louise Shaxson
[email protected]
The Series
• Effective Research for Development
Policy: How researchers can maximize
their influence on policy.
• Evidence and the Policy Making
Process: How do policy makers access
research, what constitutes evidence.
• Action Research for Maximum Impact:
Some "good news" case studies, and
practical research tools for practitioners.
This Workshop
•
•
•
•
•
•
•
•
Exercise: Food in primary schools
Drivers of change
Policy development trajectory
Evidence and analysis
What policymakers want
The role of analysts
EBPM in practice
Further information
Exercise: Food in
Primary Schools
Drivers of change
• Increasing emphasis on the quality of
evidence and its use (Modernising
Government);
• To underpin & inform strategy, policy,
regulatory work, foresight; and to mitigate
risk;
• Importance of challenge to evidence (BSE
inquiry, Science Advisory Committees)
• Depth and breadth of future evidence
needs will increase given complex and
overlapping strategic priorities
Policy development trajectory
• Smaller policy core (efficiency drive in the
public sector)
• Getting rid of the ‘generalist/specialist’ label
• Improving intelligent customer capability
• Future evidence needs, when set against
strategic priorities, are complex &
overlapping. How do we deal with
‘sustainability’?
Evidence for policy is…
…any robust information that helps to turn a
Department’s strategic priorities into
something concrete, manageable and
achievable.
The nature of the evidence you need is
proportional to the nature of the risk
associated with the decision that is being
made.
What is evidence and analysis?
Evidence is:
Facts (data, known trends), judgements,
opinions, analyses, syntheses, arguments,
costings, reviews, qualitative & quantitative
survey data
Analysis is:
Lines of argument (strategy-policy), research,
interpretation
Decision makers like numbers…
…but the evidence base is built upon
• Data
• Lines of argument (analysis)
• Stakeholder opinions
Evidence-based policy making
is not a sacred cow:
There are policies that:
Use good information…
Use poor information…
…and use it well…
…and use it poorly…
Analysis & evidence for policy
Longer-term policy and strategy development
Procuring, managing
and carrying out
research to provide
new evidence
Scoping the issue,
asking the question,
deciding what sort of
evidence is needed
Interpreting & applying
new or existing evidence,
monitoring & evaluating
the policy once
implemented
Evidence and analysis needed rapidly to answer
pressing policy questions
Components of robust evidence
& analysis (supply side)
• Is the evidence credible?
• Can we make generalisations from it?
• Is it reliable enough for M&E or impact
assessments?
• Is it objective? How do we account for
bias?
• Is it rooted in an understanding of the
framing assumptions?
Components of robust evidence
& analysis (demand side)
• Is the evidence policy relevant?
• Is it timely? Has it been delivered fast
enough to inform policy decisions?
• Is it accessible to all key stakeholders, not
just researchers?
• Is the evidence cost-effective?
• Is it interdisciplinary enough to address
cross-cutting issues?
The role of analysts…
…is to provide appropriate evidence &
analysis throughout the policy making process
Quality assessment, peer review, advisory councils, G2000
Procuring, managing
and carrying out new
research
Scoping the issue,
asking the question,
deciding what sort of
evidence is needed
Negotiate how to apply the
evidence; consultancy role
Interpreting &
applying new or existing
evidence, monitoring &
evaluating the policy
once implemented
Negotiate the question, advise on
alternative sources of evidence
The role of analysts
• A smaller policy core won’t have time to do
the integration
• Intelligent customers need intelligent
suppliers
• Multiple tools: social, economic, modelling,
scientific, technical, institutional analysis…
• Help policy makers lift their eyes from their
desks to see what is possible
but
• Help them maintain their focus on the
deliverable.
EBPM in practice
• Defra’s Evidence & Innovation Strategy:
• Matching supply-side and demand-side criteria
• Based on the question ‘does it make good policy?’
rather than ‘is it good science?’
• Develop a clear line of sight between policy priorities
and evidence provision – for efficient delivery
• Analyse in relation to the policy cycle – evidence
needs change from strategy through to delivery
• Need a better understanding of innovation
How can analysts balance…
• the need to answer policy’s immediate
questions…(with robust & cost-efficient
evidence)
• while developing their role…(as people who
understand policy processes)
• to underpin broader & more strategic
approaches to policy (involving other
evidence providers, particularly other
disciplines)
Selected Bibliography
• Shaxson, L.J. (2005) “Is your evidence robust
enough? Questions for policy makers and
practitioners”. Journal of Evidence & Policy 1(1):
101-111.
http://www.ingentaconnect.com/content/tpp/ep/2005/00
000001/00000001/art00006
• Evidence-based policy making: guidance for
policy makers. Available at
http://www.defra.gov.uk/science/how/evidence.htm
• Jones, K.E. (2005) Understanding risk in
everyday policy making. Defra: UK. Available at
http://www.defra.gov.uk/environment/risk/policymaking0509.pdf
• Defra (2005) Evidence & Innovation Strategy
2005–2008 (Part 1: Aims of the consultation /
introduction). Available at
http://www.defra.gov.uk/science/how/documents/PDFs%20in
%20Parts/Part%20I.pdf
RAPID Stuff
• ODI Working Papers
• Bridging Research and
Policy Book
• Meeting series
Monograph
• RAPID Briefing Paper
• Tools for Policy Impact
• Communication Tools
• Policymaker Tools
• RAPID CDROM
• www.odi.org.uk/rapid