BCURE Evidence-Informed Decision-Making Capacity Building Workshop 1st and 2nd June 2015 Pretoria, South Africa What Is Evidence, and How Can It Improve Decision Making? Philip.
Download ReportTranscript BCURE Evidence-Informed Decision-Making Capacity Building Workshop 1st and 2nd June 2015 Pretoria, South Africa What Is Evidence, and How Can It Improve Decision Making? Philip.
BCURE Evidence-Informed Decision-Making Capacity Building Workshop 1st and 2nd June 2015 Pretoria, South Africa What Is Evidence, and How Can It Improve Decision Making? Philip Davies International Initiative for Impact Evaluation [3ie] Philip Davies www.3ieimpact.org Group Exercise What is Evidence? Philip Davies www.3ieimpact.org Some Features of Evidence • Evidence is information or data that supports, or rejects, a conclusion • Evidence has standards of validity and reporting • Evidence is almost always probabilistic • Often disagreement on what counts as ‘evidence’ • Evidence is rarely self-evident • Evidence can be about what is generalisable – and what is particular • Hence evidence in both quantitative and qualitative • Not all research is of equal value/sufficient quality • Single studies can misrepresent the balance of evidence • Hence, the need for systematic reviews/synthesis of evidence Philip Davies www.3ieimpact.org Different Notions of Evidence Policy Makers’ Evidence Knowledge Translation and Transfer Researchers’ Evidence •Colloquial (Narrative) •‘Scientific’ (Generalisable) •Anything that seems reasonable •Proven empirically •Policy relevant •Theoretically driven •Timely •As long as it takes •Clear Message •Caveats and qualifications Source: J. Lomas et al, 2005 Philip Davies www.3ieimpact.org UK Policymakers’ Views of Evidence • Focus on the ‘end product’, rather than how the information was either collected or analysed • Use of ‘anecdotal’ evidence (“tells a story”) • Drawing on such things as ‘real life stories’, ‘fingers in the wind’, ‘local’ and ‘bottom-up’ evidence But: • “If we try and move anywhere without having the scientific basis to do so we get fleeced in the House” • And: DfID Evidence into Action Team • And: BCURE Programme + DPME (South Africa) Philip Davies www.3ieimpact.org Factors Other Than Evidence Values, Beliefs and Pragmatics & Ideology Contingencies Experience & Expertise Lobbyists & Evidence Pressure Groups Habits & Bureaucratic Culture Philip Davies Judgement Resources www.3ieimpact.org Challenge for this Workshop How can we integrate the use of evidence with these other influencing factors? Philip Davies www.3ieimpact.org How Can Evidence Influence Policy? • Help people make better decisions and achieve better outcomes • Provide a better understanding of the nature, size and dynamics of the issue at hand • Compare policy options objectively and rigorously • Clarify how a policy is supposed to work (ToC) • Calculate empirically the size of likely impacts • Provide more precise estimates of risk and bias • Calculate empirically outcomes and impacts the diversity/variance of • Establish a cumulative evidence base for decision making. Philip Davies www.3ieimpact.org Group Exercise: What is Evidence versus Opinion? Philip Davies www.3ieimpact.org Evidence versus Opinion Evidence: • Data that meet the standards of internal validity, adequacy of reporting, and external validity • Key qualities: independence, objective, verifiable Opinion • Statements and claims that do not meet the standards of evidence • Key qualities: positional, subjective, partial (selective), hard to verify Philip Davies www.3ieimpact.org Evidence-Based vs Opinion-Based Policy Welfareto-Work Opinion-Based Policy Maths Teaching Evidence-Based Policy Hoodies Increasing Pressure/Time Philip Davies www.3ieimpact.org Thank you Philip Davies Email: [email protected] +44 (0)207 958 8350 Visit www.3ieimpact.org Philip Davies www.3ieimpact.org