Transcript Using Evidence in Your Work

Using Evidence in
Your Work*
From Evidence to Action
A CIHR Funded Project
*Based on a presentation for the National RAI Forum: "Making the Most of It“: Using
Evidence in Your Work, by Sarah Bowen, Ph.D. (Winnipeg, MB, May 12, 2006)
WHAT IS EVIDENCE?
Information that comes closest to the facts of a matter
- the form it takes depends on the context or nature of the problem
Findings of high-quality, methodologically appropriate
research are most accurate evidence
- but because research is often incomplete and sometimes
contradictory or unavailable, other kinds of information are
necessary supplements to or stand-ins for research
The evidence base for a decision
- multiple forms of evidence combined to balance rigour with
expedience—while privileging the former (CHSRF)
From: CHSRF
Conceptualizing and
Combining Evidence for
Health System Guidance
Values
Pragmatics &
Contingencies
Lobbyists &
Pressure Groups
Political
Judgment
Scientific
Evidence
Resources
Habits &
Tradition
Professional
Experiences &
Expertise
EVIDENCE-BASED OR
EVIDENCE-INFORMED?
• Evidence-based: roots in quantitative
research, particularly systematic
reviews, clinical trials
• Evidence-informed: recognizes other
factors affecting decision-making such
as an incomplete evidence base
IS SOME EVIDENCE BETTER?
A humorous look at hierarchies
Class 0: Things I believe
Class 0a: Things I believe despite the available data
Class 1:
Class 2:
Class 3:
Class 4:
Class 5:
Randomized controlled clinical trials that
agree with what I believe
Other prospectively collected data
Expert opinion
Randomized controlled clinical trials that
don’t agree with what I believe
What you believe that I don’t
– Levels of Belief: From Shaughnessy & Slawson, 2004
IS SOME EVIDENCE BETTER? Cont.
• Quantitative concept is dominant in health
• Emphasis is on experimental & clinical
evidence (e.g. Cochrane Collaboration)
• Hierarchies of Evidence
•
•
•
•
systematic reviews & meta-analyses
well-designed Random Control Trials
well-designed non randomized studies
well-designed multi centre non-experimental
studies
• expert opinion, descriptive studies
HIERARCHY CONCEPT LIMITS
EVIDENCE-BASED HEALTH
SERVICES
• Important perspectives of qualitative and
mixed methods devalued
• Difficulties achieving successful evidencebase for planning and decision-making
may overwhelm the importance of doing
so
• Diminished recognition of importance of
“context” for transferability of findings
LEVELS OF EIDM
LEVEL
CLINICAL
PROGRAM
POLICY
EXAMPLE of EVIDENCE USE DECISION-MAKERS
• Practice guidelines
• Program design
• Priorities for services
Direct care staff
Managers
Managers
Board & Senior
Executive
CHALLENGES DIFFER DEPENDING
ON LEVEL OF DECISION
• Clinical
– What is “best” practice?
– Implementing guidelines & standards
• Program
– Design (What does a program look like?)
– Program Implementation
• Policy direction
– Getting an issue “on the agenda”
– Informing a policy trajectory
BARRIERS TO EVIDENCE USE
Some barriers differ according to level:
– Gap between research & practice
– Differences between clinical & management:
• Culture
• Type of evidence
• Type of decisions, decision-making process
BUT there are a number of common barriers....
COMMON BARRIERS:
Inadequate
relevant, high
quality
information
Crisis
Management
culture
BARRIERS ARE NOT JUST
EXTERNAL
External (hard to modify)
• Political context
• Funding decisions
– Resource allocation
• External requirements
• Crisis management
culture
• Lack of information
• Perverse incentives
Internal (modifiable)
• Team/org culture or
structure
• Leadership
• Competing priorities
• Research “literacy”
• Acceptance of inevitability
of crisis management
culture
• Resistance to change
• Insufficient time allocated
What does it mean that there is
“not enough time”?
• Not an organizational priority? (“there is always
time for the important things”)
– Resource allocation
– Aligning resources and processes with stated values and
goals
– Viewing as an “add-on”
• Not an individual priority? (“how can you have
enough time to do it over if you don’t have
enough time to do it right?”)
ORGANIZATIONAL/UNIT/TEAM
FACTORS
• Lack of mechanisms for consultation and
input
• Centralized decision-making
• Leadership style
• Barriers to information flow
• Lack of transparency & accountability of
decision-making processes
• Lack of processes & structures to allow
“reflection” time
WHAT WORKS IN PROMOTING
USE OF RESEARCH EVIDENCE?
• Collaboration
• Relationships
• “Personality factor”
• Leadership commitment to EI change
• Resource availability
• Willingness to change processes
• Sustainable interventions
Decisions AND implementation
USING EVIDENCE IN YOUR WORK
• Determine the level
– Policy
– Program
– Clinical
• Clarify the challenge
1. Determining ‘best’ practice
2. Getting support for ‘best’ practice
3. Implementing ‘best’ practice
DETERMINING ‘BEST’ PRACTICE
• Assess the research
– Literature analysis
– Systematic reviews weighted
– The trap of “decision-based evidence
making”
• Evaluate for your setting
– Transferability, context, resources, values
• Identify gaps
• Identify local evidence, experience of
other programs (e.g. QI, evaluation
activities)
GETTING SUPPORT FOR ‘BEST’
PRACTICE
•
•
•
•
•
•
•
Identify champions
Align with strategic priorities
Use existing activities and structures
Build collaborative relationships
Integrate research with local evidence
Speak to audience/stakeholder concerns
Use effective communication strategies
IMPLEMENTING THE EVIDENCE
• Recognize implementation challenge
– Plan, accountability, resources
– “the personality factor”
– “change management”
• Communicate evidence effectively
– Link to personal, professional objectives
• Build collaborative relationships
– Respect expertise of stakeholders
• Implementation & formative evaluation
– Positive forum for identifying barriers
SUMMARY
• Evidence informed decision-making is
important at all levels of health care
provision: clinical, planning and policy
• Even with limited resources, we have
opportunities to address internal barriers to
evidence-informed decision-making
• Evidence should be used at all three stages
of decision-making: determining a solution,
getting support for a proposed solution, and
implementing the decision.