Transcript National Registry for Evidenced Based Programs and Practices

1
“THE LISTS” FOR
EVIDENCE BASED
PRACTICES
A Brief Guide to Resources to Review
Evidence Based Programs for Youth
2
The Short List of Lists
• National Registry for Evidence Based Programs
and Practices- NREPP
• Crimesolutions.gov & OJJDP Models Program
Guide
• Blueprints for Healthy Youth Development
• Coalition for Evidence Based Policy
NATIONAL REGISTRY FOR
EVIDENCED BASED
PROGRAMS AND PRACTICES:
NREPP
HTTP://NREPP.SAMHSA.GOV/INDEX.ASPX
Substance Abuse and Mental Health Services
Administration SAMHSA
5
What Interventions are on NREPP?
• All interventions in the registry have met NREPP’s
minimum requirements for review and have been
independently assessed and rated for Quality of Research
and Readiness for Dissemination.
• NREPP is a voluntary, self-nominating system in which
intervention developers elect to participate. There will
always be some interventions that are not submitted to
NREPP, and not all that are submitted are reviewed.
6
What is Included?
General information about the intervention
A description of the research outcomes reviewed
Quality of Research and Readiness for Dissemination
ratings
A list of studies and materials reviewed
Contact information to obtain more information about
implementation or research
7
How to Use NREPP
• NREPP can be a first step to promoting informed decision making.
• The information in NREPP intervention summaries is provided to help
you begin to determine whether a particular intervention may meet your
needs.
• Direct conversations with intervention developers and others listed as
contacts are advised before making any decisions regarding selection or
implementation of an intervention.
• A list of potential questions to ask developers is available from NREPP to
facilitate these conversations.
• NREPP ratings do not reflect an intervention's effectiveness. Users
should carefully read the Key Findings sections in the intervention
summary to understand the research results for each outcome.
8
Review Process
• NREPP staff identify the reviewers.
• NREPP staff send review packets to two pairs of
reviewers. One pair of reviewers focuses on Quality of
Research, while the other pair looks at Readiness for
Dissemination.
• Each of the reviewers independently reviews the
materials provided and calculates ratings using the
predefined Quality of Research and Readiness for
Dissemination review criteria.
• The reviewers submit their ratings to NREPP.
• If their ratings differ by a significant margin, NREPP staff
may hold a consensus conference to discuss and resolve
the differences.
9
Quality of Research Review
• Each reviewer independently evaluates the Quality of
Research for an intervention's reported results using the
following six criteria:
1.
2.
3.
4.
5.
6.
Reliability of measures
Validity of measures
Intervention fidelity
Missing data and attrition
Potential confounding variables
Appropriateness of analysis
• Reviewers use a scale of 0.0 to 4.0, with 4.0 being the highest
rating given.
10
Readiness for Dissemination Review
• Each reviewer independently evaluates the intervention's
Readiness for Dissemination using the following three
criteria:
1.
2.
3.
Availability of implementation materials
Availability of training and support resources
Availability of quality assurance procedures
• Reviewers use a scale of 0.0 to 4.0, with 4.0 being the
highest rating given.
13
Reporting the Results
•
NREPP staff compile the ratings and descriptive
information into an intervention summary that is shared
with the developer.
•
Once the developer approves the intervention
summary, SAMHSA reviews the summary and publishes it
on the NREPP Web site.
15
Crimesolutions.gov
• On CrimeSolutions.gov you will find:
• Research on the effectiveness of programs and practices
as reviewed and rated by Expert Reviewers
• Easily understandable ratings based on the evidence that
indicates whether a program or practice achieves its
goals: (Program Review and Rating from Start to Finish
and Practice Review and Rating from Start to Finish)
• Effective
• Promising
• No Effects
• Profiles of programs and practices with research findings
16
20
Blueprints for Healthy Youth Development
• Blueprints is a project at the Center for the Study and
Prevention of Violence at the University of Colorado
Boulder. It identifies, recommends, and disseminates
programs for youth that, based on scientific evaluations,
have strong evidence of effectiveness.
• Blueprints serves as a resource for governmental
agencies, schools, foundations, and community
organizations trying to make informed decisions about
their investments in youth programs.
• Its ultimate goal is to reduce antisocial behavior and
promote a healthy course of youth development.
21
Criteria for Blueprints
Blueprints considers four criteria:
• Evaluation quality—Can we be confident in a program’s
evaluation
• Intervention impact—How much positive change in key
developmental outcomes can be attributed to the
intervention
• Intervention specificity—Is the intervention focused,
practical, and logical
• Dissemination readiness—Does the program have the
necessary support and information to be successfully
implemented
22
Model and Promising Designations
• Both promising and model programs meet basic
Blueprints standards, but model programs meet additional
requirements.
• Promising programs must have evidence from an
experimental or two quasi-experimental designs, clear
findings of positive impact, carefully defined goals, and
sufficient resources to help users.
• Model programs must, in addition, have evidence from an
experimental design, a sustained impact, and a multiple
site replication. Model programs are deemed ready for
widespread use.
26
Coalition for Evidence Based Policy
“Top Tier” Standard
• Interventions shown in
well-designed and
implemented
randomized controlled
trials, preferably
conducted in typical
community settings, to
produce sizable,
sustained benefits to
participants and/or
society.
“Near Top Tier” Standard
• Interventions shown to
meet almost all elements
of the Top Tier standard,
and which only need one
additional step to qualify.
• This category includes, for
example, interventions that
meet all elements of the
standard in a single site,
and just need a
replication trial to confirm
the initial findings and
establish that they
generalize to other sites.
27
Promising Practice Network
28
29
Common Features
• A starting point
• Searchable by criteria
• Describe the criteria for selection
• Intended for multiple stakeholders: funders, providers,
policy makers
• Use with caution and care
30
MacArthur Models for Change Resource
of JJ and BH
31
Weblinks
• http://www.ojjdp.gov/mpg/
• http://www.blueprintsprograms.com/
• http://cfc.ncmhjj.com/
• http://www.promisingpractices.net/default.asp