Transcript Assessment Training - Indiana Prevention Resource Center

Indiana Strategic
Prevention Framework
February 24, 2009
Chuck Klevgaard
Sheila Nesbitt
When thinking
about SPF,
do you ever
feel like this….?
2
Maybe youOr
fearthis…..?
being put on the spot?
3
Or this…..?
My life broken down into segments
4
Or maybe this…..?
5
Expectations
What are you hoping will happen
today (a topic/issue you want to learn
more about, someone you want to
network with, a question you
absolutely have to have answered)?
6
Overview of Day




Provide guidance on selecting strategies
Walk through one example together
Small groups walk through additional
examples
Discussion of next steps and additional
needs
7
Determining Fit
8
Conceptual Fit
Community
A “Good Conceptual Fit” intervention should:
 Fit into community logic model (alignment)
 Drive positive outcomes in identified substance
abuse problems (reach)
 Address the community’s intervening variables
 Be founded on evidence-based principles for
target population
 Target multiple opportunities for intervention
(sufficient mix)
9
Conceptual Fit
Screening for Alignment
Consequence
Condition
Consumption
Patterns
Mediating / Moderating
Variable
Strategies
Programs, Policies, and Practices
Underage Drinking
In preventing and
reducing
underage
drinking and
binge drinking,
there will
be a
reduction in
arrest rates
for driving
under the
influence,
public
intoxication
and
liquor law
violations.
Retail Availability of
Alcohol to Youth
Retail Availability of
Alcohol to Youth
Implement a Friday Night
Basketball Program
Administer Alcohol
Compliance Checks
The community logic model can be used
to check the alignment of interventions
10
Conceptual Fit
Screening for Reach
Consequence
Consumption
Patterns
Underage
Drinking
In preventing and
reducing underage
drinking and binge
drinking,
there will be
a reduction in
arrest rates
for driving
under the
influence,
public
intoxication
and
liquor law
violations.
Condition
Mediating / Moderating
Variable
Social Availability
of Alcohol to Youth
Social Availability
of Alcohol to Youth
Strategies
Programs, Policies, and Practices
Implement Shoulder
Taps With 6 of 47
Package Stores
Once a Year
Implement Shoulder
Taps With 40 of 47
Package Stores
Once a Year
The logic can also screen for strategies /
interventions that may not have sufficient
reach to create community level change
11
Developing a Comprehensive Approach
Sufficient Mix



Use multiple interconnected strategies in
order to reach community level change
Multiple strategies will complement and
reinforce each other
Consider number of people impacted in each
strategy
12
Practical Fit
Community






Feasible given a community’s resources,
capacities, and readiness to act
Add to or reinforce prevention strategies in the
community - synergism and layering
Consider community climate
Meet cultural needs of target population
Sustainable in community
Evaluability
13
Practical Fit
Synergy and Layering
Intervening
Variable
Existing
Programs
Limitations
Low
enforcement
of alcohol
laws
College Campus
Alcohol
Awareness
Program
Environmental
approach
doesn’t target
15-18 y/o
Possible
Additions
Expand current
environmental
approach to high
schools
Practical fit is assessed through a series of utility and
feasibility checks that grow out of the needs/resource
assessment and capacity-building activities
conducted in SPF Steps 1 and 2
14
Definitions of “Evidenced-Based”
Revised



Included on federal lists or registries of
evidence-based interventions;
Reported (with positive effects) in peerreviewed journals; and
Documented effectiveness supported by
other sources of information and the
consensus judgment of informed experts,
as described in the following set of four
guidelines.– all must be met
15
Four Guidelines for Documenting Effectiveness:
Revised
1.
2.
3.
Based in solid theory documented in a logic
or conceptual model; and
Similar in content and structure to
interventions that appear in registries or peerreviewed literature; and
The Intervention is supported by
documentation that it has been effectively
implemented in the past, and multiple times,
in a manner attentive to scientific standards of
evidence and with results that show a
consistent pattern of credible and positive
effects; and
16
Four Guidelines for Documenting Effectiveness:
Revised
4.
The intervention is reviewed and deemed
appropriate by a panel of informed
prevention experts that includes: wellqualified prevention researchers who are
experienced in evaluating prevention
interventions similar to those under review;
local prevention practitioners; and key
community leaders as appropriate ,e.g.,
officials from law enforcement and
education sectors or elders with indigenous
cultures.
17
Evidence-Based Interventions
Strength of Evidence


The strength of evidence of a tested
intervention will fall somewhere along a
continuum from weak to strong
The strength of evidence is assessed using
scientific standards and criteria for applying
these standards
18
Strength of Evidence
Components



Rigor of the study
Rigor and appropriateness of the methods
used to collect and analyze the data
The extent to which a finding can be
generalized to similar populations and similar
settings.
19
Indiana Gap Analysis Tool

Look at The Existing Approaches Tool




Pull Out Your Logic Model


Write Afternoons ROCK on line one
Identify 2-3 additional programs
Fill in the boxes as in the sample
Fill in Consequence, Focus Pop, and Conditions
Look at The Gaps Analyses Logic Model

Fill in Approaches, Limitations and Enhancements
20
Summary of Things to
Consider





Conceptual fit
Practical fit
Evidence of Effectiveness
Comprehensive approach
Reach
21
Conditions:
mediating, moderating,
intervening variables
Sheila Nesbitt
Figure 1-- Legend: Strength of (a) Evidence of relationship to underage drinking and (b)
Evidence of Population-Level Effect on Underage Drinking, Alcohol-related Problems or
Other Key Intermediate Variables Resulting from Prevention Interventions
Strong evidence of relationship and strong evidence of population
level prevention effects and or strong effect on other intermediate
variables which have population level prevention effects.
Strong evidence of relationship and moderate evidence of population
level prevention effects or moderate evidence of effect on key
intermediate variables which have population level effects
Strong evidence of relationship but only limited or no evidence of
population level prevention effects but evidence of target group
effects.
Theoretical but no empirical evidence of relationship and therefore no
evidence of population level or target group prevention effects
24
Strategy Selection


Out of two interventions, the one for which there
is stronger evidence of effectiveness should be
chosen, if the intervention is similar, equivalent,
and equally well-matched to the community’s
unique circumstances.
Reserve selecting an intervention with little or
weak evidence of effectiveness for situations in
which other interventions with stronger evidence
do not fit local circumstances.
25
Underage Drinking Outcome

In preventing and reducing underage
drinking and binge drinking,



there will be a reduction in arrest rates for
driving under the influence,
public intoxication and
liquor law violations.
26
mption
Conditions
Condition:
Retail availability
Strat
 Definition: Ease of accessing alcohol through onpremise (bars, restaurants) and off-premise (liquor,
convenience, grocery stores). Can refer to the presence and
density of alcohol outlets and the frequency of use of specific
commercial sources of alcohol by youth.
 Retailer compliance with laws: ability of underage
persons to purchase or consume, service to obviously
intoxicated.
 Sources of alcohol: actual sources of alcohol as
reported by young persons.
 Hours and days of sale
 Outlet density
 How have you assessed retail availability in your
community?
vening
iable
Strategy Practice
Policy, Program,

Alcohol Compliance Checks
 Rationale: Law enforcement confirmation
of whether outlets sell to underage person
and sanctions for illegal sales reduce
likelihood of sales
 Objective: Reduce illegal sales of alcohol
to person under 21 years of age.
 Allow local control and sanctions against
retailers that sell to underage persons.
 Increase retailer expectation that illegal sales
will result in consequences.
vening
iable
StrategyConceptual fit?
Compliance Checks:
 Fit into community logic model?
 Do compliance checks address the conditions:
mediating and moderating variables?
 Drive positive outcomes in identified
problems?
 Does this strategy impact the identified
consequence: reduction in arrest rates for driving
under the influence, public intoxication and liquor law
violations
 Will this strategy have sufficient reach to move
the needle in the community? # licensees x 2
checks (minimum), plus rechecks on failures within
90 days (Wagenaar, 2000)
vening
iable
StrategyConceptual fit?
Compliance Checks:
 Comprehensive Approaches (mix)
 Add to or reinforce a strategy in the community?
 Target multiple opportunities for intervention?
vening
iable
Strategy
Compliance Checks:
Practical Fit ?
Feasible given a community’s resources, capacities, and
readiness to act?
-
What would the process be for establishing this approach?
What tasks would need to be accomplished leading to
implementation?
 Fidelity


Can you implement in the manner it was designed?
Is there support (Training, TA, materials) for
implementation? (PIRE and U of Minnesota
implementation guides)
 Consider community climate


Does the approach account for readiness?
Does the approach meet cultural needs?
vening
iable
Compliance Strategy
Checks: Practical Fit?
 Sustainability
 Does the community have resources to sustain
this initiative?
vening
iable
Strategy
Compliance Checks:
Effectiveness?
 Evidence-based principles for target population




Is the approach research-based on a well defined
theory or model?
Is there documented evidence of effectiveness
(such as formal evaluation results)?
Have the results been replicated successfully by
different researchers over time?
Has the approach been shown to be effective for
areas similar to those you will address?
Small Group Discussions

Discuss examples specific to your priority
substance (cocaine or alcohol).
34
Large Group Discussion
35
Decision Matrix
Conceptual Fit
Practical Fit
(community resources readiness)
(Logic model, conditions, alignment, and reach)
Sufficient
Limited
High
Low
Highest Priority?
Less Priority?
High Priority?
Lowest Priority?
Balancing Art and Science
 Research, tools, and checklists will only get you so far.
 At some (hopefully many!) points, you need to see how
the big picture is coming together.
We hope that now you
feel less like this...
Yikes!
And more like this…. Ahhhhh!
39
Thank
YOU
for your
kind attention.
We hope to
see you again
sometime!
Please
complete the
evaluation.
40
Thank You!!

For questions, comments
Sheila Nesbitt
CSAP’s Central CAPT
1800-782-1878 x128
[email protected]
41