Transcript Degree to Which Innovation Defines the Organization

Evidence-Based
Social Emotional Practices:
Fidelity and Sustainability
Concerns
Phillip S. Strain, Ph.D.
University of Colorado Denver
What Makes a Practice
Evidence-Based?
1. Evaluated in a minimum of two randomized
trials or three within subject designs with
replication across a minimum of 9
participants (APA, CEC)
2. Practice is manualized (APA, CEC)
3. Practice produces effects that are
consistently found across participants,
settings, implementers, etc. (CEC, Joseph &
Strain, 2003)
What Makes a Practice
Evidence-Based?
4. There is a protocol for determining when the
practice is implemented to fidelity (e.g., TPOT) (APA, CEC, Joseph & Strain, 2003)
5. There is a defined process for training new
implementers to fidelity (Joseph & Strain,
2003)
Some S/E Practices that Meet
All These Criteria
• First Steps
• Incredible Years
• Peer-Mediated Social Skills Intervention
• Incidental Teaching
• Prompting and Reinforcement
• Pivotal Response Training
• Individualized Assessment-based PBS
• CSEFEL Tiered Model (in full)
Selection of Practices that are
Evidence-Based
While there is no set formula for
determining which intervention strategy to
use to meet a particular intervention need,
we suggest that the following questions can
help narrow the field.
Selection of Practices, cont.
• Has the intervention been evaluated in a
peer-reviewed journal?
• Has the intervention been replicated across
investigators, settings, and participants?
• Are there alternative interventions that are
less restrictive, better researched, or perhaps
more effective or efficient?
Selection of Practices, cont.
• Is the intervention within the existing skill
set of practitioners, or do they need prior
training and consultation?
• Has the intervention been shown to
produce outcomes like the ones intended?
• How will we evaluate the intervention if
we decide to implement?
What Do Evidence-Based
Practitioners Do?
Evidence-based practitioners engage in the
following ongoing activities that are consistent with
and necessary for the use of evidence-based
practices.
• To maintain awareness of evidence-based
practices through ongoing education, including
reading current professional journals, books, and
other materials; accessing web sites devoted to
evidence-based practice (such as
www.challengingbehavior.org); and participating
in workshops on evidence-based practice.
What Do Evidence-Based
Practitioners Do?
• To select overall curricula that have peer-reviewed
data to support use with a particular population of
children.
• To employ daily data collection systems that track
children's progress and use this information to
plan and refine instruction.
• To provide families with support, information, and
training sufficient to meet their desires for
participation in their child’s educational program.
What Do Evidence-Based
Practitioners Do?
• To remain open to changes in service delivery
based on new ideas, new data, and trends in the
field that are evidence-based.
• To access learning opportunities to enhance
instructional, administrative, and interpersonal
skills that are evidence-based.
• To promote the use of evidence-based practices by
the staff you supervise. Supervisors should
encourage staff to learn about evidence-based
practices, try new evidence-based approaches, and
engage in an array of continuous professional
development activities.
Fidelity: The Difference Between Doing
Something and Doing Something
Correctly
What Fidelity is NOT!
What Fidelity IS!
• Buying a curricula
• Buying a curricula and
reading it
• Going to a workshop
• Incorporating a practice
into a mission statement,
job description or PR
piece
• Getting better at a
practice
• Doing a practice(s)
exactly how it is intended
with everyone and all the
time.
Challenges to Fidelity
•
•
•
•
Individualistic models of service delivery.
Poor availability for coaching support.
Lack of administrative support.
Disconnect between hiring/promotion policies
and fidelity of practices.
• Length of time to reach fidelity.
Some Important
Questions About Fidelity
• What is the usual “trend” toward fidelity?
• What is the data supporting EBP
blending?
• How does one decide about certification?
What Ensures Sustainability?
Degree to Which Innovation
Defines the Organization
• “The headline”
• No Compromise
• Staff come and go, innovation remains
Proactively Rejecting the
“Latest” Thing
• Data-based
• OK for others
• Reaffirming commitment
Embodiment of Innovation in
Policy and Practice
• Hiring criteria
• Promotion criteria
• Organizational mission statement
Behavior of Leader During Crisis
• Opportunity versus challenge
• Rallying advocates
• Dismissing rumors
Quality Control Over Innovation
• Can’t be a “cowboy” unless you walk, talk,
act and dress like one
• Recognition of excellence
• Re-training with reference to “standard”
Grateful Client Giving Systematized
• Dollars (fund raised outside standard
funding stream)
• Time given in services
• Advocacy
Maintained Bipartisan
Political Support
• Advisory board
• Fund raising
• Limited public statements vis-à-vis
politics
Public Promotion of Outcomes
(not services)
• Local media several times per year
• Developed research/evaluation
collaborations with
consultants/universities
• Showed powerful video to non-traditional
groups (churches, business groups,
city/county governing body)
Discussion Items
• If moving toward fidelity is non-linear, more
like fits and starts, more like a threshold, then
what are the implications for supporting staff?
• With any adult learning curve (change in
behavior), efficiency or output is likely to
suffer.
• There are natural “friends of change” and
“enemies of change.” How do we capitalize on
both?
References
• Joseph, G.E. & Strain, P.S. (2003). Comprehensive
evidence-based social-emotional curricula for young
children: An analysis of efficacious adoption potential.
Topics in Early Childhood Special Education, 23, 65-76.
• Horner, R.H., Carr, E.G., Halle, J., McGee, G.G., Odom,
S., & Wolery, M. (2005). The use of single subject
research to identify evidence-based practices in special
education. Exceptional Children, 71, 165-179. (CEC)
• Lonigan, C.J., Elbert, J.C., & Bennett-Johnson, S. (1998).
Empirically supported psychosocial interventions for
children. Journal of Clinical Child Psychology, 27, 138145. (APA)