Transcript From Evidence-based Practice to Practice

From Evidence-based Practice to
Practice-based Evidence: Behavior
Analysis in Special Education
Ronnie Detrich
Wing Institute
What is Evidence-based Practice?
• EBP is a decision-making approach that
places emphasis on evidence to:
 guide decisions about which interventions to
use.
 evaluate the effects of an intervention.
Sackett et al (2000) defined evidence-based practice in
medicine as: “the integration of best research evidence
with clinical expertise, and patient values.”
What is Evidence-based Practice?
• Ultimately, EBP is a consumer protection
issue.
 Assumes that evidence-based interventions are
more likely to be effective than interventions
that are not evidence-based.
 By validating interventions as evidence-based
there is the implication that there are standards
for reviewing interventions.
Standards should be transparent.
The Scope of the Problem
• Kazdin (2000) identified 550 named
interventions for children and adolescents.
 A very small number of these interventions have
been empirically evaluated.
 Of those that have been evaluated, the large
majority of them are behavioral or cognitivebehavioral.
The Scope of the Problem
• We have a great deal of evidence but much
of that evidence is not being applied in
practice settings.
 Evidence-based interventions are less likely to
be used than interventions for which there is no
evidence or there is evidence about lack of
impact (Kazdin, 2000).
 In many instances practitioners are not aware
of evidence-based interventions (Kratochwill,
Albers & Shernoff, 2004).
 There is very little guidance to parents and
practitioners about how to decide among
interventions.
Why the Increased Interest in
Evidence-based Interventions?
• No Child Left Behind (NCLB) requires that
interventions used to improve educational
performance are based on scientific
research.
 In NCLB there are over 100 references to
scientific research.
• Individuals with Disabilities Education
Improvement Act [IDEIA] (2004) requires
that interventions are scientifically based
instructional practices.
Special Education and Evidence-based
Interventions
• Pre-service and professional development
for all who work with students with
disabilities to ensure such personnel have
the skills and knowledge necessary to
improve the academic achievement and
functional performance of children with
disabilities, including the use of
scientifically based instructional practices,
to the maximum extent possible.
Special Education and Evidence-based
Interventions
• Scientifically based early reading programs,
positive behavioral interventions and
supports, and early intervention services to
reduce the need to label children as
disabled in order to address the learning
and behavioral needs of such children.
Special Education and Evidence-based
Practice
• The Individualized Education Program
(IEP) shall include a statement of the
special education and related services and
supplementary aids and services, based on
peer-reviewed research to the extent
practicable, to be provided to the child, or
on behalf of the child, and a statement of
the program modifications or supports for
school personnel that will be provided for
the child.
Ethical Conduct and Evidence-based
Interventions
• Most national psychological and
educational organizations have ethical
standards requiring science-based practices
to address problems.
 American Psychological Association Ethical
Standard 2.04:
Psychologists’ work is based on the established
scientific and professional knowledge of the discipline.
Ethical Conduct and Evidence-based
Practice
• National Association of School
Psychologists
 Standard III F 4.
School psychology faculty members and clinical or
field supervisors uphold recognized standards of the
profession by providing training related to high
quality, responsible, and research-based school
psychology services.
Ethical Conduct and Evidence-based
Practice
• National Association of School
Psychologists

Standard IV 4.
School psychologists use assessment techniques,
counseling and therapy procedures, consultation
techniques, and other direct and indirect service
methods that the profession considers to be
responsible, research-based practice.
Ethical Conduct and Evidence-based
Practice
• Behavior Analysis Certification Board
 Standard 2.09a
The behavior analyst always has the responsibility to
recommend scientifically supported, most effective
treatment procedures. Effective treatment procedures
have been validated as having both long-term and
short-term benefits to clients and society.
 Standard 2.09b
Clients have a right to effective treatment (i.e., based
on the research literature and adapted to the
individual client).
Controversies in Evidence-based
Education
• There is no consensus about what
constitutes evidence.
 The definition offered in NCLB permits both
quantitative and qualitative evidence without
specifying the types of questions that each
approach best answers.
 In this context, we are most often concerned
with evidence that establishes a causal relation
between an intervention and a class of social or
academic behaviors.
.
Controversies in Evidence-based
Education
• There is no agreed upon standard for either
the quantity or the quality of evidence
necessary to validate an intervention as
being evidence-based.
 Several organizations have established
standards but there is limited agreement among
them.
It is possible for an intervention to meet one standard
but not a second.
Applied Behavior Analysis and
Evidence
• The research methods of single participant
design are convincing demonstrations of
causal relations.
• Behavior analysis research has often been
criticized for limited generalizability
because of the small number of participants
in a research study.
Applied Behavior Analysis and
Evidence
• Generally, the developed standards for
validating an intervention as evidencebased have relegated single participant
designs to a lower quality of evidence.
 This is problematic because in some areas of
educational research almost all of the evidence
is based on these designs.
Autism
Developmental Disabilities
Applied Behavior Analysis and
Evidence
• There are no established standards
within applied behavior analysis for
validating interventions.
 There is no single resource that decision-makers
can turn to for guidance about the best
intervention to use for a particular problem.
May lead decision makers to base decisions on criteria
other than evidence.
Applied Behavior Analysis and
Evidence
 Some related fields have published standards:
Horner, et. al. (2005) provides guidelines for
evaluating evidence established with single
participant research.
Flay, et. al. (2005) standards of Society for Prevention
Research.
National Standards Project is developing standards of
evidence for interventions for individuals with autism.
Task Force on Evidence-based Interventions in School
Psychology.
What Works Clearinghouse.
 Only WWC has reviewed the literature in a
specific area to provide guidance about effective
interventions.
Implementation: Research to Practice
Where good interventions go to die
• Identifying evidence-based interventions is
necessary but not sufficient to assure they
will be implemented in practice settings.
• It is also necessary to address complex
issues associated with implementation
(research to practice).
Four Concerns for Implementation
• Dissemination of evidence-based
interventions in a manner that is accessible
and useful to decision-makers.
 Publication in professional journals is not an
effective dissemination strategy for decision
makers.
Evidence about interventions are distributed across
multiple journals.
Most practice level decision-makers are not adequately
trained to evaluate primary source data.
Because of time constraints, reading professional
journals is a low priority activity for many decisionmakers.
Four Concerns for Implementation
• Selection of evidence-based interventions.
 Often variables unrelated to effectiveness
influence selection of an intervention.
Cost
Expert opinion
Personal experience
Effort associated with systems change
Four Concerns for Implementation
• Initial Implementation of new intervention
 What is necessary to gain practitioner support?
Often inconsistent with previous training and
experience.
 What is required to train practitioners?
 What is necessary to assure integrity of
implementation?
Implementation without integrity is implementation
of an unspecified intervention with unknown effects.
 How can new intervention be adapted to meet
local circumstances?
If changed too much it becomes a different
intervention for which there are no data.
If not modified to meet local circumstances it may not
be implemented at all.
Four Concerns for Implementation
• Sustainability is the ultimate goal for
successful interventions.
 Definition of sustainable programs:
Maintains over time.
Maintains across generations of practitioners.
Supported with existing resources of system.
 If effective interventions are not sustainable
then the risk is great that alternative,
ineffective programs will emerge.
 The larger the scale of implementation the more
complex all of the issues become and
sustainability is more difficult to achieve.
Applied Behavior Analysis and
Implementation
• Service delivery in applied behavior
analysis is a mediated model.
 Behavior Analysts coach parents, teachers, or
other mediators to implement an intervention
plan.
 Mediated model requires that behavior analysts
address many of the issues of implementation
for each project.
Progress Monitoring
Practice-based Evidence
• Even if an evidence-based intervention is
implemented well, it is still necessary to
systematically evaluate the impact.
 Progress monitoring is practice-based evidence
about the effects of an evidence-based practice.
 It is difficult to predict which students will
benefit from intervention prior to
implementation.
 No intervention will be effective for all students.
 Progress monitoring is consistent with ethical
guidelines.
Ethical Standards and Progress
Monitoring
• National Association of School
Psychologists
 Standard IV C 1b.
Decision-making related to assessment and
subsequent interventions is primarily data-based.
 Standard IV 6.
School psychologists develop interventions that are
appropriate to the presenting problems and are
consistent with the data collected. They modify or
terminate the treatment plan when the data indicate
the plan is not achieving the desired goals.
Ethical Standards and Progress
Monitoring
• Behavior Analysis Certification Board
 Standard 4.04
The behavior analyst collects data or asks the client,
client-surrogate, or designated other to collect data
needed to assess progress within the program.
 Standard 4.05
The behavior analyst modifies the program on the
basis of data.
Progress Monitoring and Special
Education
• Progress monitoring is fundamental to the
IEP process.
• Progress monitoring is at the heart of
Response to Intervention (RTI).
 All students are routinely and systematically
monitored to determine if they are making
adequate progress against established
standards.
Progress Monitoring and Behavior
Analysis
• Data collection and review is the sine qua
non of applied behavior analysis.
 It is not applied behavior analysis if
intervention data are not collected and reviewed
to determine impact.
 Behavior analysts rely on direct measures of
behavior and its products to determine progress.
Words read correctly as a measure of reading ability.
Frequency of specific problem behaviors or adaptive
behaviors.
Progress Monitoring and Behavior
Analysis
• Behavior Analysis best represents the
relationship between evidence-based
practices and practice-based evidence.
 For many areas in special education there are a
wide range of behavior analytic evidence-based
interventions.
 The fundamental emphasis on progress
monitoring in behavior analysis makes it well
suited for leading the movement to a
comprehensive approach to evidence-based
practices.