Transcript Evidence Based Practice

Evidence Based Practice
Steven R. Pruett, Ph.D, CRC
SERNRA Conference
May 18, 2005
Evidence-Based Practice
Since the passage of the HMO Act of
1973, there have been many changes in
the American healthcare system.





Consumer-driven
Market-based
Customer choice
Customer satisfaction
Health outcomes
Evidence-Based Practice
Rehabilitation Healthcare systems,
researchers, practitioners and other
stakeholders need to provide evidence
to support the effectiveness of services.

Particularly relevant in rehabilitation has
has given rise to the emphasis on
consumerism, consumer involvement, and
program evaluation.
Evidence-Based Practice
One of the ways healthcare providers
have responded to these managed care
system demands is through evidence-
based practice.
Definitions
Rosenberg and Donald (1996) defined evidencebased medicine as a process of turning clinical
problems into questions and then systematically
locating, appraising, and using contemporaneous
research findings as the basis for clinical decisions.
Sackett, Rosenberg, Gray, Haynes, and Richardson
(1996) described evidence-based practice as the
conscientious, explicit, and judicious use of current
best evidence in making decisions about the care of
individual patients.
Definitions
DePalma (2000) further refined the definition of evidence-based
practice as a complete process beginning with knowing what
clinical questions to ask, how to find the best practice, and how
to critically appraise the evidence for validity and applicability to
the particular care situation. The best evidence then must be
applied by a clinician with expertise in considering the patient's
unique values and needs. The final aspect of the process is
evaluation of the effectiveness of care and the continual
improvement of the process.
Ottenbacher and Mass (1998) indicated that the “best evidence”
used to support evidence-based practice is derived from a series
of research studies results in an empirical consensus regarding
the effectiveness of a treatment approach.
Evidence-Based Practice &
Rehab Case Management
Limited empirical support for rehab case mgt
interventions.

Central focus of this service is to evaluate,
managed & coordinate services for PWDs
(Chronister, da Silva Cardoso, Lee, Chan & Leahy, 2005)
 Medical Case Mgt assesses, plans, implements
coordinates, monitors & evaluates options and services
to meet individual’s health needs through
communication and available resources to promote
quality & cost-effective outcomes. (Mullahy, 1995)
 Voc case mgt is the movement of a client through the
rehab process & includes the mgt and coordination of all
services needed to achieve successfully the rehab goal.
(Cox, Connolly & Flynn, 1981)
Evidence-Based Practice &
Rehab Case Management
So…case management emphasizes:



Evaluation
Outcomes
Quality assurance
Therefore…case managers are already
working, in part, as evidence-based
practitioners without explicitly
recognizing it.
Rehab and Evidence-Based
Practice
Quality Assurance – standards of practice



Case Management Society of America
Individual Case Management Assoc & Aetna
Health Plan developed Case Management Practice
Guidelines.
CRCC code of professional ethics.
Decision-making process

Evaluation needs to be based on credible scientific
evidence versus subjective experience.
Rehab and Evidence-Based
Practice
Rehabilitation researchers must conduct
systematic research to demonstrate the
effectiveness of rehabilitation counseling/case
management interventions.
RCs/case managers must be able to assist
clients in selecting the most appropriate
medical, psychological, educational, social
and vocational interventions for a client’s
particular situation.
Rehab and Evidence-Based
Practice
To better practice evidence-based
rehabilitation RC/case managers must
have the following knowledge:



Research design
Statistics
How to access pertinent research
Evidence-Based Practice
General Concepts
Gold standard of best practice in medicine

In medicine, with its positivist scientific methods
tradition, the “gold standard” for scientific
evidence is still randomized clinical trials and the
method of choice for determining the cumulative
evidence of the effectiveness of a treatment is
meta-analysis.
 Randomized clinical trials
 Meta-analysis
Evidence Based Practice
Steps for the evidence-based practice of
medicine by practitioners:
1.
2.
3.
4.
Formulate a clear clinical question from a patient's problem.
Search the literature for relevant clinical articles.
Evaluate (critically appraise) the evidence for its validity and
usefulness.
Implement useful findings in clinical practice.
(Rosenberg & Donald, 1996)
Evidence-based Practice:
Some Rehab Questions
What processes/techniques make a
specific rehabilitation intervention work?
For whom is the intervention most
effective?
Are certain interventions/programs
better for certain populations?
Who should receive a specific
intervention or program? When? And
for how long?
A Hierarchy of Levels of Best
Evidence
Level 1 evidence is defined as strong evidence from
at least one systematic review of multiple welldesigned randomized controlled trials.
Level 2 evidence is defined as strong evidence from
at least one properly designed randomized controlled
trial of appropriate size.
Level 3 evidence is defined as evidence from welldesigned trials without randomization, single group
pre-post, cohort, time series, or matched casecontrolled studies.
A Hierarchy of Levels of Best
Evidence
Level 4 evidence is defined as evidence from welldesigned non-experimental studies from more than
one center or research group.
Level 5 evidence is defined as opinions of respected
authorities, based on clinical evidence, descriptive
studies, or reports of expert committees.
Related Concepts – Empirically
Supported Treatment
APA Division 12 defined empirically supported
treatment (EST) as clearly identified psychological
treatments shown to be efficacious in controlled
research studies with a delineated population.
EST should be evaluated in terms of efficacy
(statistical and clinical significance), effectiveness
(clinical utility), and efficiency (cost-effectiveness).
Related Concepts – Empirically
Supported Treatment
Patterned after the FDA guidelines for approval of
new drugs, the APA Div 12 Task Force on the
Promotion and Dissemination of Psychological
Procedures established two criteria for establishing
the empirical validity of a psychotherapeutic
approach:
 The approach is superior to a placebo or other
treatment or
 The approach is equal to an established
treatment, in at least two studies established by
different investigators.
Related Concepts
– Meta-Analysis
Meta-analysis is a method used to review
research literature based on statistical
integration and analysis of research findings.
In treatment effectiveness meta-analysis, the
dependent variable is the effect size (i.e., the
outcomes or results of each study selected
for review transformed into a common metric
across studies) and the independent
variables are study characteristics (i.e.,
participants, interventions, and outcome
measures).
Related Concepts
– Meta-Analysis
Meta-analysis is an effort to review the results of a
research domain in quantitative terms. The intent is to
identify what significant relationships exist between study
features (independent variables) and effect sizes
(dependent variable).
Benefits: The benefits of meta-analysis include its ability
to:



Synthesize the results from many studies succinctly and
intuitively for nonscientific communities,
Illustrate the amount and relative impact of different
programs on different criteria for policy decision-making
purposes, and
Identify the most effective programs and highlight gaps or
limitations in the literature to suggest directions for future
research
Related Concepts
– Meta-Analysis
A common index of the size of the effect produced by
each study is the effect size index g, which is the
standardized difference between the sample mean of
the treatment group and the sample mean of the
control group (Wampold, 2001). A positive score
indicates that the treatment group outperformed the
control group, and a negative score has the reverse
meaning. However, the effect size index g is a
sample statistics. As such, it is a biased estimator of
the true (i.e., population) effect size.
Related Concepts
– Meta-Analysis
Hedges and Olkin (1985) provided the effect size
index d as a good approximation of the unbiased
estimator and the index d+ for aggregating the effect
sizes across studies as estimate of population effect
size. The unbiased effect size indexes d and d+ are
commonly reported in meta-analytic studies.
Standardized Mean Difference Effect Size
 large effect:
d = .80
 medium effect:
d = .50
 small effect:
d = .20
Evidence-Based Practice:
Implications for Rehab Counseling
1.
2.
Implications for counseling research – The need for
level 1 evidence (empirical supported treatment and
meta-analysis) related to the effectiveness of
rehabilitation interventions (individual
ingredients/components of RC as well as RC as
interventions)
Implications for practitioners – For practitioners,
evidence-based practice is a research utilization issue
(ability to judge the quality of an individual research
study and a collection of studies, the ability to select the
best interventions on an individualized basis, and the
ability to search for research information using the
Internet and other library tools)
Resources for Evidence-Based
Practice
Agency for Healthcare Policy and Research
(AHRQ)



Title IX of Public Health Services Act
Healthcare Research and Quality Act of 1999
Primary agenda is to research health outcomes,
develop effective outcome measures & evaluate
overall quality of care.
 Patient Outcomes Research Teams (PORTs)
 Clinical Practice Guidelines
 Clinical performance measures
 Report Cards
PORTs
Designed to determine the most effective
treatment and pattern of care for a specified
clinical area through lit reviews & metaanalysis.
Partnership with National Institute of Health.

Some PORTs that are relevant to rehab:
 Effectiveness of alternative treatments for type II
diabetes & mental health problems in the MexicanAmerican population (funded in part by UT Health
Science Center)
 Secondary prevention of stroke, hip fx repair,
osteoarthritis, TKR, back pain tx and assessment.
Clinical Practice Guidelines
Frequently developed from PORT findings
Offer healthcare providers well-founded, costeffective treatment methods for various clinical
conditions.

PORT study on Stroke Rehabilitation led to the clinical
practice guideline “Post-stroke Rehabilitation” (AHRQ, 2000).
Development & use of clinical practice guidelines has
become so well accepted that AHRQ no longer
sponsors development since many public & private
entities are doing this independently.
National Guideline Clearinghouse
(www.guideline.gov)
Clinical Performance Measures
Used to assess services of a healthcare professional by
reviewing



appropriateness of service
timeliness of service, and
access to service
Measures are detailed and condition specific
Attempts to measure quality of care.
Currently there are approximately 1,200 performance measures
Are grouped by




Disease/condition
Treatment/intervention
Domain
Organization
Available at the National Quality Measures Clearinghouse
(www.qualitymeasures.ahrq.gov)
Clinical Performance Measures
Measures developed by:






Joint Commission on Accreditation of Healthcare
Organizations
Health Plan Employers Data & Information Set
Healthcare Cost & Utilization Project
Outcome & Assessment Information Set
The University of Wisconsin Nursing Home quality
indicators.
VA external review program
Report Cards
Information obtained through these Clinical
Performance Measures are frequently used for
marketing and other communication purpose to
consumers and other purchasers through “Report
Cards”
Report Cards are a response to the consumer-driven
healthcare system and provides consumers with
information concerning the care recommended by
clinical practice guidelines, outcomes expected under
specific situations and the wide range of performance
measures used to evaluate the quality different
aspects of care offered in a user-friendly format.
Other resources
American Congress of Rehabilitation
Medicine
www.acrm.org/Resources/Evidence-BasedResources.htm
Research Issues
Rehabilitation research has identified
important rehab counseling/case mgt
functions and knowledge domains,
BUT…
As of this time, no empirical evidence
has been generated that supports the
effectiveness of these functions or
knowledge domains on rehab
outcomes.
Research Issues
It is possible to infer some evidence
using research from allied disciplines
(e.g., counseling and clinical
psychology)
For example: the process variable of
“working alliance” has gained
overwhelmingly strong empirical
evidence as a primary influence on
counseling outcomes (Wampold, 2001).
Research Issues
There is some evidence for effectiveness of
counseling factors in rehab counseling


Bolton and Akridge’s (1995) meta-analysis of 15
experimental evaluations of 10 skills training
interventions (e.g., social skills, stress mgt.,
problem-solving skills, & career decision-making
skils) across some 61 outcome measures.
Estimated true effect size of +.93 indicated skills
training services substantially benefit the typical
VR client.
Practice Issues
Rehab Counselors/case managers need
to aware of contemporary research in
rehab as well as in allied fields.
This will promote evidence-based
practice and insure people with
disabilities and chronic illnesses receive
effective services.
Practice Issues
RC Master’s education curricula has only one
class in research methods

Probably the most unpopular course and
unfortunately translates to negative attitudes
towards using research.
Other problems to research utilization:




Practicing counselors generally don’t believe
research results can translate into their work
Lack of time on the job
High cost of continuing education
Weak research analysis skills
Practice Issues
How to correct these problems:

Rethink curricula regarding research
 Need for more creative activities that promote
working knowledge of manual and computer
searches
 Solid understanding of different research
designs and issues related to power analysis,
effect size and meta-analysi.

Regular in-service training within rehab
work environments
Summary
Rehab is one of many health-related
disciplines facing demands of a
managed-care system.
Professionals in the social and
behavioral sciences are having to rely
on credible evidence to justify their
interventions and maintain their identity
in a climate of healthcare outcomes.
Summary
Evidence-based practice and empirically
validated interventions dominate the
healthcare research and are most likely
here to stay.
How well rehabilitation researchers and
practitioners work together to respond
to the demands of this evidence-based
climate is crucial for the profession’s
viability.
Summary
Rehabilitation researchers need to conduct
more experimental studies to validate the
effectiveness of specific rehab
counseling/case management interventions
as well as to determine the overall
effectiveness of rehab counseling and case
management.
Rehabilitation educators need to focus on
improving the quality and effectiveness of
courses in research methods to facilitate
evidence-based practitioners.
Summary
Rehabilitation students need to be able to
understand a wide array of research designs
and methodologies as well as access and
critically evaluate research from rehab and
related literature.
Work settings need to provide pre-service and
in-service training regarding evidence-based
practice and incorporate this type of service
delivery into their job descriptions.
Summary
Evidence-based practice is the
standard of practice in healthcare.
The extent to which rehabilitation
counselors/case managers, researchers and
educators prepare for and accept EBP will
dictate the degree to which rehabilitation can
be performed correctly and adequately by the
professionals providing the service.
(Chronister et al., 2005)