Implementing Evidence-Based Practice: An Overview of the BEST Training with Practitioner Responses Presented by: Sarah E.
Download ReportTranscript Implementing Evidence-Based Practice: An Overview of the BEST Training with Practitioner Responses Presented by: Sarah E.
Implementing Evidence-Based Practice: An Overview of the BEST Training with Practitioner Responses
Presented by:
Sarah E. Bledsoe, Ph.D., M.Phil., M.S.W.
School of Social Work University of North Carolina at Chapel Hill
Jennifer L. Bellamy, Ph.D., L.M.S.W.
George Warren Brown School of Social Work Washington University
Implementing Evidence-Based Practice: An Overview of the BEST Training with Practitioner Responses
Sarah E. Bledsoe 1 Jennifer L. Bellamy 2 Lin Fang 3 Jennifer I. Manuel 3 Edward J. Mullen 3 for the BEST Team 1 School of Social Work, University of North Carolina at Chapel Hill 2 George Warrant Brown School of Social Work, Washington University 3 Columbia University School of Social Work Supported in part by the Willma & Albert Musher Program at Columbia University
Introduction
Underutilization of scientific knowledge Evidence-practice gap 2 Challenges: Lack of knowledge and training in EBP & ESI Discrepancies in the definition of EBP & ESI Consequences: Practice outcomes Translational research and knowledge generation
BEST Project
Bringing Evidence for Social Work Training Multi-phase exploratory study Training-based collaborative agency-university partnership 3 social agencies Response to: Limited EBP/ESI implementation research Push for EBP/ESI in practice and education
Evidence-based practice is the integration of best research evidence with clinical expertise & patient values
From: Shlonsky, A., & Gibbs, L. (2004). Will the real evidence-based practice please stand up? Teaching the process of evidence-based practice to the helping professions. Brief Treatment and Crisis Intervention, 4, 137-153.
Empirically Supported Interventions
Interventions for which there is consistent scientific evidence showing that they improve client outcome Interpersonal psychotherapy for depression Therapuetic foster care Assertive community treatment teams
(Drake, Goldman, Leff, et al., 2001).
EBP in the BEST Training
Conceptualized as a professional model of practice: 1.
2.
3.
4.
5.
6.
7.
Motivation Question development Search for relevant research Research appraisal Application Evaluation Dissemination
BEST Training
(Gibbs, 2003)
BEST Training
10 training modules Designed to: Increase knowledge of EBP Improve EBP research skills Foster positive attitudes toward EBP Identify factors that promote the use of EBP in social service organizations
Overview of the BEST Training with Practitioner Responses
Today we report: 10 training modules Agency practitioner perceptions of the training Trainer experiences of implementation
Methods
Convenience sample of 3 agencies (n=16) Process notes and post-training focus groups Tape recorded and transcribed verbatim Interviewer notes of emerging themes NVivo Pre-identified and emergent themes Data analytic strategies: Triangulation of data sources Multiple coders Member checking
Sample
Age Female White/Caucasian Asian 35K-50K Masters Degree Social Work Social Work License Time at current agency 34.9 years (mean) 87.5% 37.5% 37.5% 56.3% 50.0% 81.3% 50.0% 6.3 years (mean)
Intervention
Cooperative agency-university partnership Earliest stages included partner agencies Pretraining meetings and focus groups Guided by agency teams & administrators Practice-relevant topic selected by agency team Hands-on training using problem-based learning strategy Research team provided technical assistance & guided the group process
Modules of the BEST Training
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
EBP Introduction and Overview Question Selection Overview of Research Evidence Search Tools Search Demonstration Troubleshooting the Search Evaluating the Evidence General Findings and Observations Synthesizing Evidence Found Action Plan
Module One: EBP Introduction and Overview
I. Definition of EBP a. Brief history of EBP b. Major motivations to use EBP c. Brief overview of different models d. Highlight Gibbs (2003) model: Steps as a general guide e. What we mean when we say “evidence” -research evidence -practitioner experience -client preference -agency mission and values -ethics f. Disconfirming and confirming evidence g. Feedback from the group/questions/any tweaking necessary?
Practitioners Reactions Module One
“
I think we have a fuller view of EBP
…[the training] definitely deepened our understanding about the process and let us have a chance to try it out.” “It feels much more tailored to where we’re coming from and what our interests and values are and
our practice not like there’s going to be a protocol that is going to be put onto
that we would have to somehow manage to squeeze ourselves into and that wouldn’t be desirable.”
Module Two: Question Selection
I. Question selection a. List project focus area for the question b. Discuss elements of “researchable” questions -Relevance to practice -Not something you already have the answer to -Specific c. Practice making up a few questions d. Discussion, refinement & selection of question II. Background Information for the Question a. Why this question in particular?
b. Population -Age, race/ethnicity -Resources -Key mental health/substance abuse issues c. Agency -Referral or recruitment process -Resources available -Current available services
Practitioners Reactions Module Two
“When you start to research [the practice problem], it can become so overwhelming and
this is a way to narrow it
so that you get what you want, so that it’s more practical. Its
consuming and taxing, but on the other hand, its a time saver a little bit more time
, more effective.” “Learning under the discipline of coming up with a question,
we struggled with that for weeks
…If we ask about it one way, we’re asking about effectiveness. We ask it another way, we could be asking a comparison question. So all the ways there are to ask a question and the importance of how you ask [is related] to what you get.”
Module Three: Overview of Research Evidence
I. Defining different sources of research evidence a. Review of different types of “evidence” from module two b. Review of different types of research evidence: Outcome studies, Meta Analyses, Program Evaluations, etc.
Practitioners Reactions Module Three
“There are certain more theoretical readings that aren’t particularly related to empirical demonstrations of different techniques…and I know they have a place in this…I didn’t emerge with a lot of clarity about where they fit in to the process, but sometimes I think
time to thinking about [theory]
.”
it would be useful for us to dedicate some
Module Four: Search Tools
I. Search Engines that are Electronically Available a. Types of search engines & overlap between them b. Starting from meta-analyses & systematic reviews to more specifics II. Using Search Terms a. Example of search terms b. Group brainstorm about search terms for question c. Group search “plan” III. Using MOLES (Methodology Orienting Locators for Evidence Search)
Practitioners Reactions Module Four
In reference to handouts listing available search engines identified as free or fee based: “That is awesome, that’s great. If we got nothing at all but that, this would have been a bonus.” “We found that pay and
everything
.”
the tools
the search…they were great. My problem is, really,
not having the time
you gave us, to streamline . It’s a really big problem. And also, some of the engines you gave us, we’re not going to be able to access because we would have to
it would be nice to have access to
“I need more time to use the tools because I don’t know how to.
I need to actually be practicing. I don’t have a real grasp
.”
Module Five: Search Demonstration
I. Electronic Demonstration of Search Using a Database a. Getting to a database b. Starting a search c. Keeping track of searched terms d. Using abstracts II. Introduction to Other Potentially Helpful Websites III. Review of Search Plan a. How does the agency see their ability to search?
b. Will there be a single person who is a “searcher”? Several people? Will everyone search? Is there no good candidate?
c. Search worksheet
Practitioners Reactions Module Five
“The
organization of [the search]
organize the [search]…I think [providing tools] like these would make people more willing to do this.” helped…using a table to “It was really good to have a concrete example so we could do an
actual practice run through the process
that was extremely helpful.” . Just to do it, try out our own key words in session on our computers. I thought “I think the major issue probably is literature…that’s the only real difficult part. So, other than that, the process is fine.”
access
to the actual “I don’t think people are going to take the
time
to search for a list of sites and to search for different key words.”
Module Six: Trouble Shooting the Search
I. Review of Search Experience a. Amount of material found b. Where material was found c. What was easy d. What was difficult e. Address general questions and issues
Practitioners Reactions Module Six
“To base practice on the evidence that we were [able to access] for this question, I mean
we were struggling
. We were struggling.” “When I come to articles on certain populations…I’m not confident enough to use [them on my own] because there aren’t people available for
consultation
.”
Module Seven: Evaluating the Evidence
I. Examining Evidence Found a.
Review of different types of evidence b. Example of quality review c. Examples of quality rating scales d. Identify the types of research evidence found by team members e. Value of meta-analyses & systematic reviews II. Evidence Quality a. Review of the different philosophies/ratings of evidence b. Match of evidence/research type with question- the same type of research is not appropriate for all questions c. General indicators of quality d. Ratings worksheet e. In vivo practice using worksheet via discussion of evidence found III. Refinement of Search Plan Based on Findings To-Date a. Any search engines not covered?
b. Any types of evidence missing?
c. Continue collecting and assigning evidence
Practitioners Reactions Module Seven
“I think the tools were wonderful, but extremely rigorous and I think
we would probably do it a little more impressionistically
…I think if practitioners could read [evidence] together, almost like a book review, talk about it, that would feel almost in sync with how we approach issues and challenges.” “
I think it’s really important if you actually were able to wrap your head around the math
calculating [a statistic].” …I still don’t know which study’s data is really that clean…or if the methods are really that good…having a [statistic] helps you decide that but I can’t see myself going through 15 articles and “I can find research articles…but
I don’t have confidence interpreting the findings
.” “I start to zone out.”
Practitioners Reactions Module Seven
“One question is,
how much research is out there that is relevant?
” “Sometimes I have difficulty [evaluating evidence] because a lot of research is done with…other populations and…
I have to think, can I really generalize these techniques and would it be applicable to [the] specific population I’m working with?
A lot of times I find [the empirical evidence] seems really good but I wonder if it’s helpful or good [for my clients]. This evaluation would be challenging to me.” “In terms of fit, so
it almost would require another piece of training
or knowledge around how you judge evidence that’s been produced out of a completely different population. In this case, we had difficulty finding research dedicated to the population that looks like [our clients].”
Module Eight: General Findings and Observations
I. Review of General Findings a. Overall, does the research answer our question of interest?
b. Is there a great deal of research related to our question of interest?
c. Which types of research are missing?
II. Review of Research Found a. Start with research syntheses b. Move to individual research articles and other pieces III. Research synthesis and summary handouts a. General overview of the research question or hypothesis b. How recent is the evidence?
c. Main findings d. What is the general quality of the evidence?
e. How well does it fit?
Practitioners Reactions Module Eight
“I really think the
good research can incorporate the practice perspective
.” “I just want to say, I did get a
sinking feeling
when we were…actually evaluating the articles we all found…like,
[best available evidence]?
” is this the
Module Nine: Synthesizing the Evidence Found
I. Research Synthesis a. Title (Year) b. Source Type c. Major Findings d. Quality and Fit Notes II. Summary Statement a. Body of evidence currently available b. Summary of findings c. Recommended plan of action (if any) III. Refinement of Group Search Plan Based on Findings To-Date a.
Any search engines not covered?
b. Any types of evidence missing?
c. Continue collecting and assigning evidence
Practitioners Reactions Module Nine
“What we’ve learned about [EBP]..it’s helping us understand how to find the areas of research that are missing and figure out
places where other contributions need to be made
.” “There’s a
gap between what has been done on the research level and what’s being done everyday on the frontline
. It would be great to break that and its hard to do. I know we came up with some ideas about trying to implement some of this research knowledge in more day to day functions here. I think it would be interesting and helpful to do that.”
Module Ten: Action Plan
I. Possible Next Steps a. Summary Statement can form the basis for: -Letter to administration -Training to share findings -Seek training for an identified intervention -Use or development of new instruments or tools -Resource manual -Procedural changes -Request for additional resources or funds -Research at the agency b. Team Plan -What will we do with our findings?
-Who will carry out which steps?
-When will these steps be carried out?
c. EBP Next Steps -How will you plan to continue using EBP?
-Who will carry out these steps?
-When will these steps be carried out?
Practitioners Reactions Module Ten
“We need to
incorporate it into our daily practice
. Our regular team meetings will focus on [EBP]…and incorporate [EBP] into agency structure and promote that type of agency structure.” “
Training is important
so people can read the research and…apply it to the situation. I believe we have the responsibility, the agency has the
responsibility
. I think it is a really
challenging
process.” “As problems come up, as puzzles arise. I mean,
time
.”
I have a plan to use this. I’m just waiting for the
Practitioners Reactions Module Ten
“You’re seeing how
hard it is to take the little attention that we have
, compared to everything else we have to do here,
to think this way
.” “
We don’t generally have someone like [the research team]
to call up and say, ‘We really want to get an article on x or y, could you find 6 things or help us find 6 things?’, ‘What do you think about the key [search terms]?’, or ‘Where would you start?” “The
only drawback is time
…But it could never be a drawback to using it. It’s valuable.”
“We’re willing to do what we can.”
Limitations
Exploratory study Sample size Purposive sample New York City agencies
Recommendations
Participants highlighted need for: Additional training/supervision in EBP & ESI Continued consultation & technical support More evidence addressing real world practice problems & populations Increased technical & statistical skills to evaluate the evidence Addressing real barriers Limited time & competing demands Limited funding to support EBP Limited access to sources of empirical evidence Agency/organizational culture
Recommendations for Social Work & Social Work Education
Training Programs (classroom and field) Process of EBP Specific ESI Research Skills Statistical Skills Research Increase partnerships with practitioners Increase practice relevant research Continuing Education Library access for Alumni & Field Agencies Quality continuing education (EBP & ESI)