Transcript National Wraparound Initiative The Wraparound Fidelity Index, version 4 www.rtc.pdx.edu/nwi Introduction The Wraparound Fidelity Index was developed to assess the implementation of the wraparound process through brief.

National
Wraparound
Initiative
The Wraparound Fidelity Index, version 4
www.rtc.pdx.edu/nwi
Introduction
The Wraparound Fidelity Index was developed to assess the
implementation of the wraparound process through brief interviews
with multiple respondents (e.g., caregivers, youths, and facilitators).
Previous versions of the WFI, primarily the WFI-3, have been used
widely in research on wraparound and as quality assurance
mechanisms by communities and programs. The WFI-3 has been
found to possess good psychometric characteristics, including testretest reliability, inter-rater agreement, and internal consistency
(Bruns, Suter et al., 2004). Validity has been established through
studies showing agreement with an external expert’s assessment of
wraparound quality, and correlation with child and family outcomes
(Suter et al., 2004).
At the same time, limitations of the WFI-3 have been noted,
primarily, that the WFI-3 only assesses adherence to principles, not
fidelity to a model or set of specific activities. Recent development of
a model that includes a description of specific phases and activities
of the wraparound process (Bruns, Walker et al., 2004) has allowed
for a revision to the WFI-4, which assesses both adherence to
principles as well as fidelity to these activities. In addition, this
revision provides an opportunity to replace a number of WFI-3 items
that have shown poor variability and discriminant validity.
Results of an Initial Pilot Test
Eric J. Bruns, Jesse C. Suter, Jim Rast, Janet S. Walker, & Michelle Zabel
Results: Scale Scores
Results: Engagement Phase Items
No. Item
Were you given time to talk about your family's
1.1
strengths, beliefs, and traditions? Did this help you
CC
appreciate what is special about your family?
1.2 Did your facilitator fully explain the wraparound
FVC process and the choices you could make?
At the beginning of the process, did you have a
1.3
chance to tell your facilitator what things have
Ind
worked in the past for your child and family?
1.4 Did you select the people who would be on your
TB youth and family team?
1.5
TB
1.6
SB
1.7
NS
Is it difficult to get agency representatives and other
team members to attend team meetings when they
are needed?
At the beginning of the wraparound process, did you
have a chance to describe your family's vision or
goals for the future?
Did the wraparound facilitator help you to
understand why it would be helpful to have friends
and family on the youth and family team?
1.8 Did you go through a process of identifying crises &
OB what leads to crises for your child and your family?
Mean
SD
Min- Admin
Max probs.
Conclusions
Four Phases of Wraparound
Engagement
87.1
Planning
1.79 .52 0-2
80.6
Implementation
1.72 .61 0-2
83.5
2
Transition
1.74 .69 0-2
1.41 .73 0-2
73.9
TOTAL
81.4
0
1
10
20
30
40
50
60
70
80
A pilot version of the WFI-4 was developed based on the model
developed by the National Wraparound Initiative (NWI; Walker &
Bruns, in press). The pilot WFI-4 includes 49 items assessing
adherence to the 31 activities and 10 principles of wraparound. Items
were organized by the four phases of the wraparound process. The
pilot WFI-4 was created through several iterations of indicator
development, review by NWI advisors, and revision based on
feedback. The current study presents the first data to be collected
using the pilot version of the WFI-4. Our main research aims were:
1. To examine the properties of the 49 individual WFI-4 items
2. To explore the psychometric properties of potential WFI-4 scales
3. To compile feedback from interviewers about the ease of
administration and any problems with specific items
Method and Sample
In January 2006, the pilot WFI-4 was distributed to eight
communities in six states nationally that expressed interest in
participating in the study. Local evaluators contacted caregivers of
children receiving services via the wraparound process and obtained
verbal consent to participate in fidelity interviews. After a brief
orientation to the WFI-4 and its administration and scoring rules, the
evaluator or other interviewers (e.g., members of local family
organizations) contacted consented caregivers and administered the
caregiver version of the pilot WFI-4.
Thus far, data have been returned on N=60 caregivers from seven
sites in five states: Oklahoma (n=17); Missouri (n=12); Massachusetts (2 sites, n=14); Maryland (2 sites, n=10); and New Jersey
(n=6). Four interviews were excluded because they were incomplete.
Data are presented for the final sample of N=56. Youths in the
sample were 11.8 years old on average (SD = 3.9, range 2-17) and
were 52% male. Seventy-two percent of caregivers/respondents were
biological or adoptive parents, 15% grandparents or other relatives,
and 13% foster parents. Half (n=28) of the youths currently were in
or had previously been in state custody. Youths had been enrolled in
wraparound 6.85 months on average (SD = 7.8, range 2-24 months).
90 100
Percent of possible total score
10 Principles of Wraparound
1.52 .67 0-2
87.7
P e r si st e nc e
1.93 -- 1-2
76.2
C om m uni t y B a se d
1.86 .36 0-2
75.3
Out c om e s B a se d
1.72 .61 0-2
68.9
N a t ur a l S uppor t s
84
Results: Planning Phase Items
Te a m B a se d
82.9
I ndi v i dua l i z e d
83.5
86.7
Fa m i l y Voi c e a nd C hoi c e
No. Item
2.1
TB
Has your team developed a team mission that guides
your work together and states the ultimate goal for
the team? Can you describe what your team mission
says?
Mean
SD
MinMax
1.69 .64 0-2
2.2 Did you create a written plan that describes how the
Col team will meet your child's needs?
1.64 .64 0-2
2.3
SB
Are the supports and services in your wraparound
plan connected to the strengths and abilities of your
child and family?
1.81 .51 0-2
2.4
CB
Does the plan include strategies for helping your child
get involved with activities in her or his community?
1.32 .69 0-2
2.5 Do you feel as though there are members of your
wraparound team who do not have a role in
Col implementing your wraparound plan?
2.6 Were the services and supports in your plan
CC developed with your family specifically in mind?
2.8
NS
Does your wraparound plan include mostly
professional services?
0.69 .87 0-2
2.9
Ind
Is there a crisis or safety plan that specifies what
everyone must do to respond to a crisis? Does this
plan also specify how to keep crises from occurring?
1.43 .80 0-2
10
20 30
Internal
Consistency
1
1.91 .26 1-2
1.65 .58 0-2
OB outcomes you are working toward?
2.11 Do you feel like other people had higher priority than
FVC you deciding what would be in your wraparound plan?
During the planning process, did the team take
2.12 enough time to understand your family's values and
CC beliefs? Is your wraparound plan in tune with these
values and beliefs?
2.13 Do you feel confident that, in the event of a major
crisis, your team has a plan in place that will help
CB
keep your child or youth in the community?
0
2
1.71 .58 0-2
When working on your wraparound plan, did the team
brainstorm several strategies to address your family's
needs before selecting one?
94.3
C ul t ur a l c om pe t e nc e
Scale
2.7
Ind
2.10 Does your wraparound plan have clear goals or
Admin
probs.
1.93 .21 0-2
1.56 .69 0-2
1.86 .48 0-2
1.71 .57 0-2
Note: Because of space constraints, only Engagement and Planning phase items are
listed. Items in yellow are flagged for review due to lack of variability / “ceiling effect.”
Each item is intended to assess adherence to a principle of wraparound as well as a
specific activity. CC=Cultural competence, FVC=Family voice and choice,
Ind=Individualized, TB=Team-based, SB-Strengths-based, NS=Natural supports,
OB=Outcomes based, Col=Collaborative, CB=Community-based, Per=Persistent.
Total WFI-4
Phase 1:
Engagement
Phase 2
Planning
Phase 3:
Implementation
Phase 4:
Transition
40 50 60
70 80
Initial findings from six sites indicate that reliability as assessed by
Cronbach alpha is good for the Total WFI-4. Alpha coefficients were
found to be adequate for three of four scales combining items for the
different Phases of wraparound assessed by the measure. Alphas for
scales consisting of items that measure each principle of wraparound
were poor, likely because these scales consisted of only 4-8 items.
Findings suggest WFI-4 total scores will be useful in research
studies, but not scales constructed for the 10 principles. More data
are needed to assess utility of the Phase-specific scales.
Results also demonstrate that the measure is capable of finding
between-site differences, as demonstrated by significant results of a
one-way ANOVA, despite very small ns for each site.
Most important at this stage of development, results from this initial
pilot indicate several Pilot WFI-4 items that should be considered for
deletion or revision before a larger pilot is initiated. These include 10
items that showed little variability (see examples highlighted in
yellow at left). In addition, there were a number of items for which
revision or deletion should be considered because of difficulty in
administration or for respondents. Deletion of such items will yield a
shorter measure that will also demonstrate greater variability in total
fidelity scores, and lower mean total scores than the 81.4% found in
this pilot. Both of these outcomes will be important to ensure the
WFI-4 is feasible, sensitive, and psychometrically sound.
84.8
C ol l a bor a t i v e
S t r e ngt hs B a se d
The Current Study
The current results represent the beginning of an extensive validation
process for the WFI-4. Encouragingly, participating local evaluators
who had previously used the WFI-3 all reported that the WFI-4
represented an improvement over previous version of the measure.
90 100
Site-level
Differences
Of course, it should be noted that the current sample consists of selfidentified sites who are likely to be achieving high-fidelity
wraparound, and convenience samples of families, who may be more
likely to be experiencing success. Pilot testing of the WFI-4 will
continue with larger and more representative samples and ultimately
include validation studies of a revised WFI-4 that compare scores for
wraparound vs. non-wraparound sites and that determine association
with child and family outcomes.
items alpha
49
.84
Literature cited
8
.51
13
.62
19
.86
9
.79
Bruns, E.J., Suter, J.C., Burchard, J.D., Leverentz-Brady, K. & Force, M. (2004).
Assessing fidelity to a community-based treatment for youth: the Wraparound
Fidelity Index. Journal of Emotional and Behavioral Disorders, 12, 69-79.
Bruns, E.J., Walker, J.S., VanDenBerg, J.D., Rast, J., Osher, T.W., Miles, P., &
Adams, J. (2004). Phases and activities of the wraparound process. Portland,
OR: National Wraparound Initiative, Research and Training Center on Family
Support and Children’s Mental Health, Portland State University.
Suter, J.C., Burchard, J.D., Bruns, E.J., Force, M.D., & Mehrtens, K. (2002). User’s
Manual to the Wraparound Fidelity Index 3.0. Burlington, VT: University of
Vermont Department of Psychology.
Walker, J.S. & Bruns, E.J. (in press). Building on Practice-Based Evidence: Using
Expert Perspectives to Define the Wraparound Process. Psychiatric Services.
F (6,55) = 2.382, p<.05
Feedback on Individual Items
In addition to items that showed poor variation (highlighted in Tables 1 and
2), seven of the 49 items were noted by at least 2 interviewers to be difficult
to administer or score, or problematic for respondents. Items that showed
the most difficulty included:
•Item 2.1: Has your team developed a team mission that guides your work
together and states the ultimate goal for the team? (Difficult to understand)
•Item 3.16: Do you feel as though your child or family could get kicked out
of wraparound? (Distressing for families)
•Item 3.17: Do all the members of your team respect the values, beliefs,
and traditions of you and your family? (Duplicative of previous questions)
•Item 3.18: Does your child have the opportunity to communicate his or her
own ideas when the time comes to make decisions? (Child too young)
•Item 4.6: Has your team helped you and your child prepare for major
transitions (e.g., new school, new residential placement)? (Caregiver
reports no major transitions have occurred)
Acknowledgments
The authors would like to express our gratitude to all communities participating in
the pilot testing and validation process for the WFI-4. Development of the WFI-4
is aided by support to the National Wraparound Initiative provided by the Child,
Adolescent and Family Branch of the Center for Mental Health Services as well
as the State of Maryland Governor’s Office for Children.
For more information
To learn more about the Wraparound Fidelity Index, download a copy of this
poster, or inquire about becoming a pilot test site for the WFI-4, please visit the
Wraparound Evaluation and Research Team at http://depts.washington.edu/
wrapeval, or contact Eric Bruns at [email protected] / 206-685-2085.
To learn more about the National
Wraparound Initiative, check out our
website at www.rtc.pdx.edu/nwi.