Transcript In it for the Long Haul: The Developmental Stages and

In it for the Long Haul:
The Developmental Stages
and Challenges of Systems Change
Doug Easterling, Ph.D.
[email protected]
Liz Arnold, Ph.D.
[email protected]
Reclaiming Futures Leadership Institute
Miami, FL
May 18, 2011 (revised)
Purpose of the Session
1. Present a “developmental model” that describes how systems-change
initiatives play out over the long run
– Intended to serve as a road map that can be helpful to those involved in a
systems-change initiative (e.g., RF)
– What challenges can you expect given your initiative’s stage of development?
2. Use data from the evaluation of the North Carolina RF initiative to make the
model more concrete
3. Group discussion and collective problem-solving on how to navigate the key
challenges associated with each stage
4. Critique the model
– Are these the right stages?
– Are these the key challenges?
– What’s missing?
Format of the Session
• Overview (Doug)
– Systems-Change Initiatives
– The Developmental Model
• Presentation and Discussion around each stage of the
model
– Description of the stage and key challenges (Doug)
– Data and impressions from the NC evaluation (Liz)
– Initial thoughts from someone who has lived the process
(Benjamin)
– Group discussion
• How has this stage played out across different RF sites?
• Which challenges have been most significant and/or flummoxing?
• Promising approaches to addressing key challenges
• Final Discussion
WFU Evaluation of
North Carolina RF Initiative
o Evaluation Question: What happens when the RF model and the
associated supports (e.g., funding, fellowships, technical
assistance, coaching) are introduced into six NC communities?
o
o
o
o
o
o
o
o
Formation and functioning of local RF teams
Changes in procedures used for screening, assessment, referral, etc.
Introduction of new tools for assessment and screening
Improved monitoring and tracking
New services and programs
Improved coordination
New partnerships
Do more youth receive appropriate services?
o Evaluation Methods
– Annual interviews (telephone or in-person) with individuals in fellowship
positions
– Periodic web-based survey completed by an expanded set of players
with knowledge of the local system
Intent of Systems-Change Initiatives
• A systems-change initiative goes beyond firstorder “solutions” to generate change that is both
system-wide and systemic
• Not about introducing or changing a specific
program …
– … but rather a change in the “system” through which
services are delivered and/or problems are solved
• Exert leverage on structural factors that
influence bottom-line outcomes
– Beyond superficial fixes to root issues
What’s a “System”?
•
System = multiple players, processes, and revenue streams that are
organized to accomplish a larger purpose
•
Where do “systems” operate?
– A large institution (e.g., courts, medical center)
– A “cluster” of organizations that are focused on a common issue or population
(e.g., mental health treatment providers)
– Across an entire community
• acknowledges the breadth of actors who have influence over any complex issue
•
What sorts of changes allow systems to become more effective?
– New programs and services that address gaps in the system
– Improved programs and services
– Better tracking and monitoring of outcomes – with feedback loops to improve
services
– Improved coordination among actors (e.g., referrals, common intake, shared or
linked data systems, collective problem-solving)
– Culture change that promotes excellence, learning, and ongoing improvement
– Any other change that allows “clients” (patients, customers, families) to meet
their needs and achieve their goals
Systems-Change Initiatives: Approach
• Inputs and Tools for Systems Change
– Inter-departmental, inter-agency, or community-wide stakeholder group
– Front-end assessment
• Needs assessment
• Environmental scanning
• Evaluation of current programs and services
– Strategic planning and thinking
• Vision of what the system should look like
• Identification of strategic goals and leverage points
– Theoretical Framework with pathways to progress
– Consultants and Technical Assistance
– Opportunities for Learning
• A systems-change initiative can originate from either:
– Local actors who recognize the need for change, or
– An outside agent looking to make change happen in multiple places
• Private foundations
• Government agencies
• National or regional organizing groups (e.g., IAF,People’s Institute)
How is RF a systems-change initiative?
• Change involves new services, programs and approaches
throughout an entire system, not just within specific organizations
• Pushes for changes in how (and how well) organizations work with
one another
• Multi-agency coalition provides leadership, carries out planning,
develops and implements strategy, and monitors progress
• Local strategies are informed by a theoretical framework
• Technical assistance and opportunities for learning (from NPO)
• Accountability comes through tangible improvements in youth
outcomes (rather than simply doing new things)
Systems-Change Initiatives I have known
•
As a funder (Colorado Trust)
–
–
–
–
–
•
Colorado Healthy Communities
Initiative
Teen Pregnancy Prevention Initiative
Violence Prevention Initiative
Colorado School Health Education
Initiative
Assets for Colorado Youth (Search
Institute)
As an evaluator
–
–
–
–
–
•
Reclaiming Futures (Kate B. Reynolds)
Free to Grow (RWJF)
Strategic Approaches to Community
Safety (SACSI) [NIJ]
Community Foundation of Greater
South Wood Co, WI
Duluth-Superior Area Community
Foundation
As a design consultant to
foundations that do systems
change
–
–
–
–
•
As a member of a systems-change
coalition
–
–
•
Mary Reynolds Babcock Foundation
Winston-Salem Foundation
Community Foundation of Greater
Greensboro
National Social Capital Learning Circle
(19 foundations)
Clinical Translational Science Award
(NIH)
Out-of-school education for youth
(Wallace)
As a facilitator
–
ECHO Council (social capital coalition
in Winston-Salem)
What can we say about systems-change work?
•
•
•
•
•
•
•
Messy
Difficult
Ambitious
Complex
Rife with conflict
Political
Can threaten those in
charge (even if they
started out as an
advocate or ally)
• Can lead to burn out
• Non-linear
• Ups and downs
• Progress can emerge out
of apparent failure
• Some successes are
short-lived
• The work is never done
• Different issues arise at
different points
The Key Challenge:
Moving from
Collective Conversation
to
Collective Impact
(Kania & Kramer, SSIR, 2011)
Potential Impact of a Systems-Change Initiative Increases with Time
100
90
80
70
60
50
Degree of Impact
Possible
40
30
20
10
0
0
3
6
9
12
15
18
21
24
27
30
33
36
Months Since Inception
39
42
45
48
51
54
57
60
Attrition in Systems Change Initiatives
100
90
80
70
60
50
% of initiatives
active and effective
40
30
20
10
0
0
3
6
9
12
15
18
21
24
27
30
33
36
Months Since Inception
39
42
45
48
51
54
57
60
Attrition and Impact Over Time
100
90
80
70
60
50
% of initiatives
active and effective
40
Degree of Impact
Possible
30
20
10
0
0
3
6
9
12
15
18
21
24
27
30
33
36
Months Since Inception
39
42
45
48
51
54
57
60
Stages of Development
• Systems change initiatives have a natural
life cycle, defined by predictable stages of
development
• Moving to the next stage allows a greater
level of impact
• … but it also raises new challenges that
threaten the survival of the initiative
Stages in the Life Cycle of an
Externally-funded Systems-change Initiative
Stage
Defining Task
1. Initiation
Obtain funding
2. Team Development
Establish an effective infrastructure to carry out
systems-change work
3. Initial Impact
Accomplish a concrete, meaningful act of
systems change
4. Extending the Impact
Move from a narrow win to accomplishing
broader scale changes in the system
5. Sustaining the process
of innovation and reform
Become a permanent venue and force for
stimulating whatever changes in the system
need to occur as times and conditions change
Timing of the Stages (typical)
Stage
1. Proposal Writing
2. Team Development
3. Initial Impact
4. Extending the Impact
5. Sustaining the process
of innovation and reform
award
1yr
2yrs
3yrs
4yrs
Stages in the Life Cycle
100
90
80
70
Stage 2
Stage 3
Stage 4
Stage 5
60
50
% of initiatives
active and effective
40
Degree of Impact
Possible
30
20
10
0
0
3
6
9
12
15
18
21
24
27
30
33
36
Months Since Inception
39
42
45
48
51
54
57
60
What actually happens in each stage?
• An overall task that sets the stage for the work
• A series of more concrete tasks
• Predictable challenges
• Coping with those challenges
– (hopefully successful coping, at least for the most
part)
Stage 1: Proposal Writing
•
Defining Task: Win one of the grants
•
Key Tasks and Challenges
1. Assemble the required partners
2. Become versed in the initiative’s change model
3. Determine how the model could work locally
4. Write a competitive proposal
Stage 2: Team Development
•
Defining Task: Establish an effective infrastructure to
carry out systems-change work
•
Key Tasks and Challenges
1.
Reassemble the required partners, this time to do the real,
long-term work
2.
Learn from the funder, NPO, TA provider, evaluator, etc. what
is actually expected
3.
Work through confusion, disagreements, competing interests,
personality clashes, etc.
4.
Agree on (or at least decide on) approach, structure, goals,
processes
Themes from the NC RF Evaluation re: Stage 2
•
All 6 sites built functioning RF teams consisting of Project Director, four RF
Fellows, and other key players
•
Teams generally were meeting 1 or 2 times per month
•
Some turnover in teams occurred once the grant was awarded and the work
began in earnest
– Most of the turnover was job-related
• Some of the roles that are key to RF (e.g., treatment, community fellows) tend to be
inherently transient
• Notable move up by Robin Jenkins to leadership role within NC DJJDP
– Conversely, some fellows and partners kept their jobs, but fell off from RF
• Different level and type of contribution for pre-award vs. post-award
•
Project directors were pretty stable for first year
– First of 6 PDs to leave left approx 1 year into RF (took a new job and left the
area)
•
Teams began to come together and gain cohesion
– Easier and smoother in some sites than others
• Overall, 86% of Round 2 interviewees reported that their team worked together “great”
or “pretty good”
• But see chart on next slide
Team Functioning
• “How well does your team work together?”
–
–
–
–
–
–
Site A:
Site B:
Site C:
Site D:
Site E:
Site F:
4-“great” 1-“pretty good”
4-“great” 1-”pretty good”
2-”great” 2-”pretty good”
1-”great” 4-”pretty good”
1-”great” 1-”pretty good” 2-”mixed”
3-”pretty good” 2-”mixed”
• Most sites report improvement over time in
communication and understanding one
another’s perspectives
Stage 3: Initial Impact
•
Defining Task: Accomplish a concrete, meaningful act
of systems change
•
Key Tasks and Challenges:
1.
2.
3.
4.
5.
6.
7.
Carry out a strategic analysis of what needs to change within
the system (in line with the initiative model)
Identify 1-3 specific issues that allow strategic leverage
Develop a plan for making that change (with specific
responsibilities assigned to willing partners)
Carry out the plan as well as possible
Work out who does what and who is really in charge
Deal constructively with transition, resistance and setbacks
Evaluate, learn, and adjust
Themes from the NC RF Evaluation
• In line with RF model, the 6 teams explored what
might be done to improve screening,
assessment, and treatment within their
community
• Focused initially on screening and assessment
• Looked at ways to increase the number of youth
screened and assessed, as well as whether
different tools might be needed
– Strong encouragement from NPO and DMH to adopt
the suite of GAIN tools
Changes in Screening
• Prior to RF:
– the standard approach to screening was for Court Counselors to
administer DJJDP Risk & Need Assessment
• Other tools in limited use: GAIN-SS, MASSI, MAJORS screener
– Screening was NOT carried out on all youth entering the court
system
• 1 ½ years into RF:
– All six sites had implemented more systematic approaches to
screening (including conducting screening in more locations)
– In all sites, interviewees report an increase in # youth
screened
– In most sites, interviewees report that Court Counselors are
explicitly seeking to screen all youth entering court system
– Standard screening tool in all sites: GAIN-SS
– In 5 sites, an interviewee mentioned that RF team had
developed specific criteria for referring youth to assessment
Changes in Assessment
•
By the time of the second interview, there had been some improvements in
the assessment process, but more modest than what occurred with
screening
– In 3 sites, interviewees report improvement in scheduling assessments, or
reduced time between screening and assessment
– Limited reports of more youth being assessed
– Some critical review of tools that are used for assessment
– In 2 sites, interviewees report change in the tools that providers are using for
assessment (GAIN-I, JASAE)
•
Use of GAIN tools for assessment:
– In no site is a GAIN assessment tool being used universally by MH/SA Tx
providers
– In 1 site, a key provider is using GAIN-I
– In 2 other sites, some providers have been trained in GAIN-I or GAIN-Q
– In remaining 3 sites, there appears to be little to no interest in using GAIN
assessment tools
Stage 4: Extending the Impact
•
Defining Task: Move from a narrow win to accomplishing broader
scale changes in the system
•
Key Tasks and Challenges:
1.
Revisit the strategic analysis, this time with benefit of real-world
experience and additional data
Decide what to do next
2.
•
•
•
3.
4.
5.
6.
7.
new approach to an unsolved problem,
address a new aspect of the system, and/or
reach a new population
Identify and recruit additional partners who are crucial to success
Revise and extend the initial strategic plan to be more
comprehensive, focused and realistic
Carry out the plan as well as possible
Deal constructively with transition, resistance and setbacks
Evaluate, learn, and adjust
Themes from the NC RF Evaluation
•
Success with improved screening provides a concrete point of departure for
further changes to the system
•
Different sites focus on different areas for change
•
Assessment
– Changes that ensure more youth are assessed
– Do assessment sooner after screening
– Not a lot of emphasis on changing the tools for assessment
•
Treatment
– In some sites, key services do not exist or slots are limited
– Special problems for:
• youth without Medicaid
• youth who don’t speak English
– Limited progress in adding new services and programs
• Mentoring
•
Continue to try to develop a data system to track RF outcomes
•
Initial work on Community Engagement
Stage 4 Challenges and Struggles in NC
• How to maintain progress (or even traction) with all the funding cuts
for services and changes in mental health system
• Shortage of funding for any services that go “beyond treatment”
• Many struggles with collecting data from different providers and
combining data into a common database.
– Almost everything is done “by hand” in excel spreadsheets
– Looking for ways to tie into NC DJJDP data system (NC-JOIN)
– Experimenting with new templates and overlays
• Teams get frustrated as the work gets more diffuse and complex
– During Round 2 interviews, 86% reported that their team worked
together “great” or “pretty good”
– Down to 64% during Round 3 interviews
Stage 5: Sustaining the process of
innovation and reform
•
Defining Task: Become a permanent venue and force for
stimulating whatever changes in the system need to occur as
times and conditions change
•
Key Tasks and Challenges
1.
2.
3.
4.
5.
6.
Assess and acknowledge accomplishments to date
Recognize that the coalition and its work are “part of the system” (no
longer a “special” effort to change the system)
Revisit mission: Identify what needs to happen for innovation and
reform to continue (rather than protecting the initial round of systems
change)
Evolve the structure to meet the new mission. Where should the
work reside?
Find the resources to support the new work
Enter into a new phase of innovating, planning, initiating change,
assessing, learning, and evolving.
Themes from NC
• The 6 NC sites are just beginning to think about this
stage (i.e., what RF looks like when it is institutionalized)
• The prospect of the KBR grant ending has stimulated
questions about “long-term RF”
– What position might the PD convert into?
– Which funded positions overlap with RF’s intent?
– Where should RF staff reside over the long haul?
• Opening of a state RF office in NC brings up parallel
questions re: institutionalizing RF throughout the state.
Final Discussion
Stage
Defining Task
1. Initiation
Obtain funding
2. Team Development
Establish an effective infrastructure to carry out
systems-change work
3. Initial Impact
Accomplish a concrete, meaningful act of
systems change
4. Extending the Impact
Move from a narrow win to accomplishing
broader scale changes in the system
5. Sustaining the process
of innovation and reform
Become a permanent venue and force for
stimulating whatever changes in the system
need to occur as times and conditions change