Transcript So You Think There`s No Funding?

$o You Think There’s No Money?
Securing Nonprofit Funding in Today’s Economy
presented by:
Debra Thompson, MBA, President & CEO, Strategy Solutions
Joseph Jones, Ph.D., Senior Researcher, American Institutes of Research (AIR)
Today’s Agenda
•
•
•
•
•
•
•
•
•
•
About the Presenters
Today’s Learning Objectives
The Challenge
The Vision
Where are We Now?
Organizational Capacity
Product Development Process
The Role of Innovation
What is Program Evaluation?
What Should We Do Now?
About Strategy Solutions:
Creating Healthy Communities
Wealth Creating Companies
Private Sector
Wealth Creating Companies
Public Sector
Government
Workforce Development
(including Education)
Economic Development
Community Based Nonprofits
Faith Based Nonprofits
Healthcare
About Strategy Solutions:
Creating Healthy Organizations
Product Development
Manufacturing or Service
Delivery
Product Development
Manufacturing or Service
Delivery
Financial Management
Access to Capital
Fundraising
Marketing
IT / HR
Back Office Operations
About AIR:
Mission and Vision
Mission
To conduct and apply the best behavioral and social science research and evaluation
towards improving peoples’ lives, with a special emphasis on the disadvantaged.
Vision
Within the United States and internationally, AIR will be the preeminent organization that:
• produces improvements in education, health, and the workforce;
• addresses the needs of individuals, organizations, and communities;
• designs and advances statistical and research methods;
• causes practitioners and organizations to adopt evidence-based practices; and
• informs public understanding and policymaking by the best evidence.
About AIR:
Who is AIR?
• Based in Washington DC with over 1,500 research staff around
the world
• AIR is organized into seven program areas:
•
•
•
•
•
•
•
Analysis of Longitudinal Data in Education Research
Assessment
Education
Health
Human and Social Development
International Development, Education, and Research
Workforce
• Since 1946, AIR has conducted thousands of program
evaluations for states, local school districts, the federal
government, private sector companies and non-profits.
What is the Goal?
Today’s Learning Objectives
 Learn how to approach program planning,
outcomes measurement and evaluation to
ensure a sustainable business model
 Understand the definition of “evidence
based” and how it applies to your programs/
organization
The Challenge
• Funding is not going away, but…
– more and more, funding is only available to
organizations that can “prove” that they achieve
desired outcomes
• Most nonprofit organizations lack the
sophistication to measure outcomes
How sophisticated do you feel your outcomes
measurement process is?
20%
20%
20%
20%
20%
1
2
3
4
5
1. What outcomes
measurement process?
2. Not very sophisticated
3. Somewhat sophisticated
4. Moderately sophisticated
5. Very sophisticated
0/0
Sophistication
The Vision: Healthy Organizations
The price of
sustainability is
generating
enough cash to
innovate at the
rate of change of
the marketplace
(or ahead of it)
Operations
Product Development
Manufacturing or
Service Delivery
Finance
Support Services
Financial Management
Marketing
Access to Capital
IT / HR
Fundraising
Back Office Operations
Where Are We Now?
Real-time industry and product life
cycle analysis required
Non-Profit
Organizations
Are
are
Required: Organizational Capacity Building to “get to maturity” NOW…
The Non-Profit Organizational Lifecycle Model
If your organization
is ANYTHING but
mature (4, 5 OR 7),
is your business
model sustainable?
7
5
Sustain
4
Mature
2
6
1
Gr
ow
8
nt
e
c
es
g
Be
ne
cli
De
Stagnant
ol
d
A
in
Renew
Di
ss
ol
ve
3
Start-Up
Many are here
Adapted from: Navigating the Organizational Lifecycle
©Strategy Solutions, 2011
by Paul Connolly
What stage of the organizational lifecycle are we
currently in?
1.
2.
3.
4.
5.
Start up
Transitioning to adolescence
Declining before adolescence
Adolescent, striving for maturity
Sustaining, starting to build new
curve
6. Declining before maturity
7. Actively building the new curve
8. Declining
13% 13% 13%
1
2
3
13% 13% 13% 13%
13%
4
8
5
6
7
Life Cycle Stage
0/0
4 Components of Capacity
The ability to
monitor, assess,
respond to and
stimulate internal
and external
changes
Ensure the effective
and efficient use of
organizational
resources.
Adaptive
Management
Leadership
The ability of leadership
and staff and to inspire,
prioritize, make
decisions, provide
directions and innovate.
Technical
The ability to implement all
of the key organizational
functions and deliver
products and services.
Source: Navigating the Organizational Lifecycle: A Capacity-Building Guide for
Nonprofit Leaders by Paul Connolly
Essential Elements of Adaptive Capacity
•
•
•
•
•
•
Needs assessments
Organizational assessment
Program evaluation
Knowledge management
Strategic planning
Collaborations and partnerships
Technical Capacities
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
Product Development Planning
Outreach and Advocacy
Program Operations
Outcomes Measurement, Management AND Program Evaluation
Earned Income Generation
Information Technology
Finance (including access to capital)
Accounting (including process cost accounting)
Fundraising
Legal
Facilities
Marketing and Communications
Human Resources
Other(s)…..
Limitation of Technical Capacities
• The most significant challenge today is that training
and capacity building alone (and gap filling
consulting projects) are not going to be sufficient
for the nonprofits to address these technical needs.
• All nonprofits need to build capacity in EACH of
these areas on an ongoing basis to be sustainable.
Board Program Committee Job Description
1.
Advise the board on industry trends and strategic challenges relating to
the mission of the organization, and assess community need for
programs and services.
2.
Ensure that programs are developed and implemented that meet the
needs of the community and achieve desired outcomes
3.
Identify key indicators for measuring the quality and success of
programs
4.
Track program indicators; and monitor how well agency achieves
outcomes and impacts
5.
Plan and implement strategies to increase awareness and participation
in programs
How strategic is our board Program Committee
Work (compared to this job description)?
1. What board program
committee?
2. Not very strategic
3. Somewhat strategic
4. Strategic
5. Very strategic
0/0
20%
20%
20%
20%
20%
1
2
3
4
5
Strategic
Program Development Challenges Today
• Staffing
– Can organizations afford to have someone
designated as program developers today (even as
part of their job(s)?
• Organization/Process
– Is the program development function and process
organized in a way that actually produces a new
(and funded!) program in a reasonable period of
time?
How effective is your program development
function?
1. What program development
function?
2. Not very effective
3. Somewhat effective
4. Very effective, we plan and
implement new programs and
get them funded very quickly
0/0
20%
20%
20%
20%
20%
1
2
3
4
5
Effective
Have you read this book?
Innovation’s Missing Link
Source: The Other Side of innovation by:
Vijay Govindarajan and Chris Trimble
The need
to reassess
organizing
and
planning is
often
overlooked
Ongoing Operations
Innovation
Strategy
Committing to an Innovative
Idea
Organizing
&
Planning
Execution
Making Innovation
Happen
You can’t ask the group
that is in charge of
today to also be in
charge of tomorrow,
because the urgent
always squeezes out
the important.
Organizing an Innovation Initiative
Project team = Dedicated Team + Shared Staff
The Dedicated Team is custom-built for the initiative.
The shared staff retains its existing responsibilities and supports the initiative.
Performance Engine
Shared
Staff
Partnership
Dedicated
Team
Source: The Other Side of innovation by:
Vijay Govindarajan and Chris Trimble
Steps for Building the Project Team
• Divide the labor: decide how responsibilities for
executing the initiative will be split between the
Dedicated /Team and the Shared Staff.
• Assemble the dedicated team: Determine who will
serve on the Dedicated Team and how to define their
roles and responsibilities.
• Manage the partnership: Establish clear
expectations for each partner and mediate the
inevitable conflicts that will arise between the
dedicated team and shared staff.
Source: The Other Side of innovation by:
Vijay Govindarajan and Chris Trimble
Executing the Initiative
The Partnership, not just the Dedicated Team, executes the initiative!
Shared
Staff
(routine tasks)
Dedicated
Team
(non routine
tasks)
Partnership
One project plan
Source: The Other Side of innovation by:
Vijay Govindarajan and Chris Trimble
The Challenges of the Partnership
Performance
Engine leaders
Challenge 2:
The divided
attentions of
the Shared
Staff
Challenge 1:
Competition with
Performance
Engine leaders for
scarce resources
Shared
Staff
Innovation
leader
Partnership
Challenge 3:
Disharmony
in the
partnership
Source: The Other Side of innovation by:
Vijay Govindarajan and Chris Trimble
Dedicated
Team
Performance Engine Limitations
• If the work relationships inside the
Performance Engine are inconsistent with
what is needed for a certain portion of the
innovation initiative, then that portion must
be assigned to the Dedicated Team
• You can’t ask the group that is in charge of
today to also be in charge of tomorrow,
because the urgent always squeezes out the
important.
Source: The Other Side of innovation by:
Vijay Govindarajan and Chris Trimble
Product Development Process:
Stage/Gate Model
Initial
Screen
Gate 1
Ideation
Identify
dedicated team
and shared staff
who will work on
idea; create
clear operational
definitions of
idea/initiative
Second
Screen
Stage
1
Gate 2
Post
Development
Review
Decision on
Business
Case
Stage
2
Gate3
Preliminary
Investigation
Detailed
Investigation
Identify interest
(through interviews
or survey); review
potential revenue
and costs and/or
cost savings;
vett concerns
Detailed market
and cost benefit
analysis
Stage
3
Gate 4
Development
Detailed operations
analysis and
business planning
Decision to
Launch
Stage
4
Gate 5
Testing and
Validation
Pilot
implementation,
evaluation and
refinement
Stage
5
Full
Operations
and Launch
Product Development Process:
Ideation and Preliminary Investigation
1. Ideation
•
•
Identification of outcomes and impact and
The evaluation process
2. Preliminary Investigation
•
•
Literature review of evidence based programs and best
practices
Determine type of program the agency wishes to
implement
Source: Product Development for the Service Sector
Product Development Process:
Detailed Investigation
3. Detailed Investigation – including Building the
Business Case
•
•
•
•
•
Identify the human, capital and operating resources to
“deliver” the impact (and measure it)
Direct program resources and costs
Indirect organizational sustainability resources and costs
Outcomes Measurement
Identify capacity building needs and fill gaps
Source: Product Development for the Service Sector
Product Development Process:
Development, Testing and Launch
4. Development
•
•
•
Calculate the “return on investment” based on the
outcomes, impacts and the “costs” to achieve the desired
outcomes
Identify funding sources and make the case to fund a
demonstration project or pilot
Develop a project plan for implementation
5. Testing and Validation (including evaluation)
6. Full Operations and Market Launch
Source: Product Development for the Service Sector
Types of Programs
• Programming that is based in tradition,
convention, personal or group beliefs, or
anecdotal information
• Good Ideas or Good Practice
• Promising Practices or Best Practices
• Evidence-Based
Getting
harder
to fund
Funders
want
these
How would you rate your programs today?
1. None of our programs are
evidence based or best
practices
2. Some of our programs are
best practices/evidence based
3. Most of our programs are
best practices/evidence based
4. All of our programs are best
practices/evidence based
0/0
25%
25%
25%
25%
1
2
3
4
Programs
What is program evaluation?
The systematic method for collecting, analyzing, and
using information to answer questions about projects,
policies and programs, particularly about their
effectiveness and efficiency.
Purposes of Program Evaluation
• Ensure transparency and accountability
• Measurement of outcomes and impact
• Form the basis of future programming
• Inform those who provide and administer
programming
• Provide information concerning what works, what
doesn’t work and why
Types of Evaluation
• Formative Evaluation – aims to improve the
program structure or implementation
• Summative Evaluation – assesses the merit or
worth of the program
• Developmental Evaluation – intended for situations
where it is acknowledged that the program and
organization are constantly changing
Program Evaluation Approach
Practicality,
need-to-know
applied science
Proper
methods,
data collection
& analyses
Program Evaluation Measures
Process/Output:
Outcome:
Impact:
• Are we reaching who
we should reach?
• Number of people
reached
• Program Strengths and
Weaknesses
• Feedback/ satisfaction
• What knowledge,
attitudes, behaviors or
other predetermined
variables changed AS
A RESULT of the
program/ efforts?
•
What are the longterm changes that are
a result of the
changes in outcome
variables?
• New impact
question: how does
the program “save
the system
money?”
Program Evaluation Measures
• Measures are used to make inferences about the
program
– E.g., “Results from the survey indicate that our customers
were happy with the program”
• Garbage In = Garbage Out
– Poorly planned and designed measurement leads to invalid
inferences
– E.g., “We based our survey results on 2 questions about
how often each year 3 of our 100 most loyal customers
attended our weekly program on a yearly basis”
What is a Logic Model?
• A Logic Model is a graphic representation of:
•
•
•
•
A program
series of IF, THEN statements
a series of relationships
the program roadmap
A Logic Model is a tool that is helpful to identify outcomes measures
What percentage of your current programs have logic models
(where the identified outcomes are actually measured?
1. None of them
2. Some of them (less than half)
3. Majority of them (50 to less
than 100%)
4. All of them (100%)
25%
25%
25%
25%
1
2
3
4
Measurement
0/0
What percentage of your current programs have any type of
external evaluation of the outcomes/impact?
1. None of them
2. Some of them (less than half)
3. Majority of them (50 to less
than 100%)
4. All of them (100%)
25%
25%
25%
25%
1
2
3
4
Outcomes
0/0
Logic Model:
Inputs, Outputs and Outcomes
• Inputs - The resources that go into the program
• Outputs - The activities the program undertakes
• Outcomes - The changes or benefits that are a result of the
program
H
U
N
G
E
R
Get Food
Eat Food
Feel
Better
Building a Logic Model
Inputs
Outputs
Activities
Outcomes
Participation
Short
Program
Investments
What we
invest
What we do
Who we
reach
Medium
What Results
Long
Building a Logic Model
Inputs
What we
invest
Staff
Volunteers
Money
Research Base
Equipment
Technology
Outputs
What we do
Who we
reach
Train
Participants
Teach
Clients
Deliver
Customers
Services
Agencies
Network
Decision
Partnerships
Makers
Assess
Politicians
Facilitate
Media
Outcomes
Short Term
Learning
Knowledge
Awareness
Attitudes
Skills
Opinions
Aspirations
Intentions
Medium
Term
Long Term
Action
Behavior
Decision
Making
Policy
Social Action
Conditions
Health
Well-being
Economics
Civics
Environment
Example Process, Outcome and Impact Measures
Process
Outcome
Impact
Resources (Staff, Location,
Program, Supplies, etc.)
Knowledge (E.g., understanding
the relationship between diet,
physical activity and diabetes)
Decrease in costs
associated with diabetes
care
Delivery (Protocol, Written
Training Process, Fidelity
Plan, etc.)
Attitudes (Empowerment, Desire
to change, etc.)
Decrease in hospitalization
associated with diabetes
Target Audience (Workplace, Behaviors (Dieting, Physical
Community, Enrollment
Activity, etc.)
Protocol, Drop Protocol, etc.)
Decrease in numbers of
individuals diagnosed with
diabetes
Program Strengths &
Weaknesses (What is
working, What is not
working, etc.)
Program Strategy (Revised
program strategy, reallocated
resources, etc.)
Improved program
efficiencies and reduced
costs
Feedback (Interviews, Focus
Groups, Surveys, etc.)
Communications and Outreach
(Revised communications,
updated outreach and
fundraising, etc.)
Increased program
participation and sustained
program funding
What is Evidenced-Based?
• Research shows that the program produces the expected
positive results;
• Results can be attributed to the program itself, rather than to
other extraneous factors or events;
• Evaluation is peer-reviewed by experts in the field;
• Program is “endorsed” by a federal agency or respected
research organization; and
• Program approach is documented so that it can be
implemented locally in a way aligned with the program design.
What is Evidenced-Based?
Washington State Institute for Public Policy
• Washington State Legislature created the Institute to
conduct research to examine the effectiveness of
state programs
• In 2009 tasked to conduct a study to examine “a
comprehensive list of programs and policies that
improve . . . outcomes for children and adults in
Washington and result in more cost-efficient use of
public resources.”
What is Evidenced-Based?
Washington State Institute for Public Policy
1. Gather studies that have already been conducted
on programs’ effectiveness
•
•
Consider all available studies they can locate on a topic
rather than selecting only a few studies
Conduct “meta-analysis” to determine what the weight
of the evidence tells them about program effectiveness
–
–
–
Statistical method to test an hypothesis based on aggregated
study results
Uses specific assumptions about the similarity between a “large
number” of studies
If assumptions are met, allows analysis without having to
conduct a “new” study
What is Evidenced-Based?
Washington State Institute for Public Policy
2. To be included in their review, evaluation’s research
design includes control or comparison groups
•
•
Random assignment studies preferred, but allow quasiexperimental or non-experimental studies when the
comparison group is well-matched to the treatment group
or adequate statistical procedures are employed to guard
against selection bias
Given the expected direction of selection biases, discount
the findings of less-than-randomized comparison-group
trials by a uniform percentage
What is Evidenced-Based?
Washington State Institute for Public Policy
3. Prefer evaluation studies that use “real world”
samples from actual programs in the field
•
•
•
Evaluations of so-called “model” or “efficacy” programs
are included in reviews, but discount the effects from
these types of studies
Presumption is that it is difficult to achieve, in actual
large-scale operation, the results of model programs
When conducting cost-benefit analyses, discounted the
statistical results of such studies by a fixed amount
What is Evidenced-Based?
Washington State Institute for Public Policy
4. If the researcher of an evaluation is also the
developer of the program, discount the results from
the study.
•
•
Sometimes it is difficult to duplicate the results achieved
by highly motivated individuals, who originate programs
There may also be potential conflicts of interest if
developers evaluate their own programs.
AIR Case Study:
The After-School Corporation (TASC)
•
•
•
•
Issue: The After-School Corporation (TASC) collaborated with community-based
organizations (Good Shepherd Services; Committee for Hispanic Children and
Families, Inc.; and Directions for Our Youth) and college partners (Fordham
University; Bronx Community College; Eugenio Maria de Hostos Community
College; and Mercy College) to implement "Bronx College Town“, a program
designed to enhance the academic, social, emotional, and physical development of
underserved elementary- and middle-school-aged students by focusing on high
school readiness and college and career planning. Program coordinators and
stakeholders sought information to improve on program outcomes and provide
the basis for future funding.
Type of Program Evaluation: Formative and Summative
Approach: Collection of evidence through student surveys, staff surveys, teachers
surveys, and observations of after school activities. In addition the evaluation team
explored the relationship between attending the 21st CCLC sites and student
achievement and grades..
Implications: Findings were published in a report that was used as guidance to
program modifications and as a basis for funding decisions.
AIR Case Study:
Open Society Institute (OSI)
• Issue: Reading and Writing for Critical Thinking (RWCT) is a
professional development project for teachers in countries
formerly under Soviet control. US educators provided the PD,
and the Open Society Institute (OSI) supported an evaluation
of the RWCT’s effectiveness.
• Type of Program Evaluation: Formative
• Approach: Collection of evidence through acquisition and
review of program background materials, interviews with
country coordinators, and interviews with students,
educators, and administrators familiar with RWCT.
• Implications: Findings were used to make in-country changes
to the processes used in implementing RWCT.
AIR Case Study:
Travel Security Administration (TSA)
• Issue: The United States Government created TSA to strengthen the
security of the its transportation systems. TSA secures the United States
airports and screens all commercial airline passengers and
baggage. Reconsideration of funding allocation due to concerns about the
effectiveness of TSA screening methods.
• Type of Program Evaluation: Summative
• Approach: Developed organizing, conceptual framework (logic model)
collaboratively with stakeholders to include program assumptions, inputs
(resources/activities), outputs, and outcomes. Metrics will be tracked to
gather evidence of program effectiveness, through performance
measures, observations, and interviews.
• Implications: Still in progress, but initial model findings are being used to
make policy and funding decisions about TSA programs.
Outcomes Measurement/ Evaluation Needs of Most
Nonprofits
• Support to continue to develop and refine the logic model(s) and the
output, outcomes and impact measures
– Management service and/or consultant support to assist
• Support to set up the “systems” to appropriately and effectively track and
monitor output, outcomes and impacts
– Requires investment in data collection systems and the incremental human
resources to run them
– Operationally define the program variables
– Finalize the “tools” (surveys, tracking and tabulation methods)
• Continually educate and ensure that the other members of the board
understand the resources and capacity required to implement this well
– Especially the Executive and Finance Committees
Tips for Effective Program Evaluation
• Understand the program
– Identify the decision makers and their decisions
– Understand the processes and resources
• Continuous Improvement Process
– Planning
– Ongoing needs assessment
• Understand the situation around the program
– Environmental scan
– Politics happen…
• Rigorous, not rigid, evaluation methods
– Data Collection/Analysis
– Interpretation
• Reporting that informs and improves the program
59
Our Recommendations
1. Understand that you are not alone if you don’t understand
the technical research terms (most people don’t)
– Arrange for a board/ leadership training to learn more about it, the
implications for your agency and how to tie your outcomes to your
fundraising strategy
2. Recognize that an ongoing investment in outcomes
measurement and evaluation is now required as a cost of
doing business. This is not an option!
– If your organization “cannot afford it,” you ought to be looking at ways
to restructure your organization to lower your agency’s fixed costs or
to work together with other agencies to afford the expertise required
Our Recommendations
3. Identify the evidence-based programs and approaches that
already exist in your industry/discipline related to your client
needs and existing programs
– Compare your existing programs to the literature.
– If your organization is doing something really unique or has the ability
to create an “ideal model” of innovative programming, invest in
developing a cost/benefit analysis for that program
4. Develop logic model(s) that outline the short term, medium
term and long term outcomes as well as the cost/benefit
“impacts” of your programs
Our Recommendations
5. Identify the indicators that should be measured and
develop an outcomes and impact measurement process
– Use an appropriate research design and technical assistance from a
qualified researcher with experience in your discipline
6. Determine if your organization would benefit from a formal
external evaluation of your programs and learn how to
select an external evaluator
Questions/Discussion