Transcript Performance Measurement for Justice Integration Projects The Illinois and Cook County Experience

Performance Measurement for
Justice Integration Projects
The Illinois and Cook
County Experience
Mark Myrent
Illinois Criminal Justice Information Authority
What is Performance Measurement?
Performance
measurement can be
defined as the process
of routinely measuring
the outputs and
outcomes produced by
a project, thereby
allowing one to assess
the effectiveness of
project investments and
activities
June 20, 2005
IL Criminal Justice Information Authority
2
Why Measure Performance?
 Build
consensus & commitment within
the justice community
 Obtain & allocate resources
 Plan and manage project execution
 Demonstrate success & improve
accountability
June 20, 2005
IL Criminal Justice Information Authority
3
A Method for Measuring Success
 Statement
of business problem
 Goals
 Objectives
 Project
objectives
 Functional objectives
 Business objectives
June 20, 2005
IL Criminal Justice Information Authority
4
Developing Business Objectives
Identify a BASIC MEASURE
 Indicate DIRECTION of the measure
 Identify the OBJECT of the measure
 Identify the expected VALUE of the
measure
 Identify WHERE the measurement will
occur
 Identify WHEN the measure will be
obtained

June 20, 2005
IL Criminal Justice Information Authority
5
The LOGIC MODEL APPROACH
INPUTS
ACTIVITIES
OUTPUTS
OUTCOMES
INPUTS are your resources, while OUTPUTS are the product of
an activity
OUTCOMES are not what the program or initiative itself does,
but the consequences of the program/initiative efforts -- changes
in conditions, attitudes or behavior of individuals or outcomes for
agencies and communities.
The STRATEGIC PLAN and THEORY OF
CHANGE links these together
June 20, 2005
IL Criminal Justice Information Authority
6
Thinking Through a Logic Model
and Articulating a Theory of Change
GOALS
In order to
solve
these
problems
Activities
We will do
the following
activities and
events
Targets
For these
people and
for this
amount of
time
THEORY
Of
CHANGE
Initial/Short &
Intermediate
Term
Outcomes
This activity
will lead to
changes in
these factors
__________,
which in turn
will lead to
solving these
problems
We will know
these changes
have occurred
If:
___________
LONG
TERM
OUTCOMES
We will know
we are
reaching
our goals
If:
___________
June 20, 2005
IL Criminal Justice Information Authority
7
Developing Business Objectives






Identify a BASIC MEASURE
Indicate DIRECTION of the measure
Identify the OBJECT of the measure
Identify the expected VALUE of the measure
Identify WHERE the measurement will occur
Identify WHEN the measure will be obtained
June 20, 2005
IL Criminal Justice Information Authority
8
The Illinois Experience:
Statement of
Business Problem
The absence of standardized arrest
booking procedures creates
inconsistencies in the criminal
history reporting process, which
contributes to breakdowns in the
quality of Illinois rap sheets.
June 20, 2005
IL Criminal Justice Information Authority
9
The Illinois Experience: Strategic
Goal for Business Problem
Recommend, and implement
mechanisms and processes to
inventory, develop, adopt, publish,
disseminate, and maintain standards
that apply to justice information
sharing.
June 20, 2005
IL Criminal Justice Information Authority
10
Ilinois Misstep: Performance
Measures from Strategic Plan
Objective: By July 2003, Standards
Committee to the Governing Board will
identify procedures for evaluating,
developing, approving, disseminating, and
maintaining standards.
Performance measures:
 Percentage of stakeholders represented &
providing input on advisory committee
 Number of standards adopted by IIJIS Gov. Bd.
 Percentage of IL agencies employing justice
information standards
June 20, 2005
IL Criminal Justice Information Authority
11
Strategic Planning is only the Start
of Integrated Justice Planning
Project
Project
Initiation
Strategic
Planning
Tactical
Planning
Project
Project
June 20, 2005
IL Criminal Justice Information Authority
12
Performance measures derived
from strategic planning are
different from those derived from
tactical planning

Strategic plans are often written at a a very high
level in order to convey the conceptual
framework (particularly in state-level plans)

Strategic performance measures often address
pre-implementation “process” activities, thus
producing a preponderance of “outputs” rather
than “outcomes”
June 20, 2005
IL Criminal Justice Information Authority
13
New Illinois Direction: Business
Objectives as Performance
Measures (from Tactical Plan)
Proposed Solution for Business Problem:
Expand the pilot project in Kane County in which:
1. Individual police departments submit data file
containing arrestee charges, demographics, and
tracking number to state CHRI repository
2. After file is posted to CHRI database, it is
instantaneously sent electronically to court clerk’s
office, along with fingerprint ID number, and is
downloaded to clerk’s database, without the need for
additional data entry.
June 20, 2005
IL Criminal Justice Information Authority
14
Developing Business Objectives
as Performance Measures
1. Identify a basic measure:
Actual Measure: Percentage of
offender records in CHRI system
having final dispositions
June 20, 2005
IL Criminal Justice Information Authority
15
Developing Business Objectives
as Performance Measures
2. Indicate direction of the
measure:
Actual Measure: Increase the
percentage of offender records in
CHRI system having final
dispositions
June 20, 2005
IL Criminal Justice Information Authority
16
Developing Business Objectives
as Performance Measures
3. Identify the object of the
measure:
Actual Measure: Increase the
percentage of felony arrests in CHRI
system having final dispositions
June 20, 2005
IL Criminal Justice Information Authority
17
Developing Business Objectives
as Performance Measures
4. Identify the expected value of the
measure of the measure:
Actual Measure: Increase the
percentage of felony arrests in CHRI
system having final dispositions to
95%
June 20, 2005
IL Criminal Justice Information Authority
18
Developing Business Objectives
as Performance Measures
5. Identify where the measurement
will occur:
Actual Measure: Increase the
percentage of felony arrests in CHRI
system having final dispositions to
95% in the target county
June 20, 2005
IL Criminal Justice Information Authority
19
Developing Business Objectives
as Performance Measures
6. Identify when the measurement
will be obtained:
Actual Measure: Increase the
percentage of felony arrests in CHRI
system having final dispositions to
95% in the target county within the
1st year after project implementation
June 20, 2005
IL Criminal Justice Information Authority
20
Lessons from the Field:
Performance measures cannot be
objectively selected; stakeholders
must reach consensus



Stakeholders, depending on their role within the
justice enterprise, are likely to have strongly
divergent views
Higher-level goals and outcomes will usually be
easier to agree upon than specific performance
measures
Performance measure selection, however, is not
entirely subjective, and some measures are
likely to be agreed upon by all
June 20, 2005
IL Criminal Justice Information Authority
21
Lessons from the Field: It is nearly
impossible to definitively establish
causal linkages to public safety
outcomes




Outcomes become more diffuse the further out
you get from actual processes
Improved information sharing is only one of a
number of factors affecting public safety
A single performance measure may be inadequate
to indicate that a system is working (or not)
A carefully selected family of measures is
preferable, and should be thoroughly tested before
being deployed.
June 20, 2005
IL Criminal Justice Information Authority
22
Lessons from the Field: Public
sector outcome measures are
driven by different factors than for
the private sector




Greater integration success by private industry due to
clearer lines of accountability
While the private sector is driven by the bottom line,
government is driven by the competing needs of many
agencies with differing missions and purposes; lines of
accountability are therefore more diffuse.
Government is accountable not only to the governor,
legislature, and public officials, but to the public as well
Concern that the public believes that the justice system
works as it is portrayed on television creates another
type of accountability
June 20, 2005
IL Criminal Justice Information Authority
23
Performance Measurement Caveats



Most people (including your employees and
consultants) can learn to make the measures
come out the way they think you want them to,
while exerting a minimum of effort in actually
improving a process
Always question the measures you’ve defined,
keeping in mind that the people applying them
could find ways of boosting the measures
without really improving anything
Test each measure to determine if it operates
as expected. Does it always go one way when
things get better and the other when things get
worse?
June 20, 2005
IL Criminal Justice Information Authority
24
Performance Measurement Musts




Early and often, measure and evaluate
progress toward the goals and objectives that
have been defined by the governing body
Continually verify that defined measures
actually correspond with the achievement of
goals and objectives
Resist overreaching : measures of long-term,
global outcomes tend to be unreliable.
Performance measurement is complex and
requires significant expertise
June 20, 2005
IL Criminal Justice Information Authority
25
Resources for Assistance with
Performance Measures (& Overall
Justice Integration Planning)

SEARCH
www.search.org

Center for Society, Law, and Justice
(University of New Orleans)
www.cslj.net
June 20, 2005
IL Criminal Justice Information Authority
26