Transcript The Berks County Juvenile Justice System Improvement

The Berks County Juvenile
Justice System Improvement
Project
Presented by:
Robert N. Williams: Berks County Juvenile Probation Office
Richard D. Steele: Juvenile Court Judges’ Commission
PCCYFS 2012 Annual Spring Conference
Evolution of an Evidence Based
Probation Department
• The search for structured decisionmaking and assessment devices
• The “JJSES” is born
• Initial EBP training
• Utilizing MacArthur funding
• Motivational interviewing
• Georgetown invitation
PCCYFS 2012 Annual
Spring Conference
2
Georgetown University’s Center
for Juvenile Justice Reform
• December 2010 release of “Improving the
Effectiveness of Juvenile Justice Programs:
A New Perspective on Evidence-Based
Practice”
• March 2011 call for letters of interest to
participate in the Juvenile Justice System
Improvement Project (JJSIP)
• Berks partners with JCJC and PCCD to
apply
PCCYFS 2012 Annual
Spring Conference
3
JJSIP
• May 2011: Berks/PA one of four sites
selected nationally
• Other sites selected:
–
–
–
PCCYFS 2012 Annual
Spring Conference
4
Pinellas County, Florida
Maricopa County, Arizona
Hartford, Connecticut
JJSIP Goal
• The reduction of crime and delinquency
and improved positive outcomes for
youth in the juvenile justice system
through the implementation of efficient
and effective juvenile justice
administration.
PCCYFS 2012 Annual
Spring Conference
5
JJSIP Expected Outcomes
• Reduced recidivism rates for juvenile
offenders
• Reduced utilization of detention and outof-home placement, particularly for lower
risk offenders
• Reduced racial and ethnic disparities and
disproportionality in juvenile justice
processing
PCCYFS 2012 Annual
Spring Conference
6
JJSIP Expected Outcomes
• Increased probation completion rates
among juvenile offenders
• More efficient use of resources
• Decreased school dropout, increased
school attendance, performance and
stability of school placement among
juvenile offenders
• Reduced mental health symptoms and
substance abuse among juvenile
offenders
PCCYFS 2012 Annual
Spring Conference
7
JJSIP Components
• OJJDP’s Comprehensive Strategy for
Serious, Violent and Chronic Juvenile
Offenders: James C. Howell and John
Wilson
• The Standardized Program Evaluation
Protocol: Dr. Mark Lipsey
PCCYFS 2012 Annual
Spring Conference
8
Comprehensive Strategy for
Serious, Violent and Chronic
Juvenile Offenders
• Developed by Howell and Wilson for OJJDP
in the 1990’s
• The “probation” aspect of the JJSIP
• Suggests a comprehensive framework and
continuum of services from prevention to
aftercare
PCCYFS 2012 Annual
Spring Conference
9
CS Core Principles
PCCYFS 2012 Annual
Spring Conference
10
• Strengthen the family in its primary
responsibility
• Support core social institutions such as
schools, religious institutions and
community organizations
• Promote delinquency prevention as the
most cost-effective approach
• Intervene immediately and effectively
when delinquent behavior occurs
• Identify and control the small group of
SVC juvenile offenders
Research Basis for the
Comprehensive Strategy for Serious,
Violent and Chronic Juvenile
Offenders
PCCYFS 2012 Annual
Spring Conference
11
• We know the key features of juvenile
offender careers
• We understand how offender careers
develop
• We have effective programs for reducing
the risk of re-offending
• We have “forward-looking” administrative
tools for managing offender risk and
matching youth to programs
Sixteen Cohorts of Juvenile
Offender Court Careers
Serious
34%
Violent
8%
Chronic
15%
C,S & V
4%
64%
PCCYFS 2012 Annual
Spring Conference
12
Maricopa Co. Study
(N=151,209)
Non-Serious
Non-Violent
Non-Chronic
Maricopa County Study Key
Findings
• Almost two-thirds (64%) of juvenile court
careers were nonchronic (less than four
referrals) and did not include any serious
or violent offenses; 18% of all careers had
serious (but nonviolent) offenses, 8% had
violent offenses, and 4% of the careers
had serious, violent, and chronic
offenses.
PCCYFS 2012 Annual
Spring Conference
13
Comprehensive Strategy for
Serious, Violent and Chronic
Juvenile Offenders
PCCYFS 2012 Annual
Spring Conference
14
Comprehensive Strategy for
Serious, Violent and Chronic
Juvenile Offenders
PCCYFS 2012 Annual
Spring Conference
15
• The CS is a two-tiered system for
responding proactively to delinquency
• The first tier (prevention, youth
development and early intervention)
focuses on reducing the likelihood that
at-risk youth will enter the juvenile
justice system
• The second tier (interventions and
graduated sanctions) addresses risk
factors of system youth with a focus on
SVC offenders
Comprehensive Strategy for
Serious, Violent and Chronic
Juvenile Offenders
• But we aren’t funded or don’t have the
time to address prevention!
• At the very least, you need to be involved
in community prevention efforts to
ensure that risk factors observed in kids
referred to the court are being addressed
• Do you have diversion programs in place?
PCCYFS 2012 Annual
Spring Conference
16
CS Intervention and Graduated
Sanctions Highlights
• Build a seamless continuum of services
with graduated placement levels
• Utilize structured decision making
instruments across the continuum
• Match offenders to the appropriate level
of supervision and services based on the
R-N assessment
• Place low risk offenders in community
programs with minimal supervision
PCCYFS 2012 Annual
Spring Conference
17
CS Intervention and Graduated
Sanctions Highlights
• Medium risk offenders are typically
placed in more structured communitybased programs with greater supervision
• Very high risk offenders are typically
placed in residential settings
• SVC offenders make up <10% of the
offending population
PCCYFS 2012 Annual
Spring Conference
18
CS Intervention and Graduated
Sanctions Highlights
PCCYFS 2012 Annual
Spring Conference
19
• Most juvenile offenders enter the system
with minor offenses and low recidivism
risk
• Some minor offenders are on pathways
toward serious, violent and chronic
offending
• Risk assessment devices are available that
measure risk well enough to guide
allocation of system response and effort
• To be effective, interventions should
address priority treatment needs as
identified through assessment
Standardized Program Evaluation
Protocol (Spep)
• The “provider” aspect of the JJSIP
• Based on Dr. Mark Lipsey’s meta-analysis
work with nearly 700 research studies
over the past twenty years
• Juveniles aged 12-21 in programs aimed
at reducing delinquency
• Focus on the programs’ effects on
recidivism (reoffending)
PCCYFS 2012 Annual
Spring Conference
20
SPEP Defined
PCCYFS 2012 Annual
Spring Conference
21
• The SPEP is a validated, data driven rating
scheme for determining how well an
existing program matches research
evidence for the effectiveness of that
particular program for reducing the
recidivism of juvenile offenders
• More simply put, the SPEP is a tool for
comparing juvenile justice programs to
what has been found effective for
reducing recidivism in the research
• Effectiveness = the ability to reduce the
recidivism of juveniles receiving a
program compared to a control group
that does not receive the program
Effects on recidivism analyzed as
a function of program
characteristics
PCCYFS 2012 Annual
Spring Conference
22
© Copyright held by Mark W. Lipsey, Peabody Research Institute, Vanderbilt
University. Not to be copied or used without explicit permission.
Program types sorted by general
approach: Average recidivism effect
Discipline
Deterrence
Surveillance
Restorative
Skill building
Counseling
Multiple services
-10
-5
0
5
10
15
% Recidivism Reduction from .50 Baseline
PCCYFS 2012 Annual
Spring Conference
23
© Copyright held by Mark W. Lipsey, Peabody Research Institute, Vanderbilt
University. Not to be copied or used without explicit permission.
Further sorting by intervention
type within, e.g., counseling
approaches
Individual
Mentoring
Family
Family crisis
Group
Peer
Mixed
Mixed w/referrals
0
PCCYFS 2012 Annual
Spring Conference
24
5
10
15
20
25
% Recidivism Reduction from .50 Baseline
© Copyright held by Mark W. Lipsey, Peabody Research Institute, Vanderbilt
University. Not to be copied or used without explicit permission.
Further sorting by intervention
type within, e.g., skill-building
approaches
Behavioral
Cognitive-behavioral
Social skills
Challenge
Academic
Job related
0
PCCYFS 2012 Annual
Spring Conference
25
5
10
15
20
25
30
% Recidivism Reduction from .50 Baseline
© Copyright held by Mark W. Lipsey, Peabody Research Institute, Vanderbilt
University. Not to be copied or used without explicit permission.
Many types of therapeutic
interventions thus have evidence of
effectiveness … but there’s a catch:
• Though their average effects on
recidivism are positive, larger and smaller
effects are distributed around that
average.
• This means that some variants of the
intervention show large positive effects,
but others show negligible or even
negative effects.
PCCYFS 2012 Annual
Spring Conference
26
© Copyright held by Mark W. Lipsey, Peabody Research Institute, Vanderbilt
University. Not to be copied or used without explicit permission.
Example: Recidivism effects from
29 studies of family interventions
Family Interventions
Covariate-Adjusted Recidivism Effect Sizes (N=29)
>.00
Average
recidivism
reduction of
13%
-.40
PCCYFS 2012 Annual
Spring Conference
27
-.30
-.20
-.10
.00
.10
.20
Median
.30
.40
.50
.60
Effect Size (zPhi coefficient)
© Copyright held by Mark W. Lipsey, Peabody Research Institute, Vanderbilt
University. Not to be copied or used without explicit permission.
Where are the brand name model
programs in this distribution?
Family Interventions
Covariate-Adjusted Recidivism Effect Sizes (N=29)
>.00
FFT
MST
-.40
PCCYFS 2012 Annual
Spring Conference
28
-.30
-.20
-.10
.00
.10
.20
Median
.30
.40
.50
.60
Effect Size (zPhi coefficient)
© Copyright held by Mark W. Lipsey, Peabody Research Institute, Vanderbilt
University. Not to be copied or used without explicit permission.
SPEP BASICS
• Lipsey’s analysis looked at how a
program’s components are related to its
impact on recidivism
• He identified the primary characteristics
of effective interventions with delinquents
• The SPEP creates a metric by assigning
points to service programs according to
how closely their characteristics match
those programs of as given type that
show the best recidivism outcomes
PCCYFS 2012 Annual
Spring Conference
29
SPEP BASICS
• Lipsey found that the effects of juvenile
delinquency interventions are mainly
related to four key aspects of an
intervention:
–
–
–
–
PCCYFS 2012 Annual
Spring Conference
30
Type of program/service
Quantity or dosage amount
Service quality
Juvenile risk level
SPEP BASICS
• The SPEP is used only to evaluate therapeutic
services (services oriented mainly toward
facilitating constructive internalized
sustained changes in behavior)
• Therapeutic services include:
–
–
–
–
PCCYFS 2012 Annual
Spring Conference
31
Restorative services
Counseling and its variants
Skill-building services
Services for special populations
• Control-oriented services and services for
which there is insufficient research are not
able to be rated
SPEP BASICS
Many types of programs for diverse needs are
supported by evidence:
–
–
–
–
–
–
–
–
PCCYFS 2012 Annual
Spring Conference
32
Interpersonal skills
Family and parenting
Individual counseling
Life Skills
Sex offender treatment
Cognitive-behavioral
Challenge programs
Milieu therapy
Victim-offender mediation
Drug & alcohol treatment
Behavioral & incentive
Group counseling
Social casework
Educational
Vocational
Psychodynamic
SPEP PROCESS
PCCYFS 2012 Annual
Spring Conference
33
• Identify programs being provided to
offenders
• Break programs down into services that
can be matched with research-based
categories
• Collect demographic, risk, quality and
quantity data for each service
• Enter data into the SPEP model to
generate SPEP score
• Evaluate performance based on SPEP
components and overall score
SPEP SCORING
PCCYFS 2012 Annual
Spring Conference
34
• Points are assigned to each category
based on the findings of Lipsey’s research
• Points are proportionate to the
contribution of each rated factor to
recidivism reduction
• Target values are set from either the
meta-analysis (generic) or program
manual for blueprint programs
• The SPEP is configured to so that the
maximum total score for any service is
100 points
• Points
assigned
proportionat
e to the
contribution
of each
factor to
recidivism
reduction
PCCYFS 2012 Annual
Spring Conference
35
• Target
values from
the metaanalysis
(generic) OR
program
manual
(manualized)
Standardized Program Evaluation Protocol (SPEP)
for Services to Juvenile Offenders©
Recalibrated version, 2012
Points
Possible
Points
Received
Primary and Supplemental Service Types
[Identified according to definitions derived from the research]
Primary Service Type for Program Being Rated
Group 1 services (5 points)
points)
Group 2 services (10 points)
points)
Group 3 services (15 points)
Supplemental Service Type
Qualifying supplemental service used: Yes (5 points)
Group 4 services (25
30
Group 5 services (30
No (0 points)
5
Quality of Service Delivery
[Determined from a systematic assessment of the relevant
features of the provider and provider organization]
Rated quality of services delivered:
Low (5 points)
Medium (10 points)
High (20 points)
20
Amount of Service
[Determined from data for the qualifying group of service recipients]
Duration [Target number of weeks specified for each service type]
% of youth who received at least the target weeks of service:
0% (0 points) 60% (6 points)
20% (2 points) 80% (8 points)
40% (4 points) 99% (10 points)
Contact Hours [Target number of hours specified for each service type]
% of youth who received at least the target hours of service:
0% (0 points) 60% (6 points)
20% (2 points) 80% (8 points)
40% (4 points) 99% (10 points)
10
10
Risk Level of Youth Served
[Determined from risk ratings on a valid instrument
for the qualifying group of service recipients]
% of youth with at least the target risk score set for the JJ system:
0% (0 points) 60% (15 points)
20% (5 points) 80% (20 points)
40% (10 points) 99% (25 points)
Provider’s Total SPEP Score
25
100
(Insert Score)
Understanding A Spep Score
• The SPEP score is configured so that the
total score indicates how closely the key
characteristics of a rated program match
those of the programs of that type that
showed at least average positive effects
in the corresponding research
PCCYFS 2012 Annual
Spring Conference
36
• The difference between the scores for the
individual SPEP components and the
maximum values possible for each
provide a diagnostic information about
where the rated program can improve
Understanding A Spep Score
• SPEP scores have been shown to relate to
programs’ effects on recidivism and can
thus provide useful guidance for program
evaluation and improvement
• However, there are almost certainly other
factors related to a program's
effectiveness that should be taken into
consideration, most notably the clinical
quality of the interaction between
juveniles and their service providers
PCCYFS 2012 Annual
Spring Conference
37
I’ve Got My Score, Now What?
• Scores should be considered a baseline
and a road map for program
improvement
• What can I do to increase my score?
• How can I work with probation and the
courts to improve my effectiveness?
PCCYFS 2012 Annual
Spring Conference
38
Hypothetical SPEP Score
Current Score
100
90
80
70
60
50
40
30
20
10
0
PCCYFS 2012 Annual
Spring Conference
39
95
46
2525
0
5
610
10
0
25
20
10
5
Program Improvement Strategies
• PRIMARY SERVICE (5-30 points)
• Are there any circumstances where a
provider might consider changing the
primary service?
• Dosage requirements can’t be met due to
caseload sizes
• Service not meeting the needs of the
target population
• A more potent service?
PCCYFS 2012 Annual
Spring Conference
40
Program Improvement Strategies
• SUPPLEMENTAL SERVICE (0 or 5 points)
• Can a supplemental service be added that
fits (reinforces or augments) with the
primary service?
PCCYFS 2012 Annual
Spring Conference
41
Program Improvement Strategies
•
•
•
•
AMOUNT OF SERVICE
DURATION (0-10 points)
CONTACT HOURS (0-10 points)
Are youth receiving the target amount of
service?
• How can retention be improved?
• Are youth removed beyond your control?
• Are therapist caseload sizes preventing
the meeting of dosage requirements?
PCCYFS 2012 Annual
Spring Conference
42
Program Improvement Strategies
•
•
•
•
•
•
PCCYFS 2012 Annual
Spring Conference
43
QUALITY OF SERVICE (5-20 points)
What are the quality indicators?
Manualized protocols
Training, coaching and feedback
Monitoring to maintain fidelity
How is corrective action taken when staff
departs from protocol or quality lapses?
Program Improvement Strategies
• RISK LEVEL (0-25 points)
• More points given for the % of high-risk
youth involved in the program
• Consider inclusionary and/or
exclusionary admission criteria with
preferred risk level indicated
• At the mercy of the courts?
PCCYFS 2012 Annual
Spring Conference
44
Hypothetical Second Year Score
Current Score
100
90
80
70
60
50
40
30
20
10
0
PCCYFS 2012 Annual
Spring Conference
45
Second Year Score
90
73
46
252525
00
5
20
15 1515 15
10 10
10
68
5
0
Why Do Some Programs Fail to
Respond to the Researcher’s Call
for Change?
PCCYFS 2012 Annual
Spring Conference
46
• Strong skepticism about the applicability of
research to practice.
• A strong belief across systems (counselors,
program administrators, judges, POs) that
the program already works.
• A belief that the program delivers other
more important outcomes beyond
recidivism reduction.
• Legitimate funding challenges associated
with change.
• Pride!
SPEP BEST PRACTICE GUIDELINES
• Therapeutic approaches are more
effective
• Larger effects will be seen with higher
risk youth
• The amount of service should at least
match the average of effective
programming according to the research
• High quality implementation, including
treatment protocols and monitoring for
adherence, is important
PCCYFS 2012 Annual
Spring Conference
47
JJSIP IMPLEMENTATION
• 18 month project
• Berks is the pilot site for an anticipated
statewide rollout
• Elements of the JJSIP have already been
incorporated into PA’s Juvenile Justice
System Enhancement Strategy
PCCYFS 2012 Annual
Spring Conference
48
History of JJSES
• Concept of JJSES “born” in June 2010 at
JCJC/Chiefs Annual Strategic Planning Meeting
• Designed to organize / sustain “lessons
learned” from Models for Change
• Risk/Need Assessment (YLS) activities were
increasing – needed to move to the next stage
• EBP Exposure through Mark Carey Training
• Leadership Team created
PCCYFS 2012 Annual
Spring Conference
49
Keith Snyder, Coordinator (JCJC)
Bob Tomassini (Adams)
Bob Williams (Berks)
Sam Miller (Cumberland)
Beth Fritz (Lehigh)
Mike Pennington (PCCD)
Rick Steele (JCJC)
• Statement of Purpose Developed
JJSES Statement of Purpose
• We dedicate ourselves to working in
partnership to enhance the capacity of
Pennsylvania’s juvenile justice system to
achieve its balanced and restorative justice
mission by:
– Employing evidence-based practices, with
fidelity, at every stage of the juvenile justice
process;
– Collecting and analyzing the data necessary to
measure the results of these efforts; and, with
this knowledge,
– Striving to continuously improve the quality of
our decisions, services and programs.
PCCYFS 2012 Annual
Spring Conference
50
“Statement of Purpose”
Endorsements
PCCYFS 2012 Annual
Spring Conference
51
• Juvenile Court Judges’ Commission (JCJC)
• Juvenile Court Section of the Pa. Conference
of State Trial Judges
• Pa. Council of Chief Juvenile Probation
Officers
• Juvenile Justice and Delinquency Prevention
Committee of the Pa. Commission on Crime
and Delinquency
• County Juvenile Probation
Departments/Juvenile Courts
• Over 30 service provider organizations and
agencies (including JDCAP, PCCYFS, PCPA)
Leadership Team’s Initial Activities
• Identification of various initiatives /
activities
• Who’s “in charge”?
• Where is the “home” of each initiative /
activity?
• What’s the status of each initiative / activity?
PCCYFS 2012 Annual
Spring Conference
52
• Is there a sustainability plan?
Elements of Pennsylvania’s
Models for Change Initiatives
PCCYFS 2012 Annual
Spring Conference
53
Challenges:
PCCYFS 2012 Annual
Spring Conference
54
•
The pieces of the puzzle were at various
stages of implementation among
jurisdictions
•
How do we “transform” the pieces of the
puzzle into a comprehensive strategy?
•
What is the recommended sequence of
activities for probation departments,
providers, and others?
•
Do we have the necessary infrastructure to
support implementation of each element?
•
Communications strategy
•
Stakeholder involvement
•
What does “evidence-based” really mean?
Programs / interventions can be
placed along a “proof of
effectiveness” continuum
How confident are we that a program will
improve outcomes?
 Best Practices
Not much
confidence
“We’ve done it
and we like it”
 Research-based
“This program is based on
Very much
sound
confidence
theory informed by research”
 Promising Practices
PCCYFS 2012 Annual
Spring Conference
55
“We really think this
will work… but we need
time to prove it”
 Evidence-based
“This program has
been rigorously evaluated
and shown to work”
Bumbarger, B. K. (2009). Promoting the Use of Evidence-based Prevention:
Application in the Real World. Available online at https://breeze.psu.edu/p22215525
Intermediate Goals / Activities
• Work with The Carey Group to develop a
framework for an implementation strategy
• Create a JJSES Workgroup to achieve broader
juvenile justice system representation and
involvement
• Develop a JJSES “Monograph”
• Develop infrastructure to support activities
• Integrate “lessons learned” from PA’s
participation in the Juvenile Justice System
Improvement Project (JJSIP)
PCCYFS 2012 Annual
Spring Conference
56
JJSES Framework
Achieving our Balanced and Restorative Justice Mission
PCCYFS 2012 Annual
Spring Conference
57
What Can We Expect?
“Research has shown that
the risk of recidivism is greatly
reduced (10-30% on average)
when attention is paid
to criminogenic needs.”
PCCYFS 2012 Annual
Spring Conference
58
SOURCE: D.A.Andrews, I. Zinger, R.D. Hoge, J. Bonta, P. Gendreau and F.T. Cullen,
Does correctional treatment work? A clinically relevant and psychologically informed metaanalysis,
Criminology, 28 (1990); Andrews (2007)
Goals for 2012
PCCYFS 2012 Annual
Spring Conference
59
• PCCD planning/implementation grants to
support local implementation of JJSES
• 6 Regional training programs for county
teams
• Development of JJSES Monograph and “How
To” Guide for Chief Juvenile Probation
Officers
• County survey to determine extent to which
elements of JJSES Framework have been
implemented
• Development of Leadership Academy /
management training curricula
• Carey Guide training
Goals for 2012
PCCYFS 2012 Annual
Spring Conference
60
• Phase IV (Final) YLS County Training
• Rollout of Standardized Case Plan tied to
YLS assessments
• Development of online “Program and
Practice Effectiveness Tool Kit” to clarify
definitions and list evidence-based curricula
• Increased capacity of Resource Center for
Evidence-based Programs and practices
• Increase capacity of PaJCMS data analysis
infrastructure
• Development of infrastructure to support
statewide implementation of SPEP
Contact Information
PCCYFS 2012 Annual
Spring Conference
61
•
•
•
•
Robert N. Williams
Chief Juvenile Probation Officer
Berks County Juvenile Probation Office
[email protected]
•
•
•
•
Richard D. Steele
Director of Policy & Program Development
Juvenile Court Judges’ Commission
[email protected]