Transcript Chapter 3: Prenatal Development, Birth, and the Newborn

Outcomes Assessment in
School Mental Health
Webinar and dialogue sponsored by the
Quality and Evidence-Based Practice (QEBP)
Practice Group and the IDEA Partnership
March 16, 2011
Today’s webinar
• Overview of QEBP practice group
• Highlights of QEBP practice group discussions at 2010
CSMH conference (Albuquerque, NM)
• Evaluation of school mental health (SMH) programs in
Minnesota Public Schools (Dr. Mark Sander)
• Psychosocial, Placement and Cost Assessment in
Maryland’s SMH programs for youth in special education
(Dr. Sharon Stephan)
• QEBP updates from the CSMH (Dr. Nicole Evangelista)
• Call for Papers on SMH outcomes assessment - special
issue of Advances in School Mental Health Promotion
(Dr. Michael Kelly)
QEBP Practice Group Mission and Priorities
•
Mission: To provide resources and promote sharing
of information across individuals and groups
interested in improving the quality of school mental
health (SMH) programs and services.
•
Priority areas include:
•
•
•
improving dissemination and sharing of evidence-based
practices in SMH;
bridging the research-practice and practice-research gaps
in the field;
and, understanding and promoting the use of the best
student-, program-, and school-level evaluation strategies.
QEBP practice group discussions at 2010
CSMH conference (Albuquerque, NM)
• Split into two groups (research and front line)
to answer 3 questions:
1) What are our most pressing concerns related to
Quality and Evidence-based Practice?
2) What is one specific action sep for our group
(research or front line) this coming year?
3) What is one specific action step you would like to
see from the “other” group this coming year?
Next Steps for QEBP Practice Group
• Future webinars and dialogue
School Mental Health: Local Evaluation
and Research Lessons Learned
Mark Sander, Psy.D., LP
Hennepin County/Minneapolis Public Schools
[email protected]
612-668-5489
School Mental Health Programs Across MN
 Minneapolis Public Schools established in 2005 in
five schools
 Gradually expanded to 15 schools
 County wide – 10 agencies, 14 school districts, over
70 schools
 State wide – 21 grantees, serving 63 counties, about
200 school districts, and 550 schools
Collaboratively Building a Evaluation Model
 What is “must have” data?
o
o
o
Mental Health Outcome (SDQ and CASII)
Educational Outcome (Suspension and Attendance)
Access and Engagement
 Who informs it?
o
o
o
Stakeholder “interviews”
Going back and talking to stakeholders
Survey of Principal, Assistant Principals and Social Workers
(later years)
Data Informed Sustainability Planning
 How do you do it?
o
o
We went low tech for the first 4 years
Now we have developed on-line database
 How do you fund it?
o
Used SS/HS funding to develop database and work with statistician
on analysis
 How do you use it?
o
o
o
o
Developed preliminary evaluation model
Referral and service trends
Impact on programming decision
Impact on financial model and cash flow for agencies
Utilizing the Data to Inform and Sustain
 Referral and Service Trends
o Impact on programming decision
o Impact on financial model and cash flow for agencies
 Case for Ancillary and Supportive Services
o Teacher Consultation
o Care Coordination
o Training for educational staff; classroom presentations
 Case for educational Benefit
o Reduction in Suspensions
o Reduction in Suspensions for students with 3 or more
o Reclaimed instructional time (not analyzed yet)
Referral Data SY06
E SM H R e f e r r a l SY 0 6
60
56
50
37
40
35
30
Ser i es1
23
23
19
17
20
12
11
10
3
1
1
0
Jul 05
Aug 05
Sept 05
Oct 05
Nov 05
Dec 05
Jan 06
Feb 06
M ar 06
Apr 06
M ay 06
Jun 06
A
ug
-0
Se 6
p06
O
ct
-0
6
N
ov
-0
6
D
ec
-0
6
Ja
n07
Fe
b07
M
ar
-0
7
A
pr
-0
7
M
ay
-0
Ju 7
n07
Ju
l-0
7
Service Data SY07
900
800
700
600
500
400
300
200
100
0
MPS Student Demographics SY06-SY09
 1017 Students
 61% male and 39% female
 General Ed: 55% and 45% Special Ed (SPED 15.9%)
 Ethnicity






NATIVE AMERICAN – 8.3% (4.5%)
AFRICAN AMERICAN – 48.9% (39.6%)
ASIAN – 2.1% (9.0%)
HISPANIC – 28.1% (17.1%)
WHITE – 12.6% (29.8%)
NATIVE HAWAIIAN PAC ISL – 0%
 ELL – 31.2 % (23.2%)
District wide percentages are in parenthesis
Evaluation of MPS ESMHP
 Access and sustained engagement in treatment
 85 % of students seen once face to face; 70% within 10 days; 65%
1st time receiving services
 Average 17 visits per school year
 Improved mental health functioning
 Parents and teachers report decreases in the emotional and
behavioral problems
 Improved school functioning
 Decrease in school suspensions for students receiving mental
health treatment
 Principals and school social workers reported reduced office
referrals and student suspensions
Impact on Suspensions
 SY07 N=298 Seen 4x and had 1 suspension SY06
(n=82); only 27.5% had suspension in SY06
 Impact on suspensions: SY06 to SY07 – N=82
 50% reduced 1-6
 32% stayed the same
Multiply Suspended SY06-07
 SY06-SY07 Multiply Suspended (N=19)




SPED=64%
ELL=37%
Male=75%
African American=63%
 2 suspensions or less=78.9%
 52.6% to 1 or less
 Moved to Zero=37%
Multiply Suspended SY06-SY07 (N=19)
9
8
7
Number of Suspensions
6
5
SY06
SY07
4
3
2
1
0
1
2
3
4
5
6
7
8
9
10
11
Individual Students
12
13
14
15
16
17
18
19
Change in Mean Suspensions: SY06-SY07
Impact on Suspensions: 156 Tx and 133 Comparison
2.00
1.80
1.80
1.60
Aver Days Suspended
1.40
1.20
Full Tx
1.00
Full Comp
0.92
0.80
0.83
0.60
0.54
0.40
0.20
0.00
Baseline
Af ter Treatment
MPS – SY07 Change Mean Suspension: Full/African American
Impact on Suspensions: Full: 156 Tx, 133 Com; AA: 83 Tx, 76 Comp
2.00
1.80
1.80
1.60
1.40
1.30
1.20
1.13
1.17
Full Tx
Full Comp
1.00
Af rican Amer Tx
0.92
0.80
Af rican Amer Comp
0.83
0.61
0.54
0.60
0.40
0.20
0.00
Baseline
Af ter Treatment
Change in Attendance: SY06 to SY07 (N=35, N=20)
 Total Sample of Treatment
kids = 159

40 (25%) had less than 90%
Attendance in SY06 (5 kids
missing data in SY07;
sample 35 students)
 Total Sample of 133
comparison

20 (15%) of them had lower
than 90% attendance in SY06
Treatment
Comparison
N
%
N
%
35
100
20
100
22
62.9%
7
35%
Psychosocial, Placement and Cost Outcomes
in Maryland’s SMH Programs for Youth in
Special Education
Sharon H. Stephan, Ph.D.
University of Maryland School of Medicine
Center for School Mental Health
Acknowledgements
• Maryland State Department of Education
• Dr. Carol Ann Heath
• Dr. Jodi King
• Prince George’s County Public Schools
• Dr. Pamela Downing-Hosten
• Dr. Florence Foreman
• Baltimore City Public Schools
• Ms. Kim Lewis
• Center for School Mental Health
• Drs. Dana Cunningham, Eric Slade, Mark Weist, Nancy
Lever
• Christianna Andrews, Kerri Chambers, Emily Sidway
Background
• Nationally, there has been an upward trend in students
identified with emotional disturbances (ED) requiring
intensive services to access educational standards.
• This trend is evident in Maryland with the Maryland State
Department of Education listing ED as a high growth
category increasing approximately 75% over the last ten
years.
• In the five years prior to initiating the Prince George’s
County SMH Initiative, the use of non-public placements in
Maryland had increased by 20%, with some districts their
use by over 30%.
Current Issues in
Nonpublic Placements
• Non-public placements cost local school districts over
40K per student
• Many districts do not have mental health services
that are devoted to students identified as disabled
• There are multiple indicators that the use of nonpublic placements are racially disproportionate
• The outcomes associated with these placements are
inconsistent
Rationale for Implementation
• Initiation of PGSMHI (2006)
• Collaboration between MSDE, state
university, and local school system
• Concern over high rate of referrals to
non-public schools
• Concern over racial disproportionality in
placement in non-public schools
• High costs of non-publics
• Relative lack of school based services
Rationale for Implementation
• Initiation of BSMHI (2009)
• Collaboration between MSDE, state
university, and local school system
• Concern over high rate of referrals to
non-public schools
• High costs of non-publics
• Documented success of PGSMHI
Target Population
• Students in special education who are at
risk of entering non-public settings due to
an increase in behavioral and/or emotional
problems
• Students in non-public settings who are
prepared to return to their home school
Program Model
• Therapists
• Case managers
• Psychiatric
consultation
• Family Support
Partners (BSMHI)
Limited Local Capacity for Data Analysis
• The data on highly restrictive placements was not
organized to answer questions and guide
interventions
• Breaking this barrier:
• Identified risk factors that contributed to restrictive
placements - Age, Grade, Compliance Issues,
Suspensions, Disability, Gender
• Analyzed national, state, local and individual data
sources
• Targeted efforts on suspected risk factors
• Collected extensive data on clients
Breaking Down Barriers
• Financing
• Total costs of Non-public and alternative programs are
hard to determine
• Non-public placements involve financing from two
sources- the state and local level
• Completed low and high estimates of cost savings for
both the state and local district
• Moved thinking from “GRANT” thinking to “DIVERSION”
thinking.
Economic Evaluation
• Determined costs associated with PGSMHI
based on 3 years of data and 77 students
• Personnel
• Trainings
• Equipment and supplies
• Determined average cost of nonpublic
placements
• Education costs
• Transportation costs
Economic Evaluation
• Moved from dichotomous thinking ….stay or go, Emotionally
Impaired or not, etc.
• to every day counts = continuous variable
success
for each student
• Question changed to how many days was the student
maintained in the public setting?
• Unit of analysis is the number of days prevented in non-public
(not whether the student was placed)
• Cost determined by the number of days placement was delayed
or averted times cost differential per day
Costs
Aggregate
Total
Nonpublic
Costs
Total
PGSMHI
Costs
Total
Savings
3,964,928
Per student Per student
per day
per year
163
39,038
732,487
30
7,212
3,232,440
133
31,826
Note: Data is based on a program year of 240 days
Cost Savings
• $133 per student/day averted
• For students receiving intense
services and are considered at
imminent risk for non-public
placement
• These savings are not expected for
students who have emotional
conditions but are not at risk for
non-public placements
• These savings are not expected for
“usual care” mental health services
provided in a school
Limitations of Evaluation
• Did not assess indirect costs of adverse
consequences for school staff and students of
retaining students in public schools
• No comparison sample of students in nonpublic
setting
• Assume that students would have been quickly
placed in a nonpublic setting without intervention
Student GPA
2.5
2
1.5
1
0.5
0
Enroll
2nd Qtr
3rd Qtr
4th Qtr
Courses Passed
8
7
6
5
4
3
2
1
0
Enroll
2nd Qtr
3rd Qtr
4th Qtr
Student Observations
35
30
25
20
Obs 1
Obs2
15
10
5
0
Off Task
Tchr
Prompt
Neg Resp
Attn
All Symptoms
3.5
3
2.5
2
Intake
Time 2
1.5
1
Youth Intake N = 59
Parent Intake N = 47
0.5
0
Youth
Parent
Teacher
BSMHI Placement Outcomes
• 93% of the students enrolled
in the BSMHI were maintained
in their current placement.
• The number of students
referred to non public
placement at primary school
decreased 50% from the
2008-2009 school year to the
2009-2010 school year.
BSMHI Cost Outcomes
• Students were diverted a total of 24.3 years
from more restrictive placements. With an
estimated additional cost of $31,915* per student
per year for a nonpublic placement,
approximately $775,534 was saved the first
year of this project.
$43,000 Nonpublic Placement for 1 Year
- $11,085 Public School Cost for 1 Year
$31,915 Add’l Cost for Nonpublic Placement
x
24.3 Placement Years Deferred
$775,534.50 Cost Savings
Center for School Mental Health
• Objective 1.2 Continue and strengthen ongoing CSMH efforts to
conduct comprehensive analyses of established, data driven and
impactful local and state SMH programs, with data-driven findings
and case examples shared in yearly summary reports (12/11,
12/12, 12/13).
• Objective 1.3 Enhance CSMH efforts to assist state and local
program leaders to identify and measure targeted student and
school-level outcomes and to incorporate data-based decision
making into practice via a quarterly SMH outcomes/data
management newsletter, annual webinar (5/11, 5/12, 5/13), and a
regularly maintained compendium of online resources.
• Objective 2.7 Narrow the research-practice divide and improve
communication and knowledge sharing, and facilitate partnerships
between researchers and other SMH stakeholders via the
development and convening of a SMH Research-Practice Network
and encouraging increased participation of researchers in the
annual conference and National COP.
Call for Papers!
• Special Issue on School-Based Mental Health Practice, DataDriven School Mental Health Practice: (How) Is It Possible?
Advances in School Mental Health Promotion
This issue will seek to better understand how data-driven and
research-based educational interventions are being manifested in
school mental health delivery.
For more information, please contact the Special Issue Guest Editor
Michael S. Kelly at [email protected].
The deadline for manuscript submission is June 15,
2011. Submissions should be sent to: [email protected]
Discussion
• Please unmute your phone so that you
can participate in the discussion.
• You also can type in questions on the
iLinc website using the instant
messaging.