Transcript Implications of CFSR 3 for IV

Implications of CFSR 3 for IVE
Programs
Richard P. Barth & Terry V. Shaw
October 23, 2014
CSWE APM
Tampa, Florida
Purpose of the Presentation
• To stimulate thinking and discussion about:
– What is the meaning of CFSR 3?
– Does it create new opportunities for SSW
University-Agency Partnerships?
– Does it suggest changes in SW Education?
CFSR 3 Guiding Principles
• States and the Federal Government are unified in a
commitment to safety, permanency, and well-being for
the children served by the public child welfare system.
• The CFSRs have led to the identification of opportunities
for:
– 1. program improvement
– 2. improvement in outcomes for children and families.
• Further improvements in the CFSR and in the related CQI
processes are critical to the continued CWs advances
–
–
–
–
Scientifically valid
Feasible
Understandable
Meet All Federal Requirements
National Standards (Issues)
Proposal
Points of
Agreement
Areas for
Discussion
Input
Aggregate
administrative data on
safety and
permanency
Recommendation to endorse
CB proposal; or
Compute State-specific rates
and calculate a national
average rate
Applied
Methodology
Multi-level Model
Risk Adjusted
Recommendation to endorse
CB proposal; or
Endorse with additional factors;
or; Do not risk adjust
Output
Set National Standard
at national average
for all measures
Proposal is
reasonable
National Standards
Safety Outcome 1: Children are, first and foremost, protected from abuse and
neglect.
Safety Performance Measure 1: Maltreatment in Foster Care
CFSR Round 2
Of all children in foster care
during the reporting period,
what percent were not
victims of substantiated or
indicated maltreatment by a
foster parent or facility staff
member?
CFSR Round 3
Of all children in foster
care during a 12-month
period, what is the rate of
victimization per day of
foster care? (most likely
reported in a rate per
100,000 days)
Differences
Child maltreatment while in care
is calculated by directly linking
data from NCANDS (numerator)
with the AFCARS file
(denominator) using CHILDID.
All instances of maltreatment will
be included regardless of type of
perpetrator (including parents).
Re-Report of Maltreatment (Issues)
Proposal
Of all the children who
were victims of
substantiated or indicated
abuse or neglect during the
first 6 months of the
reporting period, what
percent experienced
another screened-in report
with a 12-month period?
Points of
Agreement
It is important to include
a measure of repeat
maltreatment.
No support for the
measure as proposed
“…..getting the measure
right should trump
concerns over current
data capacities.”
Areas for
Discussion
Recommendations for
appropriate measure(s)?
Recommendations around
data availability, quality and
testing proposed measures?
National Standards
Safety Outcome 1: Children are, first and foremost, protected from abuse and
neglect.
Safety Performance Measure 2: Re-report of Maltreatment
CFSR Round 2
Of all children who were
victims of substantiated or
indicated maltreatment
allegation during the first 6
months of the reporting
period, what percent were
not victims of another
substantiated or indicated
maltreatment allegation
within a 6-month period?
CFSR Round 3
Of all children with a
screened-in report of
alleged maltreatment in a
12-month period, what
percent had another
screened-in report within
12 months of their initial
report?
Differences
Lengthens the timeframes being
considered (from 6 months to 12
months)
increases the scope to include all
screened-in reports and not just
Substantiated or Indicated.
National Standards
Permanency Outcome 1: Children have permanency and stability in their
living situations.
Permanency Performance Measure 1: Permanency in 12 months for
children entering foster care
CFSR Round 2
CFSR Round 3
Composite 1.3: Of all
children entering foster care
for the first time in a 6month period, what percent
discharged to reunification
(or live with relative) within
12 months of entering foster
care or by the time they
reached 18?
Of all children who enter
foster care in a 12 month
period, what percent
discharged to permanency
within 12 months of
entering foster care?
Differences
Expanded the exits from
reunification alone, to
permanency (reunification, live
with relative, adoption or
guardianship).
Expands base time from 6
months to 12 months and
includes all entries, not solely
first entries.
National Standards
Permanency Outcome 1: Children have permanency and stability in their
living situations.
Permanency Performance Measure 2: Permanency in 12 months for
children in foster care for 2 years or more
CFSR Round 2
Composite 3.1: Of all
children in foster care on
the first day of a 12-month
period who had been in
foster care (in that episode)
for 2 or more years, what
percent discharged to
permanency within 12
months of the first day or by
the time they reached 18?
CFSR Round 3
Differences
Of all children in foster care No differences.
on the first day of a 12month period who had
been in foster care (in that
episode) for 2 or more
years, what percent
discharged to permanency
within 12 months of the first
day?
National Standards
Permanency Outcome 1: Children have permanency and stability in their
living situations.
Permanency Performance Measure 3: Re-entry in 12 months
CFSR Round 2
Composite 1.4: Of all
children discharged from
foster care to reunification
or live with a relative in a
12- month period, what
percent re-entered foster
care in less than 12 months
from the date of discharge?
CFSR Round 3
Of all children who enter
foster care in a 12- month
period and discharged
within 12 months to
reunification, live with
relative, or guardianship,
what percent re-entered
foster care within 12
months of their date of
discharge?
Differences
The measure is now based on
an entry cohort instead of an exit
cohort.
The denominator has been
expanded and now includes
Guardianship as well as
reunification, and live with
relative.
National Standards
Permanency Outcome 1: Children have permanency and stability in their
living situations.
Permanency Performance Measure 4: Placement stability
CFSR Round 2
Composite 4.1: Of all
children served in foster
care during the 12- month
period, what percent had
two or fewer placement
settings?
CFSR Round 3
Differences
Of all children who enter
foster care in a 12- month
period, what is the rate of
placement moves per day
of foster care? (most likely
expressed as a rate per
1,000 days)
The measure is now based on
an entry cohort instead of all
children served and in care for a
period.
All placement moves will be
counted during a period rather
than just identification of stable
vs. not stable.
Risk Adjusted Analyses
Performance Goals and Thresholds (issues)
Points of
Agreement
Proposal
Input
Areas for
Discussion
State specific data on the most
recent 3 years of performance
Applied
Method
Bootstrapping
to set baseline
Estimate grand
mean and
standard
deviation
Applied Method
Min. & Max. Goals &
Thresholds
Min. = Top of bottom
5th
Max. = Bottom of top
5thth
Use Companion
Measures
Out Put
Improvement Factor
Grand Mean/Grand
Mean + 2SD
Goal Calculation
Baseline * Improvement
Factor
Threshold Calculation
Baseline – 2SD
Output
Measurement at 3
& 5 years for some
measures and 4 &
5 years for one
measurement
Bootstrapping
is reasonable
Recommendation to incorporate
trend consideration
Recommendation to adjust
calculations to 1 SD
Companion
Measures are
reasonable
Measurement
timeframes are
reasonable
Recommendation to set at 5th
and 45th State performance leve
Case Review Standards (issues)
Proposal
Case Review determines
substantial conformity for
24 items that assess casespecific practices in both inhome and foster care
cases.
A single standard is applied
to all Case Review items.
Points of
Agreement
Case Review is an
important component of
a CQI system.
A single standard should
be applied.
Areas for
Discussion
The Panel suggested that Case
Review should not be used to
determine Substantial Conformity.
How would Case Review results
be used to meet legislative
objectives to measure State
performance and improvement?
What should the standard be?
Case Review Standards
• In prior rounds onsite case reviews were conducted
during a one-week visit by external reviewers.
• In round 3, qualifying states may conduct their own
reviews using a revised CFSR onsite review instrument.
States have to meet three criteria in order to use their
own case review process and be able to meet the
following time lines.
– Must use a sample period of 4/1 to 9/30 of year prior to
review year.
– Conduct reviews from 4/1 to 9/30 of the year of review and
provide information to the Children’s Bureau
– Report results no later than 11/15 of the year of review to
the Children’s Bureau
Case Review Standards
• Criterion 1 - The states operates an internal case
review process at least annually that assesses
statewide practice performance for the key child
welfare areas using a uniform sampling process and
methodology
• Criterion 2 - The state has a process in place for
ensuring accurate and consistent case review ratings.
• Criterion 3 - The state uses the federal onsite review
instrument and its instructions using the sample and
method established above to collect data to be used
for the initial determination of conformity.
Sampling (issues)
Proposal
Points of
Agreement
Sample to allow for
statistical inferences
Combined foster care and
in-home case sample
Simple random sampling
design
Statewide with options for
geographic representation
Option to reduce sample
size for ongoing program
improvement
No support for a
sampling approach that
could eliminate children
in rural areas
No support for reducing
sample size for ongoing
program improvement
Areas for
Discussion
Should sample size be increased to
allow for statistical inferences for
foster care and in-home cases?
Is the proposed sample size feasible?
Is the proposed random sampling
design feasible?
What are alternative sampling
strategies?
Panel suggestion that targeted
reviews could have a role in ongoing
improvement monitoring.
Sampling
• Random selection; or a stratified selection representative
of the states demographics and includes the largest
metropolitan area.
• Minimum of 65 cases served during the sample period (40
foster care cases; 25 in-home cases, including Alternative
Response);
• Out of home cases must be consistent with the listings
reported to AFCARS during the period.
• In home cases (including AR) must have been opened for at
least 45 consecutive days during the sampling period .
• The state must be able to consistently address document
case elimination.
Standards for Systemic Measures (issues)
Proposal
The State’s CQI system
would identify sources for
quantifiable data for items
TR1 – TR4 and FAP 2.
A single standard is applied
to all systemic capacity items.
Points of
Agreement
The prior method of
assessing systemic
measures was subjective.
The evidence connecting
the systemic factors to
family outcomes is limited.
Areas for
Discussion
What is the recommended
standard and why?
90%
75%
Should the standard be
different for systemic capacity
items assessed through case
review?
Implications
1. The measures are simpler which will help in explaining
what they mean to all the various audiences - this should
make it clearer to develop strategies to influence
outcomes.
2. Opportunities for SSWs to increase their role regarding
development and implementation of CQI processes
3. Opportunities to work with state partners on the
importance of understanding the trajectory of children
into and through the service systems
4. Greater focus on CQI in social work education
5. Critiquing CFSR 3 or Designing CFSR 4 could become a
useful activity for a research or CWS course