Transcript Document

521: The Quality Service Review Process
Day 1 - Agenda
•
•
•
•
•
•
Training Overview
Overview of the QSR Process
Overview of the Indicators
Roll-Up Sheet
The QSR Interviews
Child/Youth and Family Status Indicators
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
2
Day 2 – Agenda
•
•
•
•
•
Practice Performance Indicators
Feedback Session with the Caseworker/Supervisor
Case-Specific Team Debriefing
The QSR Written Case Review Summary
Closing and Evaluation
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
3
Learning Objectives
•
•
•
•
Identify the components of the QSR Process;
Recognize the QSR Indicators;
Apply the interview techniques relative to the protocol;
Assign reliable status and performance indicator
scores;
• Demonstrate a Caseworker/Supervisor Feedback
Session;
• Identify the steps to prepare for the Case-Specific
Debriefing; and
• Recognize the components of the QSR Written Case
Review Summary.
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
4
QSR’s Connection to CQI
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
5
DAPIMTM and “Learning by Doing”
Define
Work products – baseline surveys,
alignment notes, initial feelings, ground
rules, defined areas for improvements,
desired future state
Assess
Monitor
Work products – monitoring
quick wins and other changes,
evaluations and measures of
progress and impact
Work products – findings:
strengths and gaps, root causes
and general remedies
Performance
& Capacity
Implement
Plan
Work products – team activities,
action plans, charters for working
teams, communication efforts
Work products- quick wins, mid- and longterm improvement plans, communication
and capacity plans
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
6
Purposes of the QSR Process
• To appraise the current status of the focus child/youth
and family and the performance of key practice areas;
• To assure that the actions of a team of child welfare
professionals are guided by clear and comprehensive
policies and are rooted in Pennsylvania’s Practice
Model/Standards;
• To improve the quality and usefulness of information
needed to evaluate the safety, permanence and wellbeing of children and youth;
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
7
Purposes of the QSR Process (continued)
• To assure the quality and effectiveness of services for
children, youth and families by regularly reviewing
programs and practices;
• To support the efforts of child welfare agencies to
maintain an environment that encourages learning and
program improvement; and
• To stimulate actions and next steps, to build on strengths
and support efforts to improve outcomes.
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
8
What is the QSR?
The QSR is designed to conduct an in depth review of front line practice
in specific locations and points in time.
It is used for:
• Appraising the current status of a child/youth in key life areas;
• Appraising the current status of the parent/caregiver; and
• Appraising the performance of key practices for the same
child/youth/family.
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
9
How is the QSR Different From Other Quality
Assurance Methods?
Traditional Review
Quality Service Review
Were visits timely?
Are children currently
safe?
Is the child’s IEP in
the file?
Is the child progressing well
academically?
Was there a referral
to mental health?
Is the child functioning well
emotionally?
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
10
How is the QSR Different From Other Quality
Assurance Methods? (continued)
Traditional Review
Quality Service Review
Is there a current plan in the file?
Is the plan likely to lead to
permanence?
Did the parents sign the plan?
Do the parents feel that the plan is
their own?
Is there a written
assessment?
Does the assessment accurately
identify underlying needs?
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
11
Skills and Abilities
Discuss with your partner the following
questions:
• Which three challenges or needs are most important
for you to overcome in order to become a successful
QSR Reviewer?
• On a scale of 1-10, with one being a minimal challenge
or need and 10 being a major challenge or need, how
would you rate your three identified challenges or
needs?
• What will it take for you to decrease your rating 1-2
points?
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
12
The QSR Process
1. Review Team Pairings
2. Sample Selection
3. Focus Groups and Stakeholder Interviews
4. Case Reviews
5. Feedback Session with the Caseworker/Supervisor
6. Case-Specific Debriefing
7. QSR Written Case Review Summary
8. Exit Conference (optional for reviewers)
9. Final Report
10. Next Steps Meeting
11. County Improvement Plan
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
13
What Happens Prior to the Onsite Review?
• Identification and pairing of internal and external
reviewers;
• Random sample of a county’s cases are selected and
stratified by a service area;
• Family permission obtained to participate in the
review and consents/releases obtained; and
• Key interviews in each case are identified and
appointments are scheduled.
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
14
Focus Groups and Stakeholder Interviews
Purpose:
– Obtain feedback specific to systemic issues
rather than case-specific information; and
– Provide a forum for open dialogue about
challenges and solutions within the system.
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
15
Focus Groups and Stakeholder Interviews
Invited to…
Focus Groups:
• Caseworkers, Supervisors, Managers, Service
Providers, and Resource Parents
Stakeholder Interviews:
• Judges, Attorneys, Police, Probation Officers, County
Commissioners, etc.
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
16
Exit Conference – What Happens?
• A few reviewers will be asked to present the
highlights from their case review.
• The Local and State Site Leads deliver the
aggregate findings from all the cases reviewed
along with trends discovered from the focus
groups and stakeholder interviews.
• Key strengths and areas needing improvement
are identified.
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
17
Final Report – What Happens?
• Final Report which includes analysis of demographic
information, quantitative analysis of indicator scores,
and information from the County Specific Data Packet
is shared.
• The county will use the data and analysis to develop a
County Improvement Plan.
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
18
Next Steps Meeting – What Happens?
• Approximately 6-8 weeks after the onsite review;
• Led by the Local and State Site Leads;
• Lasts approximately two hours;
• Discuss the recommendations for next steps in
working toward an action plan for improving frontline
practice and system performance; and
• Begins the next step in the CQI process.
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
19
What is Your Role?
Each pairing of reviewers will:
• Conduct case reviews:
 Case file review - overview of the case record;
 Interviews - meet with key players involved in the case;
 Roll-Up Sheet - demographic information and scoring;
• Provide feedback to the caseworker (and supervisor) - immediate
feedback including suggestions and potential next steps for the case;
• Present information gathered from case review and feedback on the
family story at the Case-Specific Team Debriefing;
• Complete a QSR Written Case Review Summary; and
• Reviewers may be invited to attend the Exit Conference.
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
20
Child/Youth and Family Status Indicators
•
•
•
•
•
•
•
•
•
•
•
Safety: Exposure to Threats of Harm;
Safety: Risk to Self/Others;
Stability;
Living Arrangement;
Permanency;
Physical Health;
Emotional Well-Being;
Early Learning and Development;
Academic Status;
Pathway to Independence; and
Parent or Caregiver Functioning.
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
21
Practice Performance Indicators
•
•
•
•
•
•
•
•
•
•
•
•
Engagement Efforts;
Role and Voice;
Teaming;
Cultural Awareness and Responsiveness;
Assessment and Understanding;
Long-Term View;
Child/Youth and Family Planning Process;
Planning for Transitions and Life Adjustments;
Efforts to Timely Permanence;
Intervention Adequacy and Resource Availability;
Maintaining Family Relationships; and
Tracking and Adjusting.
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
22
What Time Period is Considered in Scoring?
• Child/Youth and Family Status Indicators look at
the past 30 days
– Exceptions are: Stability and Academic Status
• Practice Performance Status Indicators look at
the past 90 days
– Exceptions are: Planning for Transitions and Life Adjustments
and Efforts to Timely Permanence

NOTE: Recent performance improvements have some
influence on scores
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
23
How are Cases Scored?
Each indicator is scored on a scale of 1 – 6
• 6 – Optimal
• 5 – Substantial
Acceptable Range
• 4 – Fair
• 3 – Marginal
• 2 – Poor
Unacceptable Range
• 1 – Adverse
Maintenance Zone: 6-5
Refinement Zone: 4-3
Improvement Zone: 2-1
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
24
Distinction Between 3 and 4
The Groundhog Rule
If this case were frozen in time as it is
today, would performance or status be
considered acceptable or unacceptable?
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
25
The Central Scoring Rules
Rules Include:
•
Concentrate on the Focus Measure and Core Concepts in each indicator.
•
Read each indicator’s instructions:
– Some indicators score results, others score effort or intent.
– Not Applicable rating criteria are identified as appropriate.
– Score each sub-category within each indicator, as appropriate.
•
Stay within the time-based observation windows associated with each
indicator.
•
Rate indicators based on events that have occurred or conditions that were
present within the time-based observation window.
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
26
Roll-Up Sheet Purpose
Purpose
• Gather Data and Monitor Practice Model
Implementation:
– Demographic information
– Case information
– Indicator scores
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
27
Roll-Up Sheet Completion
• Basic case information is filled out by the caseworker
prior to onsite review.
• Remaining information is gathered through case file
review and interviews.
• Reviewers will submit a validated Roll-Up Sheet
electronically to Site Leads on a flash drive on the final
day of onsite review.
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
28
Roll-Up Sheet Components
1.
2.
3.
4.
5.
6.
7.
General Review Information
Child/Youth’s Demographic Information
Child/Youth’s Current Placement
Child/Youth’s Educational Information
Case Information
Child/Youth and Family Status Domain
Practice Performance Status Domain
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
29
Planning QSR Interviews
• Know the case history;
• Plan relevant questions for each respondent;
• Have accurate directions and be punctual; and
• Organize the interview to help the respondent
participate.
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
30
Prior to the Interview
• Review the interview schedule and begin preparation
by identifying potential questions;
• Identify the lead interviewer and notetaker for each
interview;
• Remember that you are a reviewer and not a
caseworker during the QSR process; and
• Learn something about the interests of the child/youth
to better engage him/her.
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
31
During the Interview
•
•
•
•
•
Take notes to document the interview;
Take every opportunity to recognize family strengths;
Listen more than talk;
Use the list of QSR questions to keep you on track;
When needed, ask questions on the Roll-Up Sheet;
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
32
During the Interview (continued)
• Always ask about the child’s safety (if safety
threats are identified, call site lead immediately);
• Be patient with family stories;
• Gather information, do not identify solutions to
the case’s or family’s concerns;
• Develop the relationship before inquiring about
sensitive family issues; and
• Avoid leading questions.
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
33
After the Interview
• Do not share information gathered during one
interview to the next interview;
• Resist reaching conclusions prior to the completion of
all interviews;
• Contact your site lead or next interview if you are
running a few minutes behind;
• Note areas needing confirmation or follow-up; and
• Contact your site lead if interviews lead to other key
people in the child/youth’s life that you feel would be
important to talk to regarding the case review.
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
34
The QSR Interview – Exploring Questions for
Providers










For clinicians - What are your goals for this child/youth/family?
What are the child/youth/family’s goals?
What are the strengths of this child/youth/family?
What worries you most about this case?
Tell me what you are working on now.
Who do you see as members of this family’s team?
What’s the next big hurdle for this child/youth/family?
What will success look like?
On a scale of 1-10, how likely do you think success will be
achieved? Ask follow-up questions.
Where do you see this case in six months?
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
35
Interviewing Practice – Sara’s Case
• Both reviewers read the handout, Facts about the Fisher
Family.
• Reviewer playing the role of the mother reads the
handout, Interview with Cynthia Fisher.
• Reviewer playing the role of reviewer considers, What
do you want to learn in the interview with Sara’s
mother?
• Conduct the interview.
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
36
Child/Youth and Family Indicators
• With your partner, review Handouts #6-10.
• Summarize the indicator, score the indicator and identify two
good questions for each indicator you are assigned.
Group 1: 5 and 1a
Group 2: 2 and 1b
Group 3: 4 and 5
Group 4: 3 and 6
Group 5: 7b and 4
Group 6: 6 and 9
• Be prepared to join other teams in a group discussion about
the scoring experience.
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
37
Child/Youth and Family Indicators
• With your partner, review Handouts #6-10.
• Summarize the indicator, score the indicator and identify two
good questions for each indicator you are assigned.
Group 1: 1a and 9
Group 2: 2 and 1b
Group 3: 4 and 5
Group 4: 3 and 6
Group 5: 7b and 4
Be prepared to join other teams in a group discussion about the
scoring experience.
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
38
Practice Performance Indicators
• With your partner, review Handouts #6-10.
• Summarize the indicator, score the indicator and identify two
good questions for each indicator you are assigned.
Group 1: 1a and 5
Group 2: 9 and 11
Group 3: 2 and 10
Group 4: 3 and 7
Group 5: 4 and 8
Group 6: 1b and 6
• Be prepared to join other teams in a group discussion about
the scoring experience.
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
39
Practice Performance Indicators
• With your partner, review Handouts #6-10.
• Summarize the indicator, score the indicator and identify two
good questions for each indicator you are assigned.
Group 1: 1a and 5
Group 2: 6, 9, and 11
Group 3: 2 and 10
Group 4: 1b, 3, and 7
Group 5: 4 and 8
• Be prepared to join other teams in a group discussion about
the scoring experience.
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
40
Caseworker/Supervisor Feedback Session
•
•
•
•
•
•
•
•
•
•
Thank the caseworker and supervisor for their time.
Review the QSR purpose.
Outline the strengths of the case.
Remind caseworker of the high QSR standards.
Very briefly summarize family story – ensure facts are correct.
Share new information, if applicable.
Ask caseworker for ideas for next steps.
Discuss the strengths and challenges of the indicators.
Share reviewer ideas for next steps.
Thank the caseworker/supervisor and emphasize that they had
a meaningful role in making a difference for children, their
families and the system.
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
41
Feedback Session Practice
• Please complete Handout #13, Notes for Providing Feedback
to Caseworkers.
• When completed, one partner will be the caseworker; the other
partner, the reviewer.
• Practice a feedback session providing the caseworker feedback on
Sara’s case.
• Compare your notes with your partner from Handout #13, did you
identify similar:
– Strengths/Accomplishments;
– Practice Challenges/Limiting Conditions;
– Opportunities for Advancing Practice/ Improving Local
Conditions (Next Steps).
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
42
Case-Specific Team Debriefing
• Inter-rater reliability check for consistency between
case description and scores.
• Identification of themes for feedback to the county and
written report.
• Preliminary communication about review findings to
leadership.
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
43
Case-Specific Team Debriefing Outline
Case Presentation Outline – 15 minutes
• Case Summary (2 minutes)
• Child/Youth and Family Status Indicators – Strengths
and Challenges (4 minutes)
• Practice Performance Indicators – Strengths and
Challenges (4 minutes)
• Next Steps (2 minutes)
• Group Questions/Feedback (3 minutes)
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
44
The QSR Written Case Review Summary Purpose
• Explain the “story” behind the data;
• Provide feedback to the caseworker and local
leadership; and
• Be utilized as an effective training tool within the
county agency.
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
45
The QSR Written Case Review SummaryAdditional Guidelines
• Should not tell the case story/history; but rather include information
you gathered to support the ratings of the indicators;
• Written about the ratings and why the factors contributing to the
indicators are either favorable or unfavorable;
• Practical and helpful recommendations; and
• Make a note of your due date:
 Three day Review – Due prior to the conclusion of the 3rd day
 Two day Review – Due approximately 7-10 days following the
onsite QSR
NOTE: There will be an editing process done by the Site Lead Team,
so you may be asked to edit your Written Case Review Summary
and/or provide more supportive information.
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
46
CQI Continues…
Remember QSR is NOT a one-time event. It’s part of the
bigger CQI process that leads to improved outcomes of
Safety, Permanence and Well-Being for children, youth
and families.
The Pennsylvania Child Welfare Resource Center
521: The Quality Service Review Process
47