Transcript Document
National Core Indicators Overview for the State of Maine Sarah Taub & Giusi Chiri Human Services Research Institute January 30, 2003
Purpose and goals What is NCI?
How will participation in NCI impact the Quality of Life Consumer Survey?
How will other NCI activities benefit the State of Maine?
What do the results look like?
How are states using the NCI data?
Project Beginnings NASDDDS and HSRI collaboration Launched in 1997 Seven field test states (plus steering committee) ~60 candidate performance indicators Development of data collection instruments
What has NCI Accomplished?
Nationally recognized set of performance and outcome indicators for developmental disabilities service systems Reliable data collection methods & tools Baseline and trend data national level at the state & Benchmarks of performance
NCI Structure Currently 22 states plus Orange County in Phase V (FY2003) HSRI provides technical assistance under subcontract to NASDDDS Subcommittees address specific issues Meet with full steering committee annually Funded primarily through state participation fees
Participating States Phase V
ME MT SD UT Orange County AL SC
What are the Core Indicators?
Consumer Outcomes: Employment (earnings, hours worked) Community Inclusion Choice and Decision-making (personal choices, support-related choices) Self-determination (new) Relationships Satisfaction (with home and job)
What are the Core Indicators?
System Performance Service Coordination Family and Individual Participation (on boards of directors) Utilization (types of services provided) Financial Level of Effort Cultural Competence (access to supports) Access (relative to population)
What are the Core Indicators?
Health, Welfare, and Rights Safety (mortality, injuries, crimes) Health (routine exams) Medications Wellness (new) Restraints Respect/Rights
What are the Core Indicators?
Staff Stability and Competence Staff Stability (turnover and vacancy rates, length of employment) Staff Competence (new)
What are the Core Indicators?
Family Indicators Information and Planning Choice & Control Access & Support Delivery Community Connections Family Involvement Satisfaction Family Outcomes
What are the data sources?
Consumer Survey Family Survey Adult Family Survey (at home, 18+) Family Guardian Survey (out-of-home) Children Family Survey (at home, <18) Provider Survey Staff Stability Board Representation System Data Incidents Mortality
What it means to be from… a new NCI state Transition to new survey tool Similar questions and structure Some methodological differences but still comparable Technical support re: training, administration, analysis Risk adjustment Adult Family Survey 53% response rate
What it means to be from… a new NCI state Provider and system data Common definitions MIS improvements General Participation on subcommittees Collaboration with other states, particularly New England region
National Core Indicators
Selected Results
Provider Survey Staff stability Rising trend in staff turnover over past three years Turnover is higher in residential settings vs. day settings In FY2002, turnover rates ranged from 33% (Hawaii) to 54% (Indiana)
Staff Turnover Rates FY1999 - FY2001 60% 50% 40% 30% 20% 10% 0% 35.2% 31.2% 1999 41.7% 35.8% 2000 52.9% 45.1% 2001 residential day
Provider Survey: Staff Stability Staff Turnover in Agencies Providing Residential Supports (FY2001) H I R COC W A W Y IN 0% 10% 20% 30% 32.6% 39.0% 40% 49.0% 52.2% 54.1% 50% 60%
Board Representation (FY2001) RCOC 15% 7% WA 28% 3% IN 24% 2% WY 25% 2% HI 0% 0% DE 0% 9% 5% 10% 15% 20% Average % Voting Members 25% 27% 30% % family % consumers
Adult Family Survey Surveys of families with an adult family member living at home 5567 total surveys (37% response rate) Average age of respondent = 57 89% of respondents were parents 96% were primary caregivers
Adult Family Survey (% shown = “yes” response) Family receives information about services Family helped develop service plan State staff respect family’s choices and opinions State staff are knowledgeable and effective Nat’l Avg 50.7% 72.9% 80.3% 73.1%
Adult Family Survey (% shown = “yes” response) Supports offered meet family’s needs Help was provided in a crisis situation Translators are available if necessary Staff help connect family to natural supports Nat’l Avg 57.2% 51.4% 67.7% 39.2%
Adult Family Survey: Choice & Control (% shown = “yes” response) Family chooses support workers Nat’l Avg 39.6% Family has control over hiring & management of workers Family wants control over hiring & management of workers Family knows how much $$ is spent on behalf of person 36.4% 55.9% 31.9%
Adult Family Survey - Choice & Control by Age 100 90 80 70 60 50 40 30 20 10 0 Choose agencies & providers Choose support workers Have control hiring staff Want control hiring staff Know MR/DD $ spent Decide how $ spent 18-24 25-34 35 & Over
Do you choose the support workers that work with your family?
State Yes
Some No n
OK VT NC WY IL IA HI WV CA-RCOC WA UT AZ PA NE
% of Total Avg - 14 states 68.1
65.6
54.5
52.8
42.2
35.9
34.9
34.8
33.3
32.1
30.0
28.9
26.9
13.8
36.3
39.6
16.9
18.5
16.3
24.5
15.4
22.8
15.4
15.8
12.3
18.2
15.9
14.7
18.1
9.7
16.2
16.8
14.9
15.9
29.2
22.6
42.4
41.3
49.7
49.5
54.3
49.7
54.1
56.4
55.0
76.5
47.5
43.7
248 227 319 53 396 281 149 184 567 352 414 273 785 340
Total n = 4,588 Total n = 14
Consumer Survey 7917 surveys completed 67% of people interviewed were able to respond to Section I
Level of MR
Unkown 2.5% Profound 14.9% No Label 3.7% Severe 14.5% Mild 38.2% Moderate 26.3%
Place of Residence Nursing Facility Other Apartment Program Specialized Facility Foster Care or Host Home Independent Home or Apt Group Home Parent or Relative Home 0.9
3.7
4.3
8.5
9.9
0.0
18.6
24.7
29.1
20.0
40.0
Scale Development
SCALES
construct.
are composite measures that sum together a set of question responses to determine the level of some theoretical
EXAMPLE:
The “C
ommunity Inclusion
” scale was created by adding up the answers to questions such as “Do you go shopping?”, “Do you always eat at home, or do you sometimes go out to eat?” and so forth.
Internal Consistency A scale can be considered a reliable measure if its internal consistency is
.
70. The statistic that assesses the scale’s reliability is called Cronbach’s alpha.
Comparisons Among States The
ANALYSIS of VARIANCE
is a collection of techniques used to test for differences among more than two groups. Post hoc (multiple comparison) tests provide information about which groups are different from each other. Alternatively one can compare each state against the national average.
Consumer Survey Analysis Four “scales” were created to combine sets of related items All scales had alpha > .70
Service Coordination = .80
Community Inclusion = .89
Support-Related Choices = .92
Personal Choices = .95
Consumer Survey Analysis Community Inclusion Scale Goes shopping Goes on errands or appointments Plays sports or exercises Goes out to eat Attends religious services Belongs to clubs or community organizations Goes out for entertainment
N.E. Results vs. National Average Community Inclusion Scale
Above average No difference Below average VT (.84) CT (.79) RI (.77)
Consumer Survey Analysis Support Related Choices Scale Chose job or day activity Chooses support staff at home Chooses support staff at job/day activity Chose service coordinator Chose residence
N.E. Results vs. National Average Support-Related Choices Scale
Above average No difference Below average VT (.84) CT (.79) RI (.77)
Consumer Survey Analysis Personal Choices Scale Chose roommate Chooses daily schedule Chooses what to do in free time Chooses what to buy with spending money
N.E. Results vs. National Average Personal Choices Scale
Above average No difference Below average VT (.84) CT (.79) RI (.77)
Areas of Strength 92% of all respondents report that they have enough privacy over 90% of respondents report that support staff treat them with respect 94% satisfied with home 96% satisfied with work/day program Participation in community activities is generally high, ranging from 69% to 96%
Areas for Improvement 77% of all respondents report that service coordinators get them what they need , compared with 90% in FY2001 48% of respondents reported “sometimes” or “always” feeling lonely only 52% of women had a GYN exam in the past year and 7% have never had one
What Can You Do with the Information?
Include on your web site Prepare annual reports Develop provider profiles Use with sister agencies Use in allocation decisions Use to spot red flags
How Are NCI States Using Their Data?
Pennsylvania – Independent monitoring and quality improvement South Carolina – Core component of external monitoring Wyoming – Annual reports, CMS review Massachusetts – Strategic planning South Dakota – Provider profiles North Carolina – Health indicators
For More Information Final Reports for Phase IV (FY2002) are available on HSRI’s website: www.hsri.org
Contact us with questions: [email protected]