Onsite Quarterly Meeting SIPP PIPs June 13, 2012

Presenter:

Christy Hormann, LMSW, CPHQ

Project Leader-PIP Team

Overview of Presentation

 Progression of SIPP PIPs  SFY 2012 validation results  Areas for improvement

SFY 2011 SIPP PIPs

 First year SIPPs completed the PIP process  Each SIPP required to submit one PIP  Total of 14 PIPs were submitted

SFY 2011 SIPP PIPs cont.

 One was completed through Activity VI  13 were completed through VIII  Initial scores were lower due to lack of the proper documentation

SFY 2012 SIPP PIPs

 For SFY 2012, the SIPPs were required to submit the collaborative PIP and a PIP topic of their choosing  Some of the individual SIPP topics were increasing family participation in treatment, minimizing weight gain during treatment, and reducing readmissions

SFY 2012 SIPP PIPs cont.

 There were a total of 27 PIPs submitted for validation   Six PIPs were assessed through Activity VI Four PIPs were assessed through Activity VII  Five PIPs were assessed through Activity VIII

SFY 2012 SIPP PIPs cont.

 Twelve PIPs were assessed through Activity IX  None of the PIPs were assessed for Activity X-Sustained Improvement

PIP Stages

III. OUTCOMES II. IMPLEMENTATION I. DESIGN

Study Design Stage

 Establishes methodological framework for the PIP  Includes development of study topic, question, indicators, and population (Activities I through IV)  A strong study design is necessary for the successful progression of a PIP

Study Design Stage Evaluation Elements

Activity I: Study Topic  Reflects high-volume or high-risk conditions  Is selected following collection and analysis of data

Study Design Stage Evaluation Elements

Activity I: Study Topic  Addresses a broad spectrum of care and services  Includes all eligible populations that meet the study criteria  Does not exclude members with special health care needs  Has the potential to affect member health, functional status, or satisfaction

Study Design Stage Evaluation Elements

Activity II: Study Question  States the problem to be studied in simple terms  Is answerable

Study Design Stage Evaluation Elements

Activity III: Study Indicators  Are well-defined, objective, and measurable  Are based on current, evidence-based practice guidelines, pertinent peer-reviewed literature, or consensus expert panels  Allow for the study question to be answered

Study Design Stage Evaluation Elements

Activity III: Study Indicators  Measure changes (outcomes) in health or functional status, member satisfaction, or valid process alternatives  Have available data that can be collected on each indicator

Study Design Stage Evaluation Elements

Activity III: Study Indicators  Are nationally recognized measures, such as HEDIS technical specifications, when appropriate  Includes the basis on which indicator(s) was adopted, if internally developed

Study Design Stage Evaluation Elements

Activity IV: Study Population  Is accurately and completely defined  Includes requirements for the length of a member’s enrollment in the MCO  Captures all members to whom the study question applies

SIPP Design Stage Results Partially Met

Study Stage Activity

Met Not Met

Design I. Appropriate Study Topic* II. Clearly Defined, Answerable Study Question(s) III. Clearly Defined Study Indicator(s) * IV. Correctly Identified Study Population 87% (139/160) 74% (40/54) 68% (86/127) 75% (48/64) 3% (4/160) 11% (6/54) 9% (11/127) 14% (9/64) 11% (17/160) 15% (8/54) 24% (30/127) 11% (7/64) Design Total * 77% (313/405)

* The activity or stage total may not equal 100 percent due to rounding.

7% (30/405) 15% (62/405)

Study Implementation Stage

 Includes sampling, data collection, and interventions (Activities V through VII)  During this stage, MCOs collect data, evaluate and identify barriers to performance, and development interventions targeted to improve outcomes  The implementation of effective improvement strategies is necessary to improve PIP outcomes

Study Implementation Stage Evaluation Elements

  Activity V: Sampling  Consider and specify the true or estimated frequency of occurrence  Identify the sample size Specify the confidence level Specify the acceptable margin of error

Study Implementation Stage Evaluation Elements

Activity V: Sampling  Ensure a representative sample of the eligible population  Are in accordance with generally accepted principles of research design and statistical analysis

Study Implementation Stage Evaluation Elements

Activity VI: Data Collection  The identification of data elements to be collected  The identification of specified sources of data  A defined and systematic process for collecting baseline and remeasurement data  A timeline for the collection of baseline and remeasurement data

Study Implementation Stage Evaluation Elements

Activity VI: Data Collection  Qualified staff and personnel to abstract manual data  A manual data collection tool that ensures consistent and accurate collection of data according to indicator specifications  A manual data collection tool that supports interrater reliability

Study Implementation Stage Evaluation Elements

Activity VI: Data Collection  Clear and concise written instructions for completing the manual data collection tool  An overview of the study in written instructions

Study Implementation Stage Evaluation Elements

Activity VI: Data Collection  Administrative data collection algorithms/ flow charts that show activities in the production of indicators  An estimated degree of administrative data completeness

Study Implementation Stage Evaluation Elements

Activity VII: Interventions  Related to causes/barriers identified through data analysis and quality improvement processes  System changes that are likely to induce permanent change  Revised if the original interventions are not successful  Standardized and monitored if interventions are successful

SIPP Implementation Stage Results Partially Met

Study Stage Activity

Met

Implementation V. Valid Sampling Techniques (if sampling was used) Implementation Total 86% (6/7) VI. Accurate/Complete Data Collection * VII. Appropriate Improvement Strategies 58% (159/272) 70% (45/64) 61% (210/343)

* The activity or stage total may not equal 100 percent due to rounding.

0% (0/7) 13% (36/272) 11% (7/64) 13% (43/343)

Not Met

14% (1/7) 28% (77/272) 19% (12/64) 26% (90/343)

Outcomes Stage

 The final stage of the PIP process (Activities VIII through X)  Involves data analysis and the evaluation of improvement based on the reported results and statistical testing  Sustained improvement is achieved when outcomes exhibit improvement over multiple measurements

Outcomes Stage Evaluation Elements

Activity VIII: Data Analysis  Are conducted according to the data analysis plan in the study design  Allow for the generalization of results to the study population if a sample was selected  Identify factors that threaten the internal or external validity of findings  Include an interpretation of findings

Outcomes Stage Evaluation Elements

Activity VIII: Data Analysis  Are presented in a way that provides accurate, clear, and easily understood information  Identify the initial measurement and the remeasurement of the study indicators  Identify statistical differences between the initial measurement and the remeasurement

Outcomes Stage Evaluation Elements

Activity VIII: Data Analysis  Identify factors that affect the ability to compare the initial measurement with the remeasurement  Include an interpretation of the extent to which the study was successful

Outcomes Stage Evaluation Elements

Activity IX: Real Improvement  The remeasurement methodology is the same as the baseline methodology  There is documented improvement in processes or outcomes of care  The improvement appears to be the result of planned intervention(s)  There is statistical evidence that observed improvement is true improvement

Outcomes Stage Evaluation Elements

Activity X: Sustained Improvement  Repeated measurements over comparable time periods demonstrate sustained improvement or that a decline in improvement is not statistically significant

SIPPs Outcomes Stage Results

Study Stage Activity

Met Partially Met

Outcomes VIII. Sufficient Data Analysis and Interpretation * IX. Real Improvement Achieved X. Sustained Improvement Achieved 54% (63/116) 54% (26/48) ‡ 17% (20/116) 21% (10/48) ‡ Outcomes Total * 54% (89/164)

* The activity or stage total may not equal 100 percent due to rounding.

‡

The PIPs did not progress to this phase during the review period and could not be assessed for real or sustained improvement.

18% (30/164)

Not Met

28% (33/116) 25% (12/48) ‡ 27% (45/164)

SIPP Indicator Results

    There were a total of 44 study indicators 22 were not assessed for improvement 15 demonstrated improvement Of those that demonstrated improvement, 11 demonstrated statistically significant improvement

SIPP Indicator Results

Plan A

SIPPs SFY 2012 Performance Improvement Project Outcomes for the SIPPs (N=27 PIPs) Comparison to Study Indicator Results from Prior Measurement Period Total Number of Study Indicators Declined Statistically Significant Decline Improved Statistically Significant Improvement Not Assessed Sustained Improvement 1

3 0 0 0 2 1 Plan B 3 0 0 0 0 3 Plan C 3 0 1 0 1 1 Plan D 3 0 0 2 0 1 Plan E 3 1 1 0 0 1 Plan F 2 0 0 0 0 2 Plan G 3 0 0 0 2 1 Plan H 3 0 0 0 0 3 Plan I 3 0 0 0 2 1 Plan J 3 0 1 0 1 1 Plan K 3 0 0 1 1 1 Plan L 3 1 0 1 0 1 Plan M 4 0 2 0 0 2 Plan N

Overall Totals

5

44

0

2

0

5

0

4

2

11

3

22

1 One or more study indicators demonstrated sustained improvement.

‡ The PIP(s) did not progress to this phase during the review period and/or required an additional measurement period; therefore, sustained improvement could not be assessed.

‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡

‡

Common Areas for Improvement

Activity I: Study Topic  No historical plan-specific data provided to support the selection of the study topic (Evaluation Element #2)

Common Areas for Improvement

Activity IV: Study Population  Length of enrollment required not specified (Evaluation Element #2)

Common Areas for Improvement

Activity VI: Data Collection  Timeline for data collection not provided (Evaluation Element #4)  Not all information regarding manual data collection was provided (Evaluation Elements #5-9)

Common Areas for Improvement

Activity VIII: Data Analysis  Baseline data and data analysis not reported in this year’s submission (Evaluation Elements #1-5)

Recommendations

 Use the PIP Summary Form Completion Instructions when documenting the PIP Summary Form  If you have questions, contact HSAG for technical assistance

HSAG Contacts

For any PIP questions or to request PIP technical assistance contact

: Christy Hormann [email protected]

602-801-6836 Jenny Montano jmontano @hsag.com

602-801-6851

Questions