Introduction to Measurement

Presenter Name

Bright Futures Training Intervention With Office Staff Workshop I 1

Session Objectives

    Describe importance of measurement and general principles of measurement for improvement.

Discuss measurement strategy for Bright Futures.

Gain familiarity with one Bright Futures data collection tool.

Review Plan-Do-Study-Act cycles and describe role of measurement within them.

2

Why Measure?

  “All improvement is change, but not all change is an improvement.” Need measures to know if we are reaching our Aims.

3

Key Principles of Improvement Measurement

  Focus on a few measures (usually 5-7).

Keep it simple and easy.

 Integrate measurement into daily routine.

 Consecutive sampling.

 Use existing data when possible.

 Plot data over time.

4

Measurement for Improvement

 IS   Designed to help your team and other teams learn. Like a growth curve: it’s not where you are, but where you are going.

 IS NOT   Designed for criticism or punishment.

Supposed to end (it should be sustainable).

5

Measurement for Research Versus Improvement Research

 Generate new knowledge.

  Single test.

Need to collect all possible data “just in case.”

Improvement

 Put new ideas into practice.

 Many sequential tests.

 Collect “just enough data.”

6

Bright Futures Core Measures

   Global project measures Overall results related to project Aim Based on 6 components of Bright Futures framework

7

Core Measure 1



Bright Futures component:

services prompting sheet Use of preventive 

Measure:

Percent of well-child visits of children younger than 5 years in which a preventive services prompting sheet is used

8

Core Measure 2



Bright Futures component:

Use of structured developmental assessment 

Measure:

Percent of well-child visits of children younger than 5 years in which Ages and Stages or Parents’ Evaluation of Developmental Status is used to evaluate development and behavior

9

Core Measure 3



Bright Futures component:

Evaluation of parental strengths and needs  Measure: Percent of well-child visits of children younger than 5 years in which parent strengths and needs are assessed

10

Core Measure 4

 

Bright Futures component:

recall and reminder system Development of

Measure:

Score on recall and reminder items on practice team survey

11

Core Measure 5



Bright Futures component:

Development of linkages to community resources 

Measure:

Score on community resource linkage on practice team survey

12

Core Measure 6



Bright Futures component:

Identification of children with special health care needs 

Measure:

Percent of well-child visits of children younger than 5 years in which there is documentation that provider asked about special health care needs

13

Measurement Tools

  Chart review Practice team survey (Bright Futures Office Systems Inventory)

14

Team Measurement Exercise: Completing the Bright Futures Office Systems Inventory

15

Additional Measures

 Pilot refinement measures  Volume 1: Improving Preventive Care in Your Office: Tools for Office Improvement usefulness  Volume 3: Training Materials to Improve Preventive Care in Office Practice: Bright Futures Workshop Materials usefulness  Balancing measures  Cycle time (average number of minutes in office per patient

16

Creating Your Practice’s Own Additional Measures

  Measure only the things you need to know.

Make sure they are related to your Aim and goals.

 Create a Measurement Plan for each additional measure.

17

Sources of Additional Measures

    Your team members.

Other teams.

Goals in your Prework.

Think of “balancing measures.”

18

Balancing Measures

 Don’t want to improve one part of system at the expense of others.

 Think of potential unintended consequences and parts of your system distant to the area you are trying to improve.

 Example: patient flow may be affected by changes to preventive visit content —measure cycle times.

19

Create a Measurement Plan

  Clarify the data collection goals.

Develop procedures for each additional measure.

 How is it defined?

 What data will be collected?

 On which patients?

 How will the data be collected?

 Who will collect the data?

 When?

20

For Each Additional Measure

  Carefully define numerator and denominator.

Begin tracking measures immediately and gather historical (baseline) data, if available.

 Develop run charts to display your measures each month (use paper/pencil or Excel).

21

Using Measures: Annotated Run Charts Clinician education Practice wide guidelines Reminder system 22

Measurement of Plan-Do-Study-Act Cycles

23

24

Global Project Measures Versus Plan-Do-Study-Act (PDSA) Measures

 Global Project Measures: Overall results related to the team Aim (core measures and teams’ additional balancing measures)  PDSA Measures  Quantitative data on the impact of a change  Qualitative data to help refine the change

25

Measurement for Plan-Do-Study-Act (PDSA) Cycles

 “Study” is specific to the PDSA cycle.

 Usually not one of core measures  Usually begins and ends with PDSA cycle  Often qualitative  Identify/create tools to be used for data collection —make it simple and easy.

26

Use the Plan-Do-Study-Act Cycle for

 Testing or adapting a change that relates to project goals  Implementing a change

27

Multiple Plan-Do-Study-Act Cycles Help Achieve Project Goals 28

Summary

     Measurement for improvement is simpler and more focused than research.

Measurement is necessary to assess success and track progress.

Usually need 5 to 7 measures to ensure that the system is improved.

Practices are encouraged to add a few additional measures, including balancing measures.

Practices complete data collection and team status reports monthly.

29