Public Performance Measurement and Management: From Theory to Practice Patricia de Lancer Julnes, Ph.D.

University of Baltimore Marc Holzer, Ph.D.

Rutgers University, Newark Robert Shick, Ph.D.

Rutgers University, Newark International Conference for Administrative Development: Towards Excellence in Public Performance Riyadh, Saudi Arabia November, 2009

Review of Performance Movement in United States

• • • • • • • Evolution of performance movement Model of performance process Performance measurement Performance-based budgeting Performance-based management Approaches to developing a performance system Challenge to implementing an effective performance system

•

Uses of Performance Measurement and Management in the Public Sector

Demonstrate accountability to citizens, elected officials, and other interested parties by reporting the services efforts and accomplishments associated with a government’s programs and activities • Improve the resource allocation process by using performance information to inform the development and enactment of a government’s budget • Assure the achievement of desired results by using performance information to monitor the delivery of services, and make adjustments, if necessary, in the service delivery

Introduction

• Century long effort to define and apply metrics to government services • Driven by management and budget initiatives, PPBS, MBO, TQM, and productivity • • • Performance measurement broadly defined as ongoing collection of data on program activities and accomplishments Indicators include: inputs, outputs, and outcomes • Indicators of efficiency, cost-effectiveness, or productivity, ratios of outputs to inputs • Outcomes divided into intermediate outcomes and end outcomes Information used for program related decision-making

Evolution of Performance Movement

• • • • • • • New York City, Bureau of Municipal Research, 1906 Brookings Institution predecessor, Institute for Government Research, whether focus of government should be efficiency, early 1900’s Taft Commission 1913, Brownlow Committee, 1 st Commission, Second Hoover Commission Hoover Nixon National Commission on Productivity, 1970 Carter Quality of Working Life 1977 Clinton National Performance Review 1993 Government Performance and Results Act 1993

Public Sector Performance Process Model

Examples

Municipal Function Input Measures Output/ Workload Measures Efficiency Measures Effectiveness measures Productivity Measures Explanatory Information

Sanitation The amount of labor-hours of the Sanitation Department, the budget of the Sanitation Department; number of vehicles Tons of refuse collected; miles of roads cleaned; Number of customers served Employee hours per ton of refuse collected; Dollars spent for one mile of snow removal Percentage of clean streets (e.g., measured by periodical visual inspection; citizen surveys) Cost per mile of a clean street (i.e., total cost of all road cleaning divided by the total miles of clean streets) Composition of solid waste; Climatic conditions; Terrain; Crew size of vehicles; Type of vehicles

PM Indicators

Resources dedicated to or consumed by a program.

• funds • staff/staff time • equipment • supplies • facilities Inputs Activities Outputs Direct products of program activities, usually measured in terms of volume or work accomplished.

• # permits issued • # brochures mailed • # residents served • # immunizations

Inputs

PM Indicators

Activities Outputs Efficiency Measures: Inputs used per unit of output

PM Indicators

Inputs Activities Efficiency Outputs Service Quality Outcomes

PM Indicators

Service Quality: Degree to which citizens are satisfied with a program, or how accurately or timely a service is provided Percent of respondents satisfied with a service - Frequency of repeat repairs - Average days to address a facility work order

PM Indicators

Outcomes - Why are we providing this service?

- Reduction in fire deaths/injuries - Increased feelings of neighborhood pride - Reduced exposure to safety hazards

A good PM system is…

• • • • • • • Results-Oriented Reliable Useful Quantitative Realistic Cost-effective Easy to Interpret • • Comparable Credible

Performance Measurement

• Holzer (1977) – demands for improved performance and government accountability have been continually present in the public sector • Ammons and King (1983) – demands for performance improvement often took a backseat to other pressing government concerns • Wholey and Hatry (1992) – explained that the value of performance monitoring rests on the information that it provides

• • •

Performance-Based Budgeting

Results budgeting or outcomes budgeting GAO (2001), Melkers and Willoughby (1997), Joyce (1993), and Grizzle and Pettijohn (2002), Smith (1999), Miller and Robbins (2004), Ngoyi, Miller, and Holzer (2007), and Barnett and Attberry (2007) Encouraging • Results budgeting has great potential to improve resource allocation decisions • Found performance planning a catalyst for setting goals, objectives, and outcome measures • Performance budgeting has promise to be a truly revolutionary movement • Performance budgets increasingly required by state legislation and include: strategic plan, annual performance plan, annual program performance report, and program evaluation • Budgeting for outcomes hold exceptional promise, and cite steps in budget process including setting measures of annual progress, monitoring, and closing the feedback loop

Performance-Based Budgeting Concerns

• Performance budgeting may be more vulnerable to fraud, falsification, and misrepresentation and shares problems of strategic planning of oversimplification and parochialism, but concedes it has greater philosophical soundness than other budget systems • Performance budgeting should be approached cautiously, difficult to agree on objectives and most successful projects focused on activities not results

• • •

Performance-Based Management

The purposeful use of resources and information to achieve and demonstrate measurable progress toward agency and program goals Wholey (1999), Berry and Wechsler (1995), Roberts (2002), Romzek and Dubnick (2008), Franklin (2000), and Ammons and King (1983) Encouraging • Viewed as a way to respond to citizen demands for accountability and productivity • Report 80% of government manager respondents thought strategic planning important for setting goals, guiding policy, and making budget decisions • Managers seem to recognize importance of planning and outcomes, but questions what it will take for performance based management with the required planning and measurement to take hold in public organizations

Performance-Based Management Concerns

• New wave of accountability will fail to achieve intended results as current reform neglects agency and personal responsibility and accountability and is deemed as a process of dialogue • Report of chief local administrators show respondents understand the meaning of performance measurement and management, the top priority concerns they identified were fiscal crisis, capital improvements, and economic development

Performance Used in Federal, State, and Local

Government

• • • • • Kravchuk and Shack (1996), Berman and Wang (2000), Poister and Streib (1999), and de Lancer Julnes and Holzer (2001) Federal level - cautions that formal approaches like GPRA can come at the expense of rational hands-on management and performance measures may misinform if users are not aware of subtle limitations of performance systems County level - effectiveness of performance measurement contingent on underlying organizational, political, and technical capacities Local level - growing impetus for performance measurement, but rhetoric has outpaced reality, and limited by the extent of commitment to performance measurement has taken hold Two stage process – adopting performance measures and implementation

Seven-Step Performance Measurement and Use Methodology

• • • • • • •

Step 1. Identification of the Programs to Measure Step 2. Statement of Purpose Step 3. Identification of Program Inputs, Outputs, Efficiency and Productivity Indicators Step 4. Setting Targets for Accomplishment Step 5. Monitoring Step 6. Performance Reporting Step 7. Analysis and Action

Step 1. Identification of the Programs to Measure

• Programs are groupings of routine activities aimed at providing support for a certain service • Example, the following four activities--street resurfacing; street patching; seal coating; and curb repairing--are activities that constitute the program that is usually called street maintenance • Clearly identify and divide activities into distinct programs

Step 2. Statement of Purpose

• One can measure the performance of a program only if he or she knows what the program should accomplish • Clearly articulate a statement of purpose for a program – a clear mission statement is a good place to start

Step 3. Identification of Program Inputs, Outputs, Efficiency and Productivity Indicators

• • • • • Input Indicators – Designed "to report the amount of resources, either financial or other (especially personnel), that have been used for a specific service or program. Input indicators are ordinarily presented in budget submissions and sometimes external management reports.“ GASB Output/Workload Indicators - Report units produced or services provided by a program. Workload measures indicate the amount of work performed or the amount of services received.

Outcome/Effectiveness Indicators - Report the results (including quality) of the service. These measures answer the question How well the organization is meeting its goals. Efficiency Indicators- Report whether the desirable ends are achieved with minimal input of resources, or that the resources we employ bring the maximum amount of desirable goal attainment. Efficiency refers to the ratio of the quantity of the service provided to the cost, in dollars or labor, required to produce the service. Productivity Indicators - Combine the dimensions of efficiency and effectiveness in a single indicator. The costs (or labor-hours) of faulty meter repairs as well as the costs of effective repairs are included in the numerator of such a calculation, but only good repairs are counted in the denominator--thereby encouraging efficiency and effectiveness of and by meter repair personnel. (Ammons, 1996)

Step 4. Setting Targets for Accomplishment

• Express the goals of a program through date (or period of time) and quantities (including percentages) • Identify service effectiveness and quality, and explicitly state how we are going to check whether our objectives have been met

Step 5. Monitoring

• Monitoring provides the user with the results needed to decide whether or not the target is accomplished or not • Each accomplishment target should be monitored on a continuous basis • Systematic monitoring gives the manager an opportunity to keep tabs on the operation of the program, and take corrective action if requested during the program

Step 6. Performance Reporting

• Performance reports should be an accessible and convenient format including the following: – Program name, organizational jurisdiction, and statement of purpose – Workloads/outputs, inputs, productivity and/or efficiency ratios – Targets for accomplishment for crucial indicators, most notably, effectiveness and efficiency – Brief synopses of additional explanatory information

Prince William County, Washington Department: Community Improvement

Step 7. Analysis and Action

• • A well-developed Performance Measurement System enables users to identify weaknesses and threats, as well as strengths and opportunities Gives users the opportunity to diagnose organizational growth capabilities and take relevant actions

First Order Components in a Performance System

• • • • • • • • • Chief Executive commitment and involvement Relevant measures of at least outputs and eventually outcomes Periodic review and revision of the performance measures Regular collection of performance data Comparison of performance data to prior periods Regular review and analysis of performance results Chief Executive and other senior management involvement in the reviews and analysis Agreement between Chief Executive and department managers on improvement plans Follow-up on achievement of improvement plans

Second Order Components in a Performance System

• • • • • • • • • Targets for performance measures Comparison of performance data to targets Comparison of performance data to similar jurisdictions Support staff involvement in the reviews and analysis of performance results Budget reviews and deliberations that consider the performance targets and results Budget resources allocated to programs rather than solely to object classes Programs for obtaining constituents’ views Existence of some process for assuring the data’s reliability Regular dissemination of performance results

Third Order Components in a Performance System

• • • • • • Strategic plan for the jurisdiction or departments Legislative branch involvement in the process External stakeholder involvement in the process Cost accounting system and cost of services Personnel evaluations, promotions, and compensation adjustments that consider program performance Involvement of non-government organizations in the process

Challenge

• Insure that the performance measurement and management system is credible, relevant, and useful to the delivery of public sector services through a continued and sustained emphasis by the Chief Executive and Department heads, and to the extent possible, institutionalize the system so it will continue in future administrations