CMMI- Information Technology and Infrastructure Systems

Grant # CMMI-0726789 : A Comprehensive Framework for the Efficiency Measurement of Road Maintenance

PI : Jesús M. de la Garza ([email protected]), Virginia Tech Co-PI : Konstantinos Triantis ([email protected]), Virginia Tech SP : Mehmet E. Ozbek ([email protected]), Virginia Tech 1) Background

 1988: A survey performed on about 10% of all USA infrastructure by the National Council on Public Works Improvement revealed that the nation’s roads were in

better than fair

condition (Mirza 2006).  In 1998 2001 2003 Similar surveys by American Society of Civil Engineers revealed that the nation’s roads were in

poor

condition (Mirza 2006). 2005  The Federal Highway Administration endorsed “

asset management

” to be the future approach of road maintenance for all state departments of transportation (DOTs) (JLARC 2002).  Allocating

resources

to preserve, operate, and manage the nation’s transportation infrastructure.  Calls for the utilization of management, engineering, and economic principles to help state departments of transportation (state DOTs) in making decisions as to how

resources

should be allocated.  Requires, as an integral part,

performance measurement

.

(Geiger 2005)

2) Significance of the Problem and the Proposed Research Current Road Maintenance Performance Measurement Systems

 Solely focus on “

effectiveness

” measures, e.g., level-of-service.

 Disregard the “

efficiency

” concept, e.g., the amount of resources utilized to achieve such level-of-service.

 Do not investigate the effect of the

environmental factors

, e.g., climate and location.

 Do not investigate the effect of the

operational factors

, e.g., traffic, load, design-construction adequacy.

Not knowing how “efficient” state DOTs are in being “effective” can lead to

excessive

and

unrealistic maintenance budget expectations

. For the cases in which

comparative analyses

are made, disregarding such external and uncontrollable factors and using pure effectiveness results may lead to unfair comparisons.



Currently

: State DOTs are implementing a variety of performance measurement systems focusing mainly on the effectiveness of their road maintenance processes. Nonetheless, state DOTs need to and in fact seek to measure not only the effectiveness of their road maintenance processes but also the efficiency of- and value added through- such processes (TRB 2006).

1

4) Literature Review

Efficiency=Q=

Output Input

(Cooper et al. 1999)

When there are multiple inputs/outputs Data Envelopment Analysis (DEA)

Partial Efficiency Measure Approach

: Has a potential to result in serious misunderstandings about the overall efficiency of a process when only a single partial efficiency ratio is used (Craig and Harris 1973).

Total Factor Efficiency Measure Approach

: May result in subjectivity as the decision-maker prescribes weights to be assigned to each input and output variable (Cooper et al. 1999).

System Dynamics

: Requires the definition of the of mathematical relationships between key variables (Chasey et al. 1997).

Regression Analysis

: Compares the efficiency of units against a hypothetical average performance (Sexton 1986).

DEA can simultaneously deal with multiple outputs and multiple inputs. DEA does not require the specification of a priori weights for the variables. DEA is non-parametric. DEA focuses on the best-practice frontiers. 5 E 4 B 3

x 2 /y

2 A D B' C F 1 0 0 lOB'l lOBl 1 2 3 4 5

x 1 /y

6 7 8 (Charnes et al. 1994, Rouse 1997, Ramanathan 2003) 9 10 11

4

The findings of the research outlined herein will contribute new knowledge to the asset management field in the road maintenance domain by providing a

framework

that is able to

differentiate

effective and efficient maintenance strategies from effective and inefficient ones; as such, the impact of such framework will be broad, significant, and relevant to all transportation agencies.

2

5) Point of Departure

1 Develop the comprehensive list of input-output variables and uncontrollable factors 2 Decide on the size of the DMU 3 Address the issue of uncontrollable factors 4 Refine the comprehensive list of input-output variables and uncontrollable factors 5 Prepare the data to be used in the DEA models Perform data mining 6 Choose the type of DEA models to be run Clean the data Allocate the data to the DMUs Perform data conversion and data rearrangement 7 Run the DEA models and obtain the efficiency score, targets, and peer(s) for each DMU; and the overall efficient frontiers 8 Derive overall conclusions (such as the reasons of inefficiency, benchmarks, best practices) that would help the decision-making process 9 Identify the effects of uncontrollable factors (i.e., environmental and operational factors) on the efficiency of the units

(Ozbek 2007)

5

3) Purpose, Objectives, and Hypothesis

The

purpose

 of this research is to develop and implement a framework that can: Measure the overall efficiency of road maintenance operations.  Consider the effects of external and uncontrollable factors on such efficiency.

The specific

objectives

of this research are, through the use of real data, to identify:

1)

The relative efficiency of different units in performing road maintenance services.

2) 3) 4)

The reasons of the efficiency differences between units.

The effects of the external and uncontrollable factors on the efficiency of units.

The benchmarks (peers) and best practices that pertain to the inefficient units.

5)

The fundamental relationships between the maintenance levels of service and the budget requirements. The

hypotheses 1)

of this research are as follows: State DOTs are more efficient when they utilize the performance-based approach instead of the traditional method-based approach for the maintenance of the roads.

2)

A significant portion of the observed efficiency differences between different units can be attributed to the effects of the environmental factors (e.g., climate, location, etc.) and operational factors (e.g., traffic, load, design-construction adequacy, etc.) faced by such units.

3)

A unit that achieves the best road maintenance level-of-service, i.e., the most effective one, does not necessarily have to be the one that utilizes the most resources to achieve such level-of-service.

3

6) Methodology

The methodology calls for the development of

modules

to accommodate

different scenarios

that relate to: (i) the different units of comparison (ii) availability of data in different degrees The specific

activities

that need to be undertaken as a part of the methodology are:

Activity 1

: Explore the approaches that can be used to deal with the uncontrollable factors. The main approaches to be investigated are: (i) single-stage approaches (ii) multi-stage approaches (e.g., bootstrapping, clustering, and regression)

Activity 2

: Explore the approaches that can be used to refine the comprehensive sets of controllable variables and uncontrollable factors. The main approaches to be investigated are: (i) judgmental approaches (e.g., analytical hierarchy process) (ii) quantitative approaches (e.g., principal component analysis) (iii) DEA-based approaches (e.g., DEA-based sensitivity analysis)

Activity 3

: Explore the types of the DEA models to be run. The DEA models to be investigated are: (i) input-oriented CCR model (ii) output-oriented CCR model (iii) input-oriented BCC model (iv) output-oriented BCC model (v) network DEA model (vi) imprecise DEA model (vii) dynamic DEA model (viii) fuzzy DEA model

6