Transcript “Assessment of the Impact of Decentralization: The Case of Colombia (1991-2001)”

Decentralization Thematic Group/Public Sector
Group Funding 2002-2003
“Assessment of the Impact
of Decentralization:
The Case of Colombia (1991-2001)”
Presentation by:
Jonas Frank (LCSPS)
March 10, 2004
The Puzzle:
What is the impact of decentralization and
can this impact be evaluated?

Better Services?

Economic Efficiency?

Democratic Government?
 Evaluation is about finding causality
The Colombia Case:

The Study:
 Sample
of 148 municipalities, out of a total of 1100
 70 indicators
 1000 pages
 It took ca. 2 years to complete the evaluation

Appropriate time-frame: process started in 1986,
evaluation covers years following 1991

Integral reform: political, fiscal, administrative

Typical sequence: political reform first, then fiscal
and administrative decentralization; municipal level
Three important questions:
 Why
evaluate?
 What
 How
should be evaluated?
can one evaluate?
I. The “Why” of Decentralization Evaluation

Objectives of Evaluation differ by actors of
decentralization:

Minister of Finance, Indigenous People, Legislators, Mayors,
Regional Governors, Donors,...

There is no single objective, but evaluations
can...
 (i)
 (ii)
improve decision-making
create accountability among actors involved
I. The “Why” of Decentralization Evaluation

Implications:

Evaluation is a continuous exercise

Transformation of data into information

A participatory exercise vs. purely technocratic
I. The “Why” of Decentralization Evaluation
Objective of the Colombia Study:

Crisis: fiscal pressures, weak budget constraints, bailouts

Inefficient services

Slow democratization
Lesson:

Evaluate as early as possible

Ownership

Too many objectives of evaluation
I. The “What” of Decentralization Evaluation
Some guiding questions:

Where is decentralization supposed to have an impact?

Were the initial goals of decentralization met and to
what extent?
Dilemas:

Decentralization goals remain vague

They are not expressively formulated and agreed upon

Decentralization is a “moving target”
I. The “What” of Decentralization Evaluation

Decentralization Goals in Colombia:
 In
1986: democratization
 In
1991: efficiency in services, redistribution, services
for the poor, improve popular participation
 In
2001: fiscal discipline, efficiency
LAC: Decentralization Objectives (1970 – 2002)
1970-1975
1975-1980
1980-1985
1985-1990
1990-1995
1995-2000
2000-2002
Venezuela
Political Dec.
Fiscal Dec.
Administrative Dec.
Social
Context
Rio
Grande
II. The “What” of Decentralization Evaluation
Colombia Study: “Comprehensive evaluation” (not only
fiscal):
1. Outcomes within eight sectors

Education

Health

Water and basic sanitation

Fiscal performance

Political decentralization

Management capability

Economic development
2. Municipal Progress: (i) sectoral outcome (ii) aggregate
outcome,
II. The “What” of Decentralization Evaluation
1. Sectoral Evaluation: Results in Education

“Decentralization has improved and contributed to:

Increase in coverage

Improved teacher/student ratio

Higher schooling levels

Reduction in illiteracy rate”
Results in Education
Percentage of illiterate population aged 15 years and above
30.0
28.0
20
00
(%
)
26.0
24.0
22.0
20.0
18.0
Brazil
16.0
Bolivia
14.0
12.0
Peru
Mexico
10.0
8.0
Paraguay
6.0
Venezuela
Argentina
4.0
2.0
Ecuador
Colombia
Chile
Uruguay
0.0
0.0
2.0
4.0
6.0
8.0
10.0
12.0
14.0
1985 (%)
Source:
ECLAC,
Social development database.
16.0
18.0
20.0
22.0
24.0
26.0
28.0
30.0
Sectoral Evaluation: Results in Education
II. The “What” of Decentralization Evaluation
1. Sectoral Evaluation: Results in Health

“Decentralization has improved and contributed to:

Increased coverage

Higher public spending

Greater equity

Lower infant mortality rates

Greater ratio of physicians per inhabitant”.
Sectoral Evaluation: Results in Health
Infant mortality by 5-year periods
Ratio of Physicians per inh.
120
3,000
110
2,700
100
2,400
90
2,100
80
Chile
19
95 70
- 60
20 50
00
Bolivia
Paraguay
40
Ecuador
Peru
Brazil
20
10
Bolivia
19
98 1,500
Colombia
1,200
Paraguay
Brazil
900
Mexic
Colombia
o
Argentin Venezuela
a
Uruguay
Chile
30
1,800
Ecuador
600
300
Uruguay
0
0
0
10
20
30
40
50
60
1980-1985
Source:
, social development database.
ECLAC
70
80
90
100 110 120
0
300
600
900 1,200 1,500 1,800 2,100 2,400 2,700 3,000
1985
Source:ECLAC, social development database.
II. The “What” of Decentralization Evaluation
2. Performance Evaluation of Municipalities

(i) Municipal Performance in Six Sectors (Health,
Education,...)

(ii) Aggregate Performance: Most municipalities have achieved
an average performance; only 4% achieved and acceptable standard

But:

What were the starting conditions?

Where there several observations in the past and when?
00
Results in Economic Development
Fiscal Performance vs. Local Economic Development
Economic
Development
60
50
40
30
20
10
0
10
18
26
34
42
50
Fiscal Performance
58
66
74
II. The “What” of Decentralization Evaluation
Lessons:


Some important areas are excluded:

Allocative efficiency

The regional level: departmental government
Conclusions about the causality of decentralization
are primarily hypothetical

“Less is more”
III. The “How” of Decentralization Evaluation

1. Base-line of evaluation

2. Finding causality

3. Indicators

4. Selection of samples of local government

5. Periodicity of evaluation
III. The “How” of Decentralization Evaluation
1. Base-line

Usually no such base-line has been established before
decentralization was implemented

The use of proxies

Select indicators that are available and mirror the
starting conditions
III. The “How” of Decentralization Evaluation
2. Finding Causality
To date, impact studies of decentralization are either:



(i) Evidence-based
(ii) Subjective
(iii) Hypothetical
All of these three types of studies provide a useful
understanding and appraisal at various stages of the
decentralization process.
III. The “How” of Decentralization Evaluation
2. Finding Causality

The Colombia study is primarily hypothetical

Statistical analysis is not sufficient to prove causality

Lessons:

Discerning between decisions that are in hands of local government
and which are not

Clarifying the value added of local government in carrying out new
responsibilities?

Continuous monitoring
III. The “How” of Decentralization Evaluation
2. Finding Causality: Example of the Education sector

Local government defines curriculum content:


”Instruction Indicator”
Local government is responsible for construction and maintenance of
school buildings


Local government develops education improvement plan


 “Planning Indicator”
Local government allocates budget


 “Infrastructure Indicator”
 “Budget Indicator”
Is this operationally possible?
III. The “How” of Decentralization Evaluation
3. Indicators

70 indicators were used in the Colombia study

Lesson:

Selection of indicators depends on “what” will be used, but:

use only a minimum set of indicators, and...

indicators that can be monitored regularly and with low cost
III. The “How” of Decentralization Evaluation
4. Comparability of data and selection of local
government

Colombia: “representative sample” = most diverse
municipalities were selected

Lesson:

Use only most similar cases because diversity increases during
decentralization process

Split them into groups: eliminate some structural factors that lie
outside control of municipalities (population size)
III. The “How” of Decentralization Evaluation
5. Periodicity of evaluation

Colombia: evaluation after 10 years of time

Lesson:

Time period sufficiently large, but...

Regular monitoring is necessary

It is important to separate between short-term, medium-term, and
long-term goals
•  helps create accountability
•  helps better decision-making
Conclusion: Risks and Opportunities

Evaluation as a purely bureaucratic exercise

(i) Evidence and (ii) subjective evaluation are still important:
the only “early-warning system” that is operational

Ready to propose and implement corrective measures: “exit”
of decentralization process?

Giving “erroneous” answers to the “wrong” questions:
“Decentralization has not worked well enough because there
was not enough of it”