Transcript Actionable Governance Indicators: Progress and Prospects DEC Seminar on Measuring Governance May 1-2, 2008

Actionable Governance Indicators:
Progress and Prospects
DEC Seminar on
Measuring Governance
May 1-2, 2008
1
Purposes of broadly-defined indicators




Research on development impact of civil
liberties, corruption, rule of law, etc.
Research on determinants/correlates of
“good governance”
Tool for reformers in government
Country-level diagnosis: more sensitive to
random measurement error
2
Purposes of narrowly-defined indicators




Monitoring progress of public sector reform
programs: are reforms being implemented? (e.g.
pay de-compression)
Measuring results: are intermediate-level
outcomes improving? (e.g. number of qualified
applicants for high-skill positions increasing)
Research on impacts of specific reforms on
performance of governance sub-systems (budget
management, civil service, tax administration)
Input into “actionable” parts of CPIA criteria
3
Actionable Governance Indicators
are the “Missing Middle”
Inputs and outputs


Bank operations consistently track
project inputs and outputs
A growing number of Bank
operations and client countries have
begun to track program and policy
outputs
The Missing Middle:


Outcomes

Broad governance outcomes are
measured by Freedom House,
ICRG, etc. and by TI, WGI
aggregate indexes
Direct impacts of institutional
reform efforts on how
particular governance subsub-systems function:
institutional arrangements,
procedures, and practices
Examples include PEFA,
Doing Business, Global
Integrity, Open Budget
Index, Statistical Capacity
Index, OECD procurement
index
4
"Effective implementation, including
further development of actionable and
disaggregated indicators, will now be
critical to achieving the GAC strategy's
desired results."
Development Committee Communiqué,
Washington DC, 15 April, 2007
5
Actionability




implies greater clarity regarding the steps
governments can take to improve their score on
an indicator
is reduced to the degree extraneous factors can
affect indicator scores
in general requires that indicators have narrow
and explicit definitions
does not preclude subjective or perceptions-based
indicators (if they have explicit and narrow
definitions)
6
Specific indicators enhance:
• Actionability: clear link between
actions and indicator values
• Replicability: independent informed
observers should arrive at same
score
• Contestability: debates over
narrower concepts are more
tractable and meaningful (DB)
7
Some specific indicators may also be
• Manipulable (design reforms
narrowly for sole purpose of
improving score)
• Irrelevant (rule of law matters for
development, but some “best
practice” advice on how to improve
rule of law may be wrong)
8
examples of narrow indicators









PEFA
Open Budget Index
OECD Procurement Assessment
Doing Business
Global Integrity
Statistical Capacity Index
Human Resource Management
Enterprise surveys
Household surveys
9
PEFA
• 28 high-level indicators measuring performance
of PFM systems (3 more on donor practices)
• a performance report based on the indicators
• applied in approximately 80 countries and by 19
donors (mostly WB and EC)
• common pool of information to promote country
leadership, coordinated and aligned donor
support (“the strengthened approach”)
• about 30 assessments available at pefa.org
10
Open Budget Index
Using questionnaire with 122 items,
the OBI assesses:
 public availability of key budget
documents
 quality of information they provide
 timeliness of their dissemination
59 countries assessed in 2006 study;
goal of 80 or more in 2008
11
OECD Assessment Methodology
for Public Procurement Systems
22 pilot countries
Baseline indicators
 12 indicators
 54 sub-indicators
 Based on review of regulatory framework
and practices
Compliance and performance indicators
• Based on sample of procurement
transactions and other performance
information
Assessment report
12
Doing Business






Contract enforceability
Registering property
Paying taxes
Number of procedures
Time
Cost
13
statistical capacity indicator (DECDG)
Countries scored against specific criteria,
using information available from the World
Bank, IMF, UN, UNESCO, and WHO.
Overall score based on 3 dimensions:
 statistical practice: ability to adhere to
internationally recommended standards
and methods
 data collection: frequency of
censuses/surveys and completeness of
vital registration
 indicator availability: availability and
frequency of key socioeconomic indicators
14
Enterprise Surveys







107 countries, 75000+ firms
“unofficial payments” as share of firm
sales
time required to resolve dispute in courts
costs of providing security against crime
time spent meeting with tax officials
percentage of firm sales reported for tax
purposes
major constraints to doing business
15
Household surveys






Gallup World Poll
TI/Gallup International “Global
Corruption Barometer”
Regional “barometer” surveys
World Values Survey
“is corruption widespread”
paid bribes to police, tax officials,
etc.
16
in-country diagnostic tools








PER, CFAA, CPAR, fiscal ROSC
WBI GAC diagnostic survey
PETS (Uganda & others)
Citizen scorecards (Bangalore)
Road composition (Indonesia)
Municipal audits (Brazil)
Pharmaceuticals (Argentina)
HRM (Albania, Macedonia, Romania)
17
Global Integrity





Assesses existence of laws and institutions
designed to curb corruption, and their
implementation
Covers most UNCaC provisions
Assessment by lead researcher in country;
two peer reviewers per country
Six broad categories, 23 sub-categories,
81 questions and numerous sub-questions
Assessments in 2007 for 55 countries;
43 countries in 2006
18
Egypt 2007
strengths
•procurement
•political financing
weaknesses
•public access to
information
•election integrity
•accountability of
public officials
•external audit
19
Hierarchy of Global Integrity Index
Overall index
Government Accountability (1 of 6 major categories)
Executive Accountability (1 of 23 total sub-categories)
Regulations governing conflict of interest by executive branch
(1 of 81 questions)
Are heads of state/government required
to file regular asset disclosure forms? Can public access
them?
20
not all categories are highly inter-correlated
100
80
60
40
20
0
Rsq = 0.0806
0
20
40
60
80
100
21
Procurement rules
implementation not as easy as passing laws
100
90
80
70
60
50
40
30
20
10
0
0
10
20
30
"in law" sub-index
40
50
60
70
80
90
100
22
enforcement: a function of income per capita
60
50
40
30
20
10
0
Rsq = 0.1783
0
10000
5000
20000
15000
30000
25000
40000
35000
45000
23
GDP per capita
How can ratings on broad
indicators be improved?






little understanding of what actions most
effective in reduce incidence of corruption
actionable indicators may improve this
understanding
criteria in broad expert ratings often not
detailed or even available
unclear how different aspects (e.g. grand
v. petty corruption) are weighted
unclear what information sources are used
by experts
ratings may lag actions or events
24
ICRG corruption ratings for Ireland
5.5
5.0
4.5
4.0
3.5
3.0
2.5
2.0
07
20
T
C 06
O 20 5
L 00
JU 2 4
R 0
AP 20 2
N 00
JA T 2 1
C 0
O 20 0
L 0
J U 20 9
R 9
AP 19 7
N 9
J A 19
T 6
C 9
O 19 5
L 99
JU 1 4
R 9
AP 19 2
N 99
JA T 1
C 91
O 19
L 990
JU 1 9
R 8
AP 19 7
N 8
J A 19
T 6
C 8
O 19 5
L 8
J U 19 4
R 8
AP 19
N
JA
1.5
25
date
timeline
Dec. 1996: story breaks
5 to 4
Oct. 1997: tribunal convenes 4 to 3
Jan. 1999: hears 1st witnesses 3 to 2
Feb. 1992: Haughey resigns as PM
Nov. 1992: retires from politics
26
some hints from the data



Examine correlations of 23 anti-corruption
policy areas from Global Integrity with
outcomes (TI, WEF) for up to 66 countries
High: conflict of interest & asset disclosure
rules for executive branch, public access
to information
Low: conflict of interest & asset disclosure
rules for judiciary, A-C laws & agencies,
ombudsman
27
TI weakly related to GI “in law”
9
CAN
8
JPN
FRAUSA
ESP
7
ISR
6
JOR
ITA
ZAF
5
LVA
CRI
BGR
COL
4 LBN
CPI
THA
EGY SENGHA
BRA
CHN MDA
MEX
PER
IND
DZA LKASRB
ROM
TZAUKR
ARM
BIH
ARG
MOZ
GEO
3 VNM YEM
VUT MWI UGA
NIC GTMNPL
TMP
KAZ
LBRZWE
PHL
BDI CMR BEN
IDN ETH RUS
AZE
ECU
NGA
SLETJK
KEN KGZ PAK
ZAR
BGD SDN
2
1
Rsq = 0.0749
50
LAWS
60
70
80
90
100
28
TI strongly related to GI “in practice”
9
CAN
8
FRA
JPN
USA
ESP
7
ISR
6
JOR
ITA
ZAFLVA
5
CRI
COL
LBN
THA
CHN
EGY
MEX
SEN
BRA
IND GHA
MDA LKA PER
DZA
ROM
SRB
ARM MOZ
TZA
BIH
ARG
GEO
UKR
UGA
VUT
MWI
YEM
VNM LBR TMP
NICPHL
KAZ
GTM
NPL
RUS
BEN
BDI
AZE
ZWE
IDN
CMRNGA
ECU KENETH
TJK
SLE
KGZ PAK
ZAR SDN
BGD
4
3
BGR
CPI
2
1
Rsq = 0.4167
20
30
IMPLEM
40
50
60
70
80
90
29
Global Integrity:
not all categories are highly inter-correlated
100
80
60
40
20
0
Rsq = 0.0806
0
20
40
Procurement rules
60
80
100
30