Transcript Assessing the effectiveness of national statistical systems

Assessing the effectiveness of national
statistical systems
Some issues and challenges
Mohamed TAAMOUTI
Director of Statistics, Morocco
1
Outline
• Effectiveness, what does it mean?
• How to asses effectiveness ?
• Assessing conditions of effectiveness
• Assessing indicators of effectiveness
• Concluding remarks
2
Effectiveness, what does it mean?
• Many references : Handbook of Statistical
Organization; fundamental principles of official
statistics, many papers and reports; …
• Ivan Fellegi: “An effective statistical system is
characterized by its ability to illuminate issues,
not just to monitor them; by its ability to evolve in
response to needs; indeed, by its great ability to
be aware of priority information needs and by its
capability to set priorities » .
3
How to assess effectiveness ?
• Conditions of sustainable effectiveness : legal
framework, independence, formal coordination,
etc.
• Indicators of effectiveness: relevance, coverage,
sound methodologies, quality, timeliness, cost
effectiveness, etc.
4
How to assess effectiveness ?
• On the basis of conditions of its sustainability?
– Difficult in the context of developing
countries.
• On the basis of the indicators of effectiveness?
– Relatively appropriate.
5
Legal framework
• An appropriate legal framework is a necessary
condition.
• Most developing countries (all Arab countries)
have a statistical law. However,
• What about law enforcement
• In many DC laws are not fully applicable :
– WB “rule of law” indicator : The percentile rank for
MENA region for example is 49,9 (over 100) with a
score of -0,02 (the percentile rank for OECD : 90,2 with a score of 1,5).
6
Legal framework: Coordination
• In general coordination is efficient where there is
a National Statistics Council allowing the NSS to
act as a coherent system.
• Following a study by the AIRTS, 9 among 19
countries in the Arab region have such a body.
• Is this always good for effectiveness: It depends.
– In some cases an NSC outside the authority of the
NSO (which should be the case) may become a source
of conflict with the NSO.
7
Legal framework: Coordination
• When coordination is weak and a large
proportion of information is produced outside
the NSO we may have a system with two tiers.
• Effectiveness of the NSS ≠ the effectiveness of
the NSO and the NSS may be poor despite an
effective NSO :
– Implementing some methodologies (ex.
Classifications) may become a too difficult and time
costly task.
8
Legal framework: Coordination
• In many countries NSO do not have access to
information produced by other components of the
NSS (Income tax registers, business registers, etc.).
• Usually, when there is no institutional mechanism
coordination is done informally (bilateral
partnerships, personal relationships, NSO as focal
points for international programs, etc.).
9
Coordination: NSDS
• Following WB data, Most DC have an NSDS at
different stages (23 planned, 31 being designed,
70 implementation).
• If real coordination lacks NSDS become difficult
to implement.
• Many countries planned their NSDS without any
real commitment and without securing necessary
human and financial resources.
10
Legal framework: Independence
• Following a study by the AITRS (2008, page 40),
Independence of the NSO is, in general, not
clearly emphasized in statistical laws in Arab
countries.
• This assessment is not, however clearly
evidenced in the report.
• A necessary condition for independence is a clear
emphasis is the statistical law, an enforced law.
11
Legal framework : Independence
• Independence is difficult to assess. Very few can
verify the reliability of data produced.
• In some cases, media play a major role in the
perception of this independence and hence the
credibility of the NSO or NSS.
• Independence matters for credibility of the
whole system.
12
Independence and credibility
• Credibility encourages cooperation between users
and producers of statistics.
– Data produced is trusted and used ;
– Assures the quality through the cooperation of
respondents (citizens, enterprises, …) by providing
reliable information.
• In Developing Countries, the discrepancy between
International agencies estimates and national
data is very harmful to the credibility of the NSS.
13
Legal framework: Confidentiality
• Confidentiality and data (micro data) accessibility,
two key elements of a statistical law.
• Few developing countries (the Arab region) have a
clear dissemination policy of micro data.
• Confidentiality is not well defined even at the
international level. Sometimes it is used against
accessibility of micro data.
• Restricted accessibility usually interpreted as “some
thing to hide”.
14
Indicators of Effectiveness: coverage
• Coverage depends on the needs of the country.
• Basic data is in general produced. The main
challenges in this way are related to :
– The lack of administrative data makes it difficult and
very costly for the NSS to provide data at sub
national level. 1 example:
• Among the 12 DC countries on the STATCAP of WB, only
one has a complete vital registration system coverage.
– In some Arab countries, it is difficult to extend
coverage to some important issues (HIV prevalence,
15
violence against women).
Methodology and Timeliness
• Statistical Capacity indicators (WB)
Methodology
2004
MENA
45
Developing countries 53
2009
55
56
Periodicity & Timeliness
2004
62
72
2009
68
77
• Subscribers to SDDS: 4 Arab countries, 4 in Africa.
• 6 Arab countries implemented SNA93.
• No clear quality policy in most developing
countries.
16
Concluding remarks
• Quantitative indicators of effectiveness show
some improvement globally however a complete
assessment is difficult in DC.
• Many initiatives for NSS assessment, but there is a
need for the development of a more complete
unified assessment framework.
• This will promote quality and credibility of NSS
and contribute to the development of NSS.
17