Transcript Framework for Quality Assurance in Population Censuses under

Framework for Quality Assurance in
Population Censuses under Fragility
and Adverse Conditions
Dr. Luay Shabaneh
Chief Technical Advisor, Population Census
UNFPA-Iraq
Introduction
•
NSOs are committed to provide accurate, timely, and
cost-effective population count through censuses.
•
Experience reveals areas of potential risk that could
hamper efforts in conducting censuses, particularly
regarding
1. quality assurance (QA)
2. quality control (QC)
Concept of Quality QC and QA
No single definition for quality in the census context
Here are some of the many definitions of “Quality.”
• Conformance to requirements.
• Conformance to mutually agreed-to expectations.
• Fitness for use.
Quality is “getting the job done, on time, within budget,
so that it meets the specified requirements”
Why Quality in Census
Census taking became more complex
• Outsourcing human resource
1. Justify public expend.
2. Gain Trust of public
3. Deserve Authority
4. Protect from interven.
QA program can decrease the risks associated with
these factors. An effective QA function can reduce the
chances of expensive missteps that require rework or
last minute work-around solutions
Example on addressing QA/QC in the Region
Egypt2006
External Audit
Palestine 2007
Structural system
PES
Sudan
2008
MoC
Iraq
2010
QA Sector
(NECIO2010)
Conceptual Framework
Quality= Fitness of use.
It has to satisfy:
Utility: usefulness for user
Objectivity: accurate, reliable, and unbiased, clear
Integrity: protection from unauthorized access
Conceptual Framework
For operational purpose, some countries
operationalized the definition of utility, objectivity,
and integrity in terms of six dimensions:
1. Relevance
2. Accuracy
3. Timeliness
4. Accessibility
5. Interpretability
6. Transparency.
Case
Tradeoff
Issue
Tradeoff Decision
Iraq Census Quality of
Postponing the census
2004
census data, and
ability to keep
professional
implementation
Iraq Census Quality of
2009
census data
Postponing the population counting stage
and continuing the work in the listing
and numbering, in addition to updating
cartography tools.
Afghanistan Ability to cover Postponed the census.
Census 2009 nationwiall
census areas
Palestine
Census
1997
Palestine
Census
2007
Ability to
cover all
census areas
particularly in
Jerusalem
Decided to go ahead regardless of
Occupation position in allowing
census teams to work in Jerusalem.
The census ended up by not
covering part of Jerusalem city.
Would a QA
Ability to
Census management decided to
framework
lead
to
cover all
make prior data collection in
different
outcome?
census areas
Jerusalem
to avoid prevention by
particularly in Israel, and accepted time lag
Jerusalem
between the data of Jerusalem and
the rest of the country.
Message 1
Quality is very important dimension. It hasNot
an Abstract Definition, it has to be derived
from the domestic situation, in line with
1. Technical capacity
2. Census infrastructure and capacity of staff
3. National and Political Context of the census
4. Potential use or misuse of the data
Quality Control and Quality Assurance
QC refers to quality related activities associated with the creation of
census deliverables. It is used to verify that deliverables are of
acceptable quality.
1. Peer reviews
2. Testing process.
QA refers to the process used to create census deliverables, and can be
performed by census management.
1.
Checklists
2.
Preventive measures system
3.
In-process census audits.
Adverse Conditions and Risk Analysis
Adverse conditions are processes of transition towards
consolidated statehood.
Types of Adverse Conditions:
1. A crisis state is a state under acute stress, and unable to
manage conflict and shocks.
2. A fragile state is a state that is failing, or at risk of failing,
with respect to authority, legitimacy and comprehensive
service entitlements of citizenry.
3.
4. A post-conflict state is a state that has emerged out of
conflict, crisis or fragility and is progressively establishing
institutions
Country Type
Challenges to launch a census
Rwanda Conflict 1. Impact of census on 1994 genocide
2. Effect of genocide on migration and
people’s displacement.
(legitimization)
Nigeria Conflict 1.
2.
3.
4.
Public trust
Conflict over resources
Violence
Potential isuse and
misinterpretation
5. Acceptance of results; 1990 census
data were accepted in 1998 because
of political disagreements
Cambodia Post
conflict
1. Lack of expertise after 3 decades
2. Use to reconstruct a country that had been
ravaged by civil strife and genocide.
Iraq
Post
conflict
1.
2.
3.
4.
Sudan
Post
conflict
1. Distribution of wealth
2. Distribution of powers
3. Use data for referendum
South
Africa
Post
conflict
1. Post apartheid situation (legitimization)
2. Mistrust
Allocation of resources
Politicization over national reconciliation
Security problems
Allocation of power
Palestine
Conflict
Afghanistan
Fragile
State
1.
2.
3.
4.
5.
Political instability
Security problems
Counting in Jerusalem
Division between WE and Gaza
Use of census for political
purposes
1. Legitimacy of the state
2. Security and counting in the
mountains
Message Two
1. Census is not only a statistical activity, it has
political significance
2. Census launching tradeoff is important
3. Quality tradeoff is important
4. Out-of-box ideas are always needed
Risk Assessment Framework
1 Select census stages &
form risk assessment team
8 Validation & feedback and
modify accordingly
2 Decompose each census
stage into main activities
7 Put measures and actions
into a systematic pastern
3 Analyze risks and identify
its sources
4 Identify the likelihood of
each risk and vulnerabilities
6 Identify preventive
measures and interventions
5 Identify the causes of
threats
Challenge
Problem
Risk
Framework of QA modalities - Regular problems
Characteristics
Measure /Modality
Regular difficulties that take place in
almost all censuses. These difficulties
occur because the census is a huge
statistical undertaking. This includes
1. Management of large numbers
2. Mobility of people and goods
3. Trransfer of knowhow
4. Technical difficulties
5. Security of questionnaires and form
6. ICT utilization
7. Coverage and content errors
Good management,
well defined plans
and procedures,
better training, QA
plans, systematic
pattern of actions,
mobilization of
national and/or
external support,
etc
Framework of QA modalities- Challenges
Difficulties that are country-specific
and related to domestic natural
and/or human made factors with
limited impact on census
undertaking, and over passing such
difficulties depends mostly on the
census team. For example:
1. Natural catastrophes
2. Reaching all remote areas
3. Lack of resources
4. etc
Special traditional
measures, separate
teams and
modalities,
emergency plans,
etc.
Framework of QA modalities- Risks and Threats
Difficulties that are country-specific
and related to domestic human
factors (mostly political and/or
security) and over passing these
risks depend mainly on other
players. This includes
1. Politicizing of the census
2. Political division across the nation
3. Potential fraud by certain group,
4. Potential violent resistance to
census
1.Out-box measures
2. External monitoring
3. Mechanisms, special
out-of-box measures
(finger print, eye
mark, observatory,
etc),
4. Political dialogue
5. International
intervention
6. Security measures
7. etc
Activities within QA framework- design stage
•
Central/regional committees for monitoring and controlling field
work.
• Adequate selection and training of staff.
• Well established database on other sources information to
compare with.
• Cross fertilization with other running surveys (sources: registers)
in terms of concepts, classifications and operational definition.
• Developing accurate maps and checking its accuracy on regular
basis through field visits and office checks.
Activities within QA framework- Census Fieldwork
•
Involve population, local authorities, religious/political/ethnic
groups/leaders, associations/NGOs, field staff (directly or indirectly)
and other relevant bodies.
• Analyze the modalities of which a risk could occur including threats
on field staff, call for boycott census among certain
ethnicity/group/area, etc.
• Prevent addition/omission of housing units, household, persons in
households, persons in institutions, destruction of forms collected
and intrusion in the information technology systems.
• Communicate on the goals of the census, addressing the concerns of
political/religious/ethnic groups
Activities within QA framework- Census Fieldwork
•
Communicate on the strong quality control mechanism, and identify
potential risks, areas and groups.
• Maintain the general features of field control which should be a part
of the quality assurance and improvement programme of the census
• Ensure the commitment of all partners and standard application
country wide.
• Involve many persons – mix ethnicities to neutralize fraud and
include as much as possible comparison with other sources
• Make listing, previous censuses, administrative sources, etc) and
applicable and acceptable by the public
Activities within QA framework- Census Fieldwork
• Involve census stakeholders on the results at each stage of launching
the census and consistency/inconsistency of results.
• Establish good enumeration and implementation environment that
does not arouse suspicion except for minor boundary issues.
• Ensure transparent, open and competitive procedure, avoid party or
special interest lists, recruit enumerators from community using
catchment zone strategy
• Recruit supervisors from other communities to avoid misuse of
power.
Activities within QA framework- Data Processing Stage
• Plan dissemination at an early stage of census design
• Identify what to release and when
• Take into consideration other national events to avoid misinterpretation
of dissemination timing, particularly in the highly politicized
atmosphere
• Try not to surprise the political level and plan very well for releasing
the census results
• Release in an official event where all relevant bodies are invited or
informed on the day of releasing the results
Activities within QA framework- Data Processing Stage
• Do not make any compromise on the professional ethics, but also
absorb the situation and not provoke any influential bodies
• Report on the development of the progress of the census in a
transparent way to all relevant bodies
• Release census data once and not to give privilege to any person or
institution or party
• Deal with census results as “public Goods”
• Do not release the data before checking it very well, including PES
Conclusions
• Adverse conditions are not exceptional anymore
• A well articulated practical and operational guidelines on census
organizations and management is needed to complement the
standard United Nations P&R
• UNSD as well as regional commissions and UNFPA country
offices might play a vital role to develop such guidelines
• In adverse conditions, census might be employed as a
peacemaking and national reconciliation tool
• Adverse conditions might hamper census if the census
management was unable to mobilize an enabling environment
to incubate it
Conclusions
• Openness on international experiences is a wise step to provide
sound technical support and motivate enabling environment
• Understanding of national and political context of the census is a
key factor
• Compilation of tradeoffs regarding the census implementation
and/or its quality based on religious risk analysis including
sources of risks, its likelihood to happen and its impact on the
census is a wise step
• Setting up a comprehensive quality assurance system based on
the results of the risk analysis is important
Conclusions
• QA system should cover all census stages including design,
planning, implementation, supervision, dissemination and
evaluation.
• QA system should be comprehensive, integrated, sound, demand
driven and country-specific.
• Census management has to absorb national local conditions,
public acceptance and political sensitivities.
• Out of box ideas, tools and approaches might be employed
within the international standards boundaries.