Transcript Conference of European Statisticians Seminar Session 2 Theme: Efficiency of Statistical Offices Discussant: Donal Garvey (Ireland) Geneva, 11 June 2007

Conference of European Statisticians
Seminar Session 2
Theme:
Efficiency of Statistical Offices
Discussant:
Donal Garvey (Ireland)
Geneva, 11 June 2007



Four Invited Papers:
 Finland, Iceland, Israel, Netherlands
Two Contributed Papers:
 Hungary, Israel
Many common themes - I will deal with some of the
“unique” themes first and then focus most of the
discussion on some common threads
Resource-based planning in Hungary

Understanding the resources really needed to carry out public
sector tasks was a challenge for the HCSO

Resources were dissipated across dispersed regional offices

HCSO reorganised from 19 county offices to six regional
offices. Other countries perhaps went even further.
Q: Is six still even too many for coherent staff mobility and
development?

2 years developing a framework of program elements (550
elements for 2007) to support resource-based prioritisation.
Q: Is the framework changing the way middle and senior managers
are approaching their work and resource utilisation?
Tracking productivity - Finland


An overt “total quality” approach to inputs/systems/outputs to
meet “stakeholders’” needs. Inputs and outputs must be
clearly defined and measured
Total productivity + 27% (2000-2006)
Q: Can you incorporate in the calculations the investment by
other authorities in their registers which contribute towards
better quality statistical outputs?
Interviewing costs - Israel



“Hard to find” populations different characteristics than those
who are “easy-to-find” but repeated attempts to make face-toface contact are expensive.
Limiting the number of attempts will help control costs but
will result in lower response rates. [Ireland: four attempts for
the first wave of the QNHS and two attempts for subsequent
waves - unless the Interviewer is “passing the door”].
Israeli Finding: Survey estimates robust for main survey
variables even when a cap on the number of attempts could
reduce response rates from 88% to 77% and no statistically
significant bias is introduced. This would save 14% of
interviewers time.
Managing/training Interviewers Questions
Q:
Q:
Q:
Q:
Israeli findings based on a post-hoc analysis. Could a cap
affect interviewer behaviour even before that cap is
reached?
Will a cap encourage more “no change” answers accepted
over the telephone?
If interviewers are paid a bonus for success rates above a
certain target level will capping the number of attempts
influence their behaviour?
Do any countries use a model to determine number of
attempts to interview different subgroups?
Functional centralisation
The paper from Iceland mentioned the synergies and efficiencies to be
gained from functional centralisation:
 co-operation rather than misguided competition
 less duplication of resources
 statistics the main rather than a secondary role
 gains in coverage, efficiency and quality
 improved methods, processes and practices
 common web-based dissemination approach
 raised competence of new and existing staff
Q: Will this lower the analytical capability within other Ministries? and
pose a risk to the status of statistics in the future?

In Ireland we have a very centralised statistical system and this lack of
analytical capability within Ministries is a challenge.

Common themes - administrative data




Strong focus to reduce administrative burden for 20 years or
more; unique identification systems for Enterprises,
Buildings, Persons (Finland)
Large efficiency gains if the national culture accepts that the
“normal way” is to maximise the value of administrative
records and public information systems (Iceland) - must be
legislated for properly
Clause in 2004 Act requires use of all available
administrative data before surveying (Netherlands)
Increasing use of identity numbers a rich and cheap source of
administrative data … “a gold mine of data of a census type”
(Israel) - use can depend on culture/legislation in a country
Downsides of administrative data Israel

Q:

Q:
“May present a dangerous path for a Democracy”
Why? The use of administrative data for statistical purposes
seems to create no extra dangers?
“May be of little statistical value”
 tend to be biased (owner agency’s priorities)
 definitional changes without prior notice
 discontinuities make change evaluation difficult
 NSI only a “residual user”: Agency sole decision maker
 Agency interested in the present; NSI in history also
How are these issues dealt with in Finland, Iceland …?
Downsides - Continued
Security risk: Israel suggests a higher security risk for administrative
data … but Iceland says that managing confidentiality is easier.
Q: Who is right?

Late availability, only when agency has completed its task (Israel)
Comment: Not necessarily so. Even in Ireland we have online
access to certain important administrative data.

Do not substitute for everything a modern society needs - must
co-exist with survey data (Israel); increasing need for survey data
also (Iceland); but “over 95% of Statistics Finland’s basic data is
derived from administration registers collected by other authorities”
(Finland)
Q: Is there any risk that a system built almost entirely on administrative
data could become less than fully responsive to the changing
information needs of society?

Reputational Challenges


Administrative data is of a “census type” so there is a
general tendency to see them as “the truth” (Israel)
 poor analysis by inexperienced researchers of microdata from administrative sources can cause problems
 different figures can unjustifiably reduce the trust in
survey data and tarnish the reputation of NSIs.
The Israeli paper gives an example of different estimates
of the annual wage bill which allowed politicians to
undermine politically challenging survey data. I will use
a simpler Irish example.
1996 Irish study
Live Register and LFS/QNHS Data
1990 to 2000
350
300
Persons 000s
250
200
150
100
50
0
1990
1991
1992
1993
1994
1995
1996
Year
Labour Force Survey
Live Register
1997
1998
1999
2000
LFS Unemployment & Registered
Unemployment (Ireland, 1996)




LFS unemployment down 15% 1994-1996; Registered
unemployment no change. Huge debate on the “real” level of
unemployment with reputation of CSO & LFS at risk.
Full administrative file requested by CSO and a 1% sample
selected. Addresses of these registrants included in the LFS (only 4
or 5 people in CSO knew - unknown to field staff) and analysed
separately.
Over 11% of the respondents indicated they had full-time jobs and
4 out of 5 of these said the job started more than one month
previously (many at least a year previously!). About 3 out of 4
indicated the job was a permanent job.
About 10% said they were working part-time and nearly 3 out of 4
of these were “Not underemployed”
1996 Irish study - continued
Conclusion: “many people who are not statistically classified as
unemployed in the LFS are included in the Live Register … also
there are unemployed persons (according to LFS definitions) who
are not on the Live Register”

Changed the debate in Ireland. Put pressure on the Registration
Authority to eliminate fraud

Prime Minister asked us to repeat the survey - CSO said “No”

Data Protection Commissioner agreed that the CSO acted within its
rights and within data protection guidelines
Q: Are there any quality issues around administrative registers in
Finland, Iceland, Netherlands … ? How do you guard against it?

Some answers (Finland)
Extensive, continuous co-operation with other
authorities

Nominated individuals link between statistics units and
register changes

Continuous quality monitoring - example: LFS a quality
check on the population register
Q: Does the population register track emigrants well? Any
interesting insights from the LFS quality checks?

Data matching/linking
Improved record linkage methodology. Administrative data
complimentary to, combined with survey data (Iceland,
Israel) EU-SILC (Iceland)

Ireland: increasingly ask for personal public service number
(PPSN) in surveys to reduce the numbers of questions and to
increase quality by pulling administrative data. EU SILC an
example

“Protocol” on our website explaining what we do; agreed
with Data Protection Commissioner; maintain a public log of
data linking activity
Q: A tricky issue for statistics. Strong statistics legislation
needed. What pitfalls should be avoided?

Methodology
Standardisation of corporate methods (also supports IT
standardisation). Only validated statistical methods within corporate
methodological framework may be used (Netherlands)

Neglect of some distinct activities in the value chain has caused
quality issues in the past. Statistical audit of key processes, rolling
5-year basis (Netherlands)

IT and methods reorganised to support practical implementation
(Finland)
Q: How do methods people respond to “implementation” roles?
Q: How do business owners respond to standardisation in Netherlands?

Comment:
In Ireland many different approaches to editing and
imputation - we are imposing a narrower corporate
framework
Redesign of business processes

Objective: better quality and efficiency through processes which
are coherent, flexible, consistent, reproducible with fewer ICT
applications (Netherlands)
 uniform IT platform with software all from “same family”
and central software deployment decision-making (Iceland)
 software like “a bowl of spaghetti” with poor
documentation and unauthorised automation. Aim to reuse applications with lower development and maintenance
costs (Netherlands)
 align technical development projects into mutually
supporting entities; more uniform framework for
developing and maintaining data systems (Finland)
Business processes - continued
Business Architecture, 4 database design (Netherlands)
 re-use of data and metadata
 distinguish design from implementation
 design driven by output requirements within frameworks
for methodology, data collection, finance, organisation
 no production without standardised metadata

Imposed on every development project through standardised
documentation
Q: If “migration will take several more years” will SN be supporting
two entirely different approaches for several years? What
problems are foreseen?

Business processes - continued
“… Program does not aim at building generic data processing
solutions. In our view that is one bridge too far” (Netherlands)
Q: Does anyone disagree that it is a bridge too far?

Substantial productivity gains to be made in new data editing
methods (Finland) - looking at editing/imputation software
developed by Statistics Canada
Q: Any useful information for the meeting?

Electronic collection of data from enterprises (Finland, Iceland)

Comment: Finland raised the possibility of more international
co-operation (also with IT service providers) in the field of
integrated data collection
Questions and
Discussion