Transcript UNITED NATIONS ECONOMIC COMMISSION FOR EUROPE CONFERENCE OF EUROPEAN STATISTICIANS Topic (ii): Centralising data collection CREATING A DATA COLLECTION DEPARTMENT: « STATISTICS PORTUGAL'S EXPERIENCE Paulo Saraiva dos.

UNITED NATIONS
ECONOMIC COMMISSION FOR EUROPE
CONFERENCE OF EUROPEAN STATISTICIANS
Topic (ii): Centralising data collection
CREATING A DATA COLLECTION DEPARTMENT:
«
STATISTICS PORTUGAL'S EXPERIENCE
Paulo Saraiva dos Santos
Almiro Moreira
25 September 2013
[email protected]
Statistics Portugal
[email protected]
Statistics Portugal
Geneva, Switzerland
Motivation
Sharing eight years of experience in
centralising data collection,
implementing an integrated and
process driven approach to change
the statistical production, improving its
efficiency and flexibility.
2
Outline
1.
2.
3.
4.
5.
6.
7.
Background and context;
Reengineering the production;
Centralised Data Collection;
Administrative Sources & registers;
Data Collection infrastructure
Benefits of a Centralised Approach;
The future of Data Collection.
3
Statistics Portugal (INE)
• Central national authority for the production
of official statistics;
• Aims at developing and supervising the
national statistical system;
• Created in 1935, has its head office in
Lisbon with delegations in Porto, Coimbra,
Évora and Faro.
4
Geographical dispersion
Oporto
Coimbra
Azores
Common technical
requirements,
methods and
infrastructure
Lisboa
Évora
Faro
Madeira
5
Statistics Portugal
• Statistics Portugal is a public institution
which has legal personality, administrative
autonomy and technical independence in
the exercise of its official statistical activity;
• It is a special public institution integrated
within indirect State administration;
• The Statistical Law confers on Statistics
Portugal statistical authority and legal
obligation to confidentiality.
6
European scope
• European and National Statistics (2013):
Statistical Operations (2013)
Statistics Portugal
European
153
82%
National / Other
33
18%
Total
186
100%
Quantity of Statistical Operations
18%
82%
7
Timeline
• From 1989 to 2003:
– Headquarters & Regional Directorates;
• Regional Directorates:
– Firstly acting as dissemination and data collection center for the
region (NUTS II);
– Gradually assumed active role in statistical production and
regional studies;
– Its organization and resources have increased fast.
• From 2003:
– Proactive evaluation of the existing model;
– Reorganization not guided by resources constraints.
8
Reorganization in 2004
• New Executive board in 2003;
• Hired external advisory company (international
strategy consultants);
• Request a Peer Review in 2004:
– Mr. Ivan Fellegi
• Former Chief Statistician of Canada from 1985 to 2008;
– Mr. Jacob Ryten
• Former Assistant Chief Statistician of Canada from 1969 to
1997.
• A proactive action: to create a new structure.
9
Former organization (2003)
Executive Board
Lisbon and
Tagus Valle
North
Dissemination
Planning and
International
National
Accounts
Agriculture
Center
Alentejo
Finance
Human
Resources
Population
and Census
Business
Legal Support
Methodology
Industry and
Services
Social
Algarve
Information
Systems
Regional Directorates
Support
Short Term
and Forecast
Subject Matter
10
Regional Directorates (2003)
An example of the organization
of a former RD.
IT Support
Social
Business
Regional
Directorate
RH & Resources
Dissemination
Studies
Three hierarchical levels
Regional Directorate (Department)
Unit
Section
11
Former architecture
Survey 1
Recolha
Recolha
Local 1
Survey 2
...
Survey n
Stovepipe systems
Recolha
Recolha
Recolha
Recolha
...
Collection
Tratamento
Collection
Tratamento
Collection
Tratamento
Tratamento
Tratamento
Tratamento
Treatment
Difusão
Treatment
Difusão
Treatment
Difusão
Difusão
Difusão
Difusão
Local 2
...
Dissemination
Dissemination
Dissemination
Local n
Complex, inefficient and not flexible
12
Former organization (2003)
• Heavy and costly organization
– 788 workers: 37% in Regional Directorates.
– 195 managers (25%):
• 14 Departments, 5 Regional Directorates, 48 Units, 128 sections
• Duplication of work, procedures and tools;
• Not flexible enough for the future.
Need to be reorganized
13
Fellegi & Ryten’s Peer Review
• Objective: to review the
Portuguese statistical
system and produce
recommendations;
• Main results:
• The diagnosis;
• Structural problems and
remedies;
• Recommendations
Production re-engineering
•
Started in 2004 and based on the Peer
Review´s recommendations;
Internal reorganization:
•
–
–
–
–
•
A central data collection department was created;
Regional directorates were extinct;
Domain departments have been merged into three
units: economics, social and national accounts;
Methods and information system were merged into
one department.
It was a successful challenge, although some
resistances and constraints.
15
New organization (2004  2013)
Executive Board
Inf Systems
Methodology
Data
Collection
Finance & HR
Dissemination
Porto
Planning
Évora
Legal Support
Coimbra
International
Faro
National
Accounts
Economics
Social
Subject Matter
Communication
Staff
Delegations
Support
Statistical Production
Three hierarchical levels
L1: Department
L2: Unit
L3: Section
16
National
Accounts
Economics
Social and
Demographics
Methods and Information
Systems
Production architecture
Data Collection
17
Impact of the reorganization
2003
2013
% Diference
L1: Departments
19
7
-63%
L2: Units
48
34
-29%
128
13
-90%
Managers
195
61
-69%
Workers / Managers
4,0
10,9
173%
Workers
788
665
-16%
Lisbon
496
508
2%
Regions
292
157
-46%
L3: Sections
Staff reduction without firing anyone
18
Human Resources Distribution
(by macro process)
Staff
9,2%
Support
31,9%
Production
58,9%
19
Centralised Data
Collection
Data Collection at
Statistics Portugal
• Survey’s data collection:
– 40% budget & 30% human resources.
Survey Data Collection is a core function
21
Data Collection
• A Data Collection department assures the collection,
processing and analysis of collected microdata, covering
all business and social surveys;
 HR  ~ 200 workers + 350 freelance interviewers
Annual figures
 120 surveys
 105 business (self-completed)
 15 by interview (CAPI and CATI).
 125.000 companies (99% SME);
 70.000 dwellings;
 35.000 farms.
22
Data Collection Department
Data Collection
Interview
Lisbon 1
Porto 1
Lisbon 4
Porto 2
Lisbon 5
Porto 3
Coimbra
Évora
Faro
Surveys
Self-completed
Surveys
Lisbon 6
Lisbon 7
Data Collection
Lisbon 3
Processes
23
Data Collection Department
Human Resources by Unit
Data Collection
Processes
13%
Self-completed
surveys
46%
Interview
Surveys
40%
Staff
1.5%
24
Data Collection Department
Organization by Unit
• Self-completed surveys:
– By project or statistical operation;
– National management of each project;
• Interview surveys:
– Sections work with the same projects;
– Share same methods, procedures and tools.
• Data collection processes;
– National coordination of interview surveys;
– CATI national coordination.
25
Management within DC
• Decentralized managed but centrally
controlled;
• One overall budget distributed through
each management level;
• Autonomy with responsibility;
• Objective definition in “cascade”;
– Department  Unit  Section  worker
• HR: matrix management;
26
Interview Management System
• Interview Management System supports all
the processes related with social statistics
and the price collection;
• The Survey Management System has
several components: team management
and the tools used by the interviewers to
collect data, transfer them to Statistics
Portugal, allowing them to work both in
face-to-face and telephone interviews.
• .
27
HR and costs control
• Assiduity control  WebRH app;
• Accounting to projects  Factiv app
– Project codes and Task codes;
– Individually daily allocation of the working time to each
project code and tasks;
• Direct HR costs are monthly calculated to each
project, according to individual wages and social
costs;
• The same with other costs and indirect costs;
• Transfers can be made between projects.
28
1
2
Specify
Design
3
4
Build
Collect
5
Process
6
Analyse
7
Disseminate
8
9
Archive
Evaluate
Needs
1.1
Determine
needs for
information
2.1 Design
outputs
3.1 Build data
collection
instrument
4.1 Select
sample
5.1 Integrate
data
6.1 Prepare
draft outputs
1.2 Consult
and confirm
needs
2.2 Design
variable
descriptions
3.2 Build or
enhance
process
components
4.2 Set up
collection
5.2 Classify
and code
6.2 Validate
outputs
1.3
Establish
output
objectives
2.3 Design data
collection
methodology
3.3 Configure
workflows
4.3 Run
collection
5.3 Review,
validate, edit
and analyze
microdata
1.4 Identify
concepts
2.4 Design
frame and
sample
methodology
3.4 Test
production
system
4.4 Finalize
collection
1.5 Check
data
availability
2.5 Design
statistical
processing
methodology
1.6 Prepare
business
case
2.6 Design
production
systems and
workflow
Levels 1 and 2
GSBPM, version 4.0
7.1 Update
8.1 Define
archive rules
9.1 Gather
evaluation
inputs
7.2 Produce
dissemination
products
8.2 Manage
archive
repository
9.2 Conduct
evaluation
6.3
Scrutinize
and explain
7.3 Manage
release of
dissemination
products
8.3 Preserve
data and
associate
metadata
9.3 Agree
action plan
5.4 Impute
6.4 Apply
disclosure
control
7.4 Promote
dissemination
products
8.4 Dispose
of data and
associated
metadata
3.5 Test
statistical
business
process
5.5 Derive
new variables
and statistical
units
6.5 Finalize
outputs
7.5 Manage
user support
3.6 Finalize
production
system
5.6 Calculate
weights
output systems
Division of work
Data Collection
Subject Matter
Shared DC/SM
5.7 Calculate
aggregates
IS & Methods
5.8 Finalize
data files
Dissemination
Quality Control
•
•
•
•
Relationship between
DC & Subject Matter (1)
One major issue at the beginning;
There were a negative perception of the
DC tasks  “a low profile work …”
Conversely, subject matter statisticians
were very “data collection oriented”;
But expectations are always high!
– “You have to do better than me (when I was
responsible for DC) …”
30
Relationship between
DC & Subject Matter (2)
Solution
• Service Level Agreements (SLA) to
manage expectations and to build trust;
• It was used a step-by-step approach, from
a simplified version and increasing
gradually the complexity.
31
Administrative
Sources & Registers
Administrative Sources (ADS)
• ADS are not (still) in the scope of the DC
Department;
• It is managed by Subject Matter
departments, supported by IS & Methods;
• Statistics Portugal is still very “survey
oriented”. Thus, ADS are not well
developed;
• But there one remarkable initiative:
– IES: Simplified Business Information
33
Data Collection
Infrastructure
Outline
• Survey Management System (SIGINQ);
• Other Data Collection Systems:
– Datawahouse;
– HomeCATI;
– Interview Management System;
– Telephone Data Entry;
35
Survey Management System
Survey Management
Re-engineering Working Group
•
•
Design a new approach of production based
on a broad integration with process and tools
standardization;
Use of an internal reference model to describe
the statistical business processes (SPPM);
Survey Management
Business
Agriculture
Social
37
Process Management System
• Management and control of all data collection
processes, including information about respondents
and paradata;
• Supported by the Metadata System;
SAGR
Business
Agriculture
Social
• Similar features, but adapted by statistical unit.
38
Process Management System
• GPap is the core for Business Surveys, linked with:
– Questionnaires and Capture (WebInq and WebReg);
– Respondent Management (GRESP),
– Business Register (FUE),
• Transfers validated microdata to Datawarehouse.
39
GPap components
Survey
Unit
Occur
Collect
Report
Analysis
Update
Manag.
Help
Method
SIGUA
block prop
Supplement
Errors
Specific
Tables
Consult
transfers
Batch
update
Table
Manag.
GPap
Specific
Manage
By mode
Status
Generic
Tables
Transfer to
analysis
Survey
Generic
Cross
Consult
Validations
Specific
Reports
Consult
Analysis
Register
Specific
Open /
Close
Primary Val
Generic
Reports
Launch
Upload
Sample
Respondent
Insert
Manage
entries
Common process
Data Entry
Specific process
40
Survey Management
Contact Centre
Interviewer Management
SAGR
BEA
Business
Agriculture
FNA
Social
41
HomeCATI
• HomeCATI is an infrastructure which
allows freelance interviewers work at
home, integrated in a virtual contact centre
and based on a voice over Internet protocol
(VoIP) solution;
• This solution has many advantages, but
there are many challenges to deal with, like
the interview supervision and monitoring.
42
Interview Management System
• Interview Management System supports all
the processes related with social statistics
and the price collection;
• The Survey Management System has
several components: team management
and the tools used by the interviewers to
collect data, transfer them to Statistics
Portugal, allowing them to work both in
face-to-face and telephone interviews.
• .
43
Telephone Data Entry (1)
• Telephone Data Entry (TDE), which is a
solution by which respondents can return
their data using the keypad on their
telephone;
• Respondents are sent a letter which informs them of the
free phone telephone number to call, their unique
respondent identification key number, and the data
required. On calling the telephone number, the
respondent can choose the appropriated survey, and a
recording of the survey questions is heard and the
respondent enters their data using their telephone keypad.
44
Automated Data Collection
• INE is developing and implementing
Automated Data Collection Methods for
Business Surveys;
• It aims to reduce the reporting burden
businesses, to improve the timeliness and
to promote a more efficient way of
collection data;
• Based on XML, it is already available for
two surveys.
45
Benefits of
Centralised Data Collection
Benefits of centralised DC (1)
1. Development and management of a common
infrastructure, both intellectual and operational,
which could only be duplicated geographically;
2. Creation of a flexible, dynamic and responsive
production architecture tied to the common
services provided by shared means of
production (sampling frames, classifications
and standards, questionnaire designs, methods
and tools, etc.);
47
Benefits of centralised DC (2)
3. Creation of right means of coordination to make
our design work in order to face future (but now
present) budgetary cuts;
4. Adoption of a cost-effective approach that
makes the most effective use of regional and
central resources. It was possible to do more
with the same.
5. Reduction of the data collection cycle, specially
the time to deliver statistical results;
48
Benefits of centralised DC (2)
6. Assistance to develop a steady culture based
on efficiency and innovation, considering the
full in-house design and development
approach;
7. Development of analytic competences in order
to improve the quality of the information (more
reviewing and validation tools);
49
Benefits of centralised DC (2)
8. Creation of an integrated Survey Management
System as well as other Data Collection tools;
9. Reduction of respondent burden:
–
Avoiding duplication of variables and offering easy
and multiple ways to provide data;
10.Reduction of production costs;
–
Estimated in 27.2% (business surveys; 2005 – 2012).
50
Cost reduction
Business data collection
Total costs
Base 2005 = 100%
100%
- 27.2%
75%
50%
25%
0%
2005
2006
2007
2008
2009
2010
2011
2012
51
Electronic Data Collection
Visits
% electronic collection
Questionnairs
2013 – 100%
52
Electronic Data Collection
Avoid variable duplication
Common
Variables
Easy
update
53
Future of Data Collection
Future of Data Collection
• Increase the use of administrative sources;
• Extend Integrated Production Systems;
• Improve Automated Data Collection and the use of
Scanner Data on price collection;
• Increase the multimodal collection capability (web
based);
• Improve the use of paradata to support the quality
processes;
• Create new processes to better understand respondent's
behavior in order to motivate their collaboration.
55
UNITED NATIONS
ECONOMIC COMMISSION FOR EUROPE
CONFERENCE OF EUROPEAN STATISTICIANS
Topic (ii): Centralising data collection
CREATING A DATA COLLECTION DEPARTMENT:
«
STATISTICS PORTUGAL'S EXPERIENCE
Thank you for your attention!
Paulo Saraiva dos Santos
Almiro Moreira
25 September 2013
[email protected]
Statistics Portugal
[email protected]
Statistics Portugal
Geneva, Switzerland