Transcript Statistical process as a structured chain of successive actions and intermediate products, supported by the coherent use of metadata by Statistics Netherlands

Statistical process as a structured chain of
successive actions and intermediate products,
supported by the coherent use of metadata
Focused on energy statistics and relation
IRES
Hans Pouwelse
Statistics Netherlands
1
content
• Brief description Business Architecture statistical
process
• Role of coherent meta data
• Focused on Energy Statistics
• Relation to IRES
2
Meta servers for
C o n c e p t u a l
M e t a d a ta
TheStatistical
StatisticalProcess
Process
The
Input-world
Output-world
Input Variables
D e s i g n
R
E
S
P
O
N
D
E
N
T
S
R
E
G
I
S
T
R
A
T
I
O
N
S
Micro Level
Input-register
BaseLine
Output Variables
S t a g e :
Micro
level
Output-register
MicroBase
4
Macro- Level
Cube
StatBase
metadata
Macro- Level
Output Database
StatLine
2
3
U
1
S
E
5
6
Data collection
& data entry
Input sphere
7
Aggregation
& Disclosure
Control
Editing &
imputation
8
Throughput sphere
Implementation
stage:
9
Selection &
Tabulation
Publication &
Dissemination
R
S
Output sphere
data
Meta servers for
P r o c e s s
M e t a d a ta
3
Conceptual metadata
Output world
• Definitions variables and classifications in output
terms (‘language’ of users)
= definitions rows en colums output tables
• Reporting period (year, month…)
• Units of measurement
• Statistical units (micro level output register)
4
Meta servers for
C o n c e p t u a l
M e t a d a ta
TheStatistical
StatisticalProcess
Process
The
Input-world
Output-world
Input Variables
D e s i g n
R
E
S
P
O
N
D
E
N
T
S
R
E
G
I
S
T
R
A
T
I
O
N
S
Micro Level
Input-register
BaseLine
Output Variables
S t a g e :
Micro
level
Output-register
MicroBase
4
Macro- Level
Cube
StatBase
metadata
Macro- Level
Output Database
StatLine
2
3
U
1
S
E
5
6
Data collection
& data entry
Input sphere
7
Aggregation
& Disclosure
Control
Editing &
imputation
8
Throughput sphere
Implementation
stage:
9
Selection &
Tabulation
Publication &
Dissemination
R
S
Output sphere
data
Meta servers for
P r o c e s s
M e t a d a ta
5
Conceptual metadata
Input world
Questionnaires
• Design questionnaires: questions, definitions (in
the ‘language’ of respondents)
• Reporting period
• Observation units (observable units)
Registrations (‘administrative data’)
• Definitions of variables and type of units
6
Meta servers for
C o n c e p t u a l
M e t a d a ta
TheStatistical
StatisticalProcess
Process
The
Input-world
Output-world
Input Variables
D e s i g n
R
E
S
P
O
N
D
E
N
T
S
R
E
G
I
S
T
R
A
T
I
O
N
S
Micro Level
Input-register
BaseLine
Output Variables
S t a g e :
Micro
level
Output-register
MicroBase
4
Macro- Level
Cube
StatBase
metadata
Macro- Level
Output Database
StatLine
2
3
U
1
S
E
5
6
Data collection
& data entry
Input sphere
7
Aggregation
& Disclosure
Control
Editing &
imputation
8
Throughput sphere
Implementation
stage:
9
Selection &
Tabulation
Publication &
Dissemination
R
S
Output sphere
data
Meta servers for
P r o c e s s
M e t a d a ta
7
Process metadata
To provide methods and rules for the process to
go from stage to stage (from database to
database)
• Sampling scemes
• Methods and rules for editing, validation,
imputation, aggregation and disclosure control
• Transformation rules to bridge the gap between
input concepts and output concepts
8
Meta servers for
C o n c e p t u a l
M e t a d a ta
TheStatistical
StatisticalProcess
Process
The
Input-world
Output-world
Input Variables
D e s i g n
R
E
S
P
O
N
D
E
N
T
S
R
E
G
I
S
T
R
A
T
I
O
N
S
Micro Level
Input-register
BaseLine
Output Variables
S t a g e :
Micro
level
Output-register
MicroBase
4
Macro- Level
Cube
StatBase
metadata
Macro- Level
Output Database
StatLine
2
3
U
1
S
E
5
6
Data collection
& data entry
Input sphere
7
Aggregation
& Disclosure
Control
Editing &
imputation
8
Throughput sphere
Implementation
stage:
9
Selection &
Tabulation
Publication &
Dissemination
R
S
Output sphere
data
Meta servers for
P r o c e s s
M e t a d a ta
9
Qualtity metadata
• Define quality standards (required output
quality)
• Rules to measure resulting quality
10
Focused on Energy Statistics
• Energy statistics are normal statistics:
logic stages sceme and metadata fully
applicable to energy statistics
• Some elements specific for energy statistics:
11
Meta servers for
C o n c e p t u a l
M e t a d a ta
TheStatistical
StatisticalProcess
Process
The
Input-world
Output-world
Input Variables
D e s i g n
R
E
S
P
O
N
D
E
N
T
S
R
E
G
I
S
T
R
A
T
I
O
N
S
Micro Level
Input-register
BaseLine
Output Variables
S t a g e :
Micro
level
Output-register
MicroBase
4
Macro- Level
Cube
StatBase
metadata
Macro- Level
Output Database
StatLine
2
3
U
1
S
E
5
6
Data collection
& data entry
Input sphere
7
Aggregation
& Disclosure
Control
Editing &
imputation
8
Throughput sphere
Implementation
stage:
9
Selection &
Tabulation
Publication &
Dissemination
R
S
National energy
policy
International
EU (en stat reg)
IEA
(ESM)
UN (IRES)
Output sphere
data
Meta servers for
P r o c e s s
M e t a d a ta
12
Conceptual metadata
Output world
• Classification Energy products
(IRES chapter 3 (SIEC); InterEnerStat, ESM)
• Classification Energy Flows, Energy balance
(IRES chapter 5 and 8; InterEnerStat, ESM)
• Classification economic activity
(ISIC, NACE)
• Joint Annual Quest, JODI, MOS etc
• Units of measurement (Joule, toe, tonnes, kWh etc.)
(IRES chapter 4)
• Caloric values (IRES chapter 4)
13
Conceptual metadata
Input world
Questionnaires
• Design energy questionnaires
Neth: commodity/energy balance format
Registrations (‘administrative data’)
• Definitions of variables and type of units
Neth: client files energy companies (unit=
connection adress)
14
Meta servers for
C o n c e p t u a l
M e t a d a ta
TheStatistical
StatisticalProcess
Process
The
Input-world
Output-world
Input Variables
D e s i g n
R
E
S
P
O
N
D
E
N
T
S
R
E
G
I
S
T
R
A
T
I
O
N
S
Micro Level
Input-register
BaseLine
Output Variables
S t a g e :
Micro
level
Output-register
MicroBase
4
Macro- Level
Cube
StatBase
metadata
Macro- Level
Output Database
StatLine
2
3
U
1
S
E
5
6
Data collection
& data entry
Input sphere
7
Aggregation
& Disclosure
Control
Editing &
imputation
8
Throughput sphere
Implementation
stage:
9
Selection &
Tabulation
Publication &
Dissemination
R
S
National energy
policy
International
EU (en stat reg)
IEA
(ESM)
UN (IRES)
Output sphere
data
Meta servers for
P r o c e s s
M e t a d a ta
15
Relation to IRES
• Attempt to link IRES chapters with a logical
place in the business arcitecture sceme:
16
Chapters IRES
Meta servers for
C o n c e p t u a l
M e t a d a ta
3, 4, 5, 8
Statistical
Process
The Statistical Process
6b?
The
Input-world
Output-world
Input Variables
6a?
R
E
S
P
O
N
D
E
N
T
S
R
E
G
I
S
T
R
A
T
I
O
N
S
Output Variables
6a?
D e s i g n
Micro Level
Input-register
BaseLine
6b?
S t a g e :
Micro
level
Output-register
MicroBase
Macro- Level
Cube
StatBase
metadata
- Level
Macro
Output Database
StatLine
U
1,2
S
E
7a
11
R
Data collection
& data entry
Aggregation
& Disclosure
Control
Editing &
imputation
Input sphere
Selection &
Tabulation
stage:
S
10
Throughput sphere
Implementation
Publication &
Dissemination
Output sphere
data
9
Metadata
Quality
7b?
Meta servers for
P r o c e s s
M e t a d a ta
17
summary (1)
• Statistical process seen as a logical output
oriented cycle
• Starts with users (identification user needs)
• Ends with users (provide desired statistical
results)
• Structured chain of successive actions
• Delimited by intermediate products (logical
databases)
• Supported by the coherent use of metadata
(conceptual, process, quality)
18
summary (2)
• Important to make clear distinction between
input world and output world
• Explicitly bridge the gap between input concepts
and output concepts:
-input definitions
output definitions
-observation units
statistical units
19
summary (3)
• Logical model applicable to energy statistics (as
being normal statistics)
• IRES may be structured according to the lines of
the model
• Which seems not completely be the case right
now!
(in particular: distinction input/output world)
20