Data, Metadata and Quality Management Framework

(

Quality and Information Management Framework at the Vietnamese Ministry of Planning and Investment

)

Michael Colledge and Bryan Fitzpatrick, Consultants European Technical Assistance Programme for Vietnam (ETV2)

Content of Presentation

1. Context – Vietnamese Ministry of Planning and Investment 2. Problems 3. Solutions 4. Quality Concepts and Management 5. Information, Data and Metadata Concepts and Mgt 6. Quality and Information Management Framework 7. Conclusions 2

1. Context: Vietnamese Ministry of Planning and Investment (VMPI)

• Core business functions • Organisation and working environment • Inputs and outputs 3

Core business functions as defined in regulations

• Developing strategies and plans for national socio economic development − includes analyzing and forecasting economic performance • Developing mechanisms and policies for economic management • Issuing decisions, instructions, and circulars on planning and investment • Managing official development assistance − including developing strategies for attracting, coordinating and reporting 4

Core business functions as defined in regulations (continued)

• Managing procurement and tendering • Managing enterprise and business registration • Developing plans for renewal and development of state enterprises • Promotion of small and medium size businesses • Appraising and monitoring individual investments 5

Core business functions

• Briefly summarised − planning, monitoring, analysis, forecasting, decision making 6

Organisation and Environment

• 28 Departments and Centres − fragmented, overlapping responsibilities • Rapid evolution − − − Transition for market economy No such ministry in western developed countries Move to role as Ministry of Economy • General Statistics Office recently added 7

Information and Data Inputs and Outputs

• Characteristics of inputs − − diverse range relatively low volume • Characteristics of outputs − mostly information rather than data 8

2. Quality Problems and Issues

• Information and data acquired are poor or unknown quality and/or lacking in timeliness • Procedures and facilities for acquiring, processing, analysing and sharing information and data are poor • Information and data often acquired and maintained on paper • Data sources not fully exploited – duplication of data collected by GSO • There are differing versions of what are nominally the same data 9

Quality Problems and Issues (continued)

• Limited metadata accompanying information and data − data are not well understood • Departments/centres receive funding directly from donors and undertake uncoordinated developments − No organisational unit with mandate or sufficient resources to take coordination or leadership role. • There is no definitive repository of information and data • MPI website is poorly populated and data are error prone 10

Underlying Problems

• Division of functions amongst departments has resulted in semi-autonomous fiefdoms with little motivation to collaborate • No organizational unit responsible for developing, facilitating and monitoring general quality management policies, guidelines and procedures for the MPI as a whole • No organizational unit responsible for developing, facilitating and monitoring general information, data and metadata management policies, guidelines and procedures 11

3. Solutions

• Introduction of the

Quality and Information Management Framework

• Development principles − − − Simplicity Use of Standards, Guidelines and Recommended Practices Harmonisation and Integration 12

Harmonisation and Integration

13

Inputs Transformation Process

Figure 1:

Outputs 14

Inputs Acquisition Process Transformation Process Distribution Process Outputs Repository

Figure 2

15

Figure 3

16

I n p u Acquisition Process Transformation Processes Distribution Process Repository

Figure 4

17

I n p u Acquisition Processes Transformation Processes Repository 2 Repository 1

Figure 5

Distribution Processes 18

4. Quality Concepts and Management

• Evolution of quality concepts − − − − Inspection Quality control Quality assurance Total quality management • International quality standards − − ISO 9000 series European Foundation for Quality Management (EFQM) Excellence Model 19

ISO 9000 Series

• ISO 9000: 2005 Quality Management Systems - Fundamentals and Vocabulary − To provide concepts and terminology • ISO 9001:2000 Quality Management Systems - Requirements − Basis for certification • ISO 9004:2000 Quality Management Systems – Guidelines − for performance improvements in mature system 20

ISO 9000 Quality Concepts

• Customer focus • Leadership • Involvement of people • Process approach • System approach to management • Continuous improvement • Fact based decision making • Mutually beneficial supplier relationships 21

ISO 9001: 2000 Quality Management Systems

• Basis for certification • Comprises five parts: − − − − − Quality Management System Management Responsibility Resource Management Product Realization Measurement, Analysis and Improvement • Does not provide implementation guidelines • Needs to be interpreted/adapted to particular circumstances 22

Interpretation/Adaptation to VMPI Context

• ISO Standards primarily designed in context of enterprises selling to market • Need to be interpreted/adapted to particular situation • Government organisation − not profit based − “user” rather than “customer” − mostly internal users − user cannot influence quality through purchase decisions • Whose products are information and data − with structural and reference metadata • structural – needed to access product • reference – needed to understand its quality 23

Interpretation/Adaptation to VMPI Context (cont)

• Statistical offices are specialists in information/data • Have developed several standards − − − − − Eurostat Quality Framework OECD Quality Framework IMF Data Quality Assessment Framework Statistics Canada Quality Framework Statistics Sweden… • Frameworks differ only slightly 24

General Definition of Product Quality

• Starting point for quality management • From customer/user perspective − − − “Totality of features and characteristics of a product or service that bear on its ability to satisfy stated or implied needs” (ISO 8402 – 1986) “Degree to which set of inherent characteristics (of product) fulfill requirements (ISO 9000) “Fitness for Use” • Need to specialise to government information agency − by identifying and defining characteristics of quality 25

Characteristics of Information/Data Product Quality

• Relevance − Degree to which products meet current and potential user needs − − Are required information/data available?

Are information/data available required?

• Accuracy/Reliability − Degree to which information/data correctly describe event/phenomenon/situation they are designed to measure/represent − Many aspects, no single overall measure • Timeliness − Length of time between information/data availability and events/ phenomena they describes − Measured relative to time period for which product is likely to be useful 26

Characteristics of Information/Data Product Quality

• Accessibility − Extent to which users informed of information/data availability − − Suitability of format and medium by which product accessed Cost of access • Interpretability − Ease with which user may understand and use informatin/data − Definitions of concepts, coverage and data elements • Coherence − Degree to which information/data products are logically consistent and complete 27

Other Quality Considerations

• Quality dimensions are overlapping and interrelated −

cannot be combined into a single indicator

• Achieving acceptable level of quality is matter of trade-off −

for example accuracy against timeliness

• Cost must also be considered as constraint on quality −

quality cost trade-off

• Product quality achieved through process quality and

performance management

28

Process Quality and Performance Management

• Two aspects to management of processes − − Effectiveness – degree to which processes generates products of high quality Efficiency – minimal use of resources in generating products 29

Effectiveness (Product Quality)

Figure 6. Process Quality and Performance Management

Quality and Performance Characteristics Relevance Methodology Management ** Information /Data Management Technology Management Human Resource Management * Accuracy Timeliness Accessibility Interpretability ** * * ** ** * * * * * * Coherence * Efficiency (performance) Resource Usage Key: ** primary responsibility; * shared responsibility * ** ** * * 30

Quality Concepts - Conclusion

• In organisation whose primary inputs and outputs and information/data

information, data and metadata management is key to quality and performance management

31

5. Information, Data, Metadata Concepts and Management

32

Information and data management concepts and management

Terminology • Data refers to numerically structured information − typically tables of numbers, graphs and charts • Data has to be accompanied by metadata − describing to what the numbers refer and how they have been produced • Used in narrow sense information refers material that are not structured − reports, correspondence, pictures, images, etc.

• Used in broad sense information includes data and metadata 33

Information Metadata Standards

• Dublin Core ISO 1588-2003 • Comprises set of descriptors for any information resource: − − − − − title, creator, subject description, publisher, contributor date, type, format, identifier source, language, relation coverage, and rights • Others?

34

(Data) Metadata Standard

ISO/IEC 11179: Information Technology Metadata Registries

Standard aims to: • provide a common understanding of data elements across organizational units and between organizations • • support re-use and integration of data over time, space, and applications harmonise and standardise data within an organization and across organizations 35

(Data) Metadata Standard

ISO/IEC 11179: Information Technology Metadata Registries (continued)

Comments: • Part 3: Registry metamodel and basic attributes – generalised, complex difficult to understand • Part 6: Registration – good starting point, will use simplified version 36

Data and Metadata Standards

ISO/TS 17369:2005 SDMX V1.0

• To support business practices by enabling efficient exchange and storage of data and metadata • Provides format for structuring and reporting metadata, including methodology • Provides an agreed structure fordata/metadata flows

SDMX V2.0 (not yet ISO standard)

• Being reviewed by WG2 of ISO TC 154 • Enhanced treatment of metadata 37

Figure 7: SDMX Top Level Model

38

SDMX Top Level Model

This is a description of an data or metadata “flow” – an abstracted data or metadata set that will potentially occur for many periods and from many providers (eg a regular table received by MPI from various sources) 39

SDMX Top Level Model

metadata set from a particular provider at a particular time, eg, a particular table from Ninh Binh province, for a particular period This is a description of an data or metadata “flow” – an abstracted data or metadata set that will potentially occur for many periods and from many providers (eg a regular table received by MPI from various sources) 40

SDMX Top Level Model

metadata set from a particular provider at a particular time, eg, a particular table from Ninh Binh province, for a particular period This is a description of an data or metadata “flow” – an abstracted data or metadata set that will potentially occur for many periods and from many providers (eg a regular table received by MPI from various sources) Provision Agreements indicate what Providers will provide what subset, when, how often, and how 41

SDMX Top Level Model

metadata set from a particular provider at a particular time, eg, a particular table from Ninh Binh province, for a particular period This is a description of an data or metadata “flow” – an abstracted data or metadata set that will potentially occur for many periods and from many providers (eg a regular table received by MPI from various sources) This identifies the Data Providers, giving indicative and contact information and linking to Provision Agreements and actual data and metadata sets Provision Agreements indicate what Providers will provide what subset, when, how often, and how 42

This describes the structure of the data or metadata flow –

SDMX Top Level Model

flow (an actual data or metadata set). Links to all other structural metadata.

This categorises all the defined data and metadata flows, providing a structuring framework and a basis for searching. Links to other structural metadata.

43

SDMX Top Level Model

44

SDMX Application at VMPI

Envisage SDMX Registry/Repository with:

• Code sets and classifications − environment for standardising and harmonising • Data structure definitions − for all regular data sets, i.e., data flows • Categorisation schemes − to index data flows • Data storage environment for data sets − initially simple file store − possibly a database store − possibly a star-schema store • with star schema, design generated automatically from structural metadata • provides options for different “cuts” through data flows 45

6. Elements of Quality and Information Management Framework at VMPI

• A: More Strategic and Integrated Perspective of VMPI Role, Functions and Information: − − A1: Review and Revision of Role and Functions A2: Comprehensive Understanding of Inputs, Processes and Outputs • B: Fully Functional Quality and Information Management Programmes: − B1: Establish Quality Management System − B2: Establish Information, Data and Metadata Management Programme 46

Elements of Quality and information management framework (continued)

• C: Improved Quality and Information Management Infrastructure: − C1: Develop Corporate Quality Management Facilities − C2: Develop Corporate Metadata Management Facilities − − − C3: Develop Corporate Facilities for Acquisition, Capture, Storage, Access and Dissemination of Data C4: Develop Corporate Facilities for Acquisition, Storage, Access and Dissemination of Information C5: Enhance Corporate Planning, Monitoring, Analysis and Forecasting Facilities 47

Elements of Quality and information management framework (continued)

• D: Continuous Improvement and Reengineering of Business Processes: − D1: Continuous Improvement of Core Business Processes − D2: Reengineering of Core Business Processes • E: Comprehensive Quality and Information Management Training Programme: − E1: Develop and Conduct Quality Awareness and Management Training − − E2: Develop and Conduct Information, Data and Metadata Management Training E3. Develop and Conduct Training in Planning, Monitoring, Analysis and Forecasting 48

Conclusions

• In an organisation whose main inputs and outputs are information and data − − − Quality management and information/data management go hand in hand Information/data management can be viewed as component of quality management Quality management is good umbrella for achieving information/data management • Need for recipe describing: − − (parts of) international standards that are readily applicable and useful additional best practices that should be considered 49