Transcript Case Study Australian Bureau of Statistics A brief history of metadata at the ABS Simon Wall Metadata Team Leader [email protected].

Case Study
Australian Bureau of Statistics
A brief history of metadata at the ABS
Simon Wall
Metadata Team Leader
[email protected]
Premetazoic era




1905-1973
before the term 'metadata' existed
some metadata management anyway
but no metadata management strategy.
Protometazoic era
 1973-1990
 created data dictionaries
limited reuse of common data elements
not fully corporate in scope
 driven by technology, not by strategy
Mesometazoic era
 1991-2000
 focus on data warehousing for output
 beginning of a strategic view
Towards a Unified Data and Metadata System at the ABS (1991)
 developed core metadata repositories
statistical activities (ABS: collections), datasets, data items
(data elements/variables), classifications, terms
 seen as overhead by subject matter areas
limited return for effort
quality questionable, not actively maintained
Neometazoic Era
 2001-2007
 reorganised as Corporate Metadata Repository
adding services interfaces
legacy repositories both asset and liability
vision greater than implementation
 Formal strategy adopted
Strategy for end-to-end management of ABS metadata (2003)
 Some new development work
11179-based Data Element Registry
Quality Declarations
Holocentric epoch
 2008+
 ABS executive seek clear and compelling picture
of long term metadata aspirations.
not provided by existing strategy.
How would it improve the way the organisation operates?
What new capabilities would it deliver?
Are they the capabilities we need most?
Is the strategy actually achievable?
Is it the most appropriate future for us to be investing in?
 now developing "2020 Vision"
encapsulating longer term ABS aspirations for metadata
Drivers for 2020 vision
 Uptake of Corporate Metadata Respository
services very slow
existing processing systems "monolithic"
not easy to modify
existing processing systems have "special needs"
customisations required
lack of funds and business drivers
disconnect between user view and description for
systematic manipulation.
processing systems unable to provide metadata to
describe transformations
Drivers for 2020 vision
 Frame of reference for ABS metadata
management requirements is no longer defined
by the boundaries of the organisation
Developing infrastructure that can be shared with
other producers of statistical data within Australia
Other government agencies now have their own
focus on interoperability, including metadata.
Software collaboration / open source software
Interoperate with agencies with a focus on
administrative, geospatial, or research-oriented data
content.
Where are we going?
 One The practical complexities and difficulties of using
metadata in an "end to end" context to actually drive
actual ABS processes is becoming clearer.
 Two There is pressure to make our metadata
capabilities less ABS specific
more flexible, interoperable and generic - .
 These factors suggest that charting a successful way
forward in regard to metadata management for the
ABS will require a "paradigm shift", a new era.