Transcript TASK TEAM ON WIGOS METADATA (TT-WMD) FIRST SESSION Manuel Bañón ([email protected]) State Meteorological Agency (AEMET), Alicante, Spain GENEVA, SWITZERLAND 11 – 15 MARCH 2013

TASK TEAM ON WIGOS METADATA (TT-WMD)
FIRST SESSION
Manuel Bañón ([email protected])
State Meteorological Agency (AEMET), Alicante, Spain
GENEVA, SWITZERLAND
11 – 15 MARCH 2013
The purpose of the WIGOS (1) initiative is to establish an upgraded
environment for observing systems that will:
Standardize observing requirements (standards, guidelines and best
practices) defined for the user’s community
Avoid multiple, overlapping sources of information in WMO and WMO
cosponsored Programmes
Metadata and Quality Management System (QMS)
Ensure that the quality of observation is documented
Ensure the availability of associated comprehensive metadata for better
products and services
Ensure implementation of basic principles of the observing system/network
targeted Quality Management System
Provide data specifications describing required data or products
Integrate observing systems following the aims and rules defined within
WIGOS framework
Standardize access to data and products
Provide the tools for easy, single access for the user’s community to the
set of standards, guidelines and best practices for the management of
observing systems
(1)
FRAMEWORK FOR WIGOS STANDARDIZATION OF THE SURFACE-BASED COMPONENTS.
EC-WG/SG-WIGOS-2/Doc. 5.1
WIGOS metadata must be considered from user
point of view, taking into account user
requirements
Meaningful use of observations requires knowledge
about the conditions under which the observation
was made; WIGOS metadata is what provides
that supporting information.
User requirements be collected
(TT-WMD) Terms of Reference
In accordance with guidance and recommendations of Cg-XVI, EC and ICG-WIGOS-1:
1.To identify the information that is needed to allow the majority of users to use
WIGOS observations in appropriate contexts and in a defensible way
2.To create the WIGOS Core Metadata Standard that allows the essential information
to be exchanged unambiguously, regardless of the format used for the transfer
3.To define a mechanism for maintaining the WIGOS Core Metadata Standard,
including how metadata might be provided that is additional to the Core and coordinate
with the ICG-WIGOS Task Team on Regulatory Material (TT-WRM) on any appropriate
documentation as needed for WIGOS related Manual(s) and Guide(s)
4.To implement within the WIGOS Core Metadata Standard, and the WMO Core
Metadata Profile, a standard method of providing users with an indication of the quality
of the data, and to do so in a way that distinguishes with the quality management of
the data (“quality of the observation”) and ensuring that the user is able to identify
which applications the data are suitable for (“classification of the observation”)
5.To coordinate regularly with the ICG-WIGOS as needed and report at least annually
to the ICG-WIGOS on the progress
6.To complete its tasks and hand over additional requirements to its successor (if
required) in time for approval by Cg-17
What is a Metadata Standard?
A Community agreed upon declaration that provides a structure to
describe data with:
•
•
•
•
Common
Common
Common
Common
terms to allow consistency between records
definitions for easier interpretation
language for ease of communication
structure to quickly locate information
In search and retrieval, standards provide:
• A reliable and predictable format for computer interpretation
• A uniform summary description of the data set
Many metadata standards exist
• ISO 19115 Geographic information: Metadata
• Emphasis on geospatial data and services
• World Meteorological Organization Core Metadata Profile (WMO) – a
profile of ISO 19115
• Emphasis on meteorological data
• Content Standard for Digital Geospatial Metadata (CSDGM)
• Federal Geographic Data Committee (FGDC)
• Emphasis on geospatial data
• Directory Interchange Format (DIF)
• Emphasis on Earth science data
• Dublin Core Element Set
• Emphasis on web resources, publications
Definition of Metadata
Information to let you and others find, understand, and use
your data both now and in the future
“Data about data”
describes data
what are the data?
where are they for?
who is responsible?
what are the units?
Need to agree
what needs to be described
how they will be organised
what they will be called
So that
non-specialists can find data
applications can handle data
You get the idea … so, what’s needed?
Negotiated between you and your team (data producers), data
developers and the data archive
• Created / collected as part of workflows associated with lifecycle of
the data
•
Based on key questions, e.g., :
• How will I and others find my data?
• What will future scientists need to know, to understand and re-use the
data?
• What will data (product) developers need to create data services and
products that use my data?
• What does the data archive team need from the data producer team (as
the ones who know the data best) in order to preserve it for the long
term?
the WHO, WHAT, WHEN, WHERE, HOW, WHY about your data
Specific’s from different perspectives?
When submitting data:
• Why was the data was created?
• What limitations, if any, do the
data have?
• What does the data mean?
• Who should be cited if someone
publishes something that utilized
your data?
When receiving data:
• What are the data gaps?
• What processes were used for
creating the current data?
• Are there any fees associated with
the data?
• In what scale were the data
created?
• What do the values in the tables
mean?
• What software do I need in order
to read the data?
• What projection is the data in?
• Can I give this data to someone
else?
METADATA (1)
Good metadata are needed to ensure that the final data user has
no doubt about the conditions in which data have been recorded,
gathered and transmitted, in order to extract accurate
conclusions from their analysis.
Metadata have a key role in the process of creating such
datasets, as the knowledge of the station history provides
increased confidence in the statistical techniques employed to
ensure that the only variations that remain in a climate time
series are due to actual climate variability and change.
Meteorological data are influenced by a wide variety of
observational practices. Data depend on the instrument, its
exposure, recording procedures and many other factors. There is
a need to keep a record of all these metadata to make the best
possible use of the data. Additional information will be of great
advantage for the data users, as well as for the providers.
METADATA (2)
A good metadata archive helps the NMHSs in asset management
and other administrative procedures, as data existences and
observing conditions are kept in good order. It also can be said
that good metadata helps society to gain a better understanding
of weather and climate related processes, as well as climate
change.
GCOS Climate monitoring principles: “The details and history of
local conditions, instruments, operating procedures, data
processing algorithms and other factors pertinent to interpreting
data (i.e. metadata) should be documented and treated with the
same care as the data themselves.” (WMO 2002).
Metadata Types and Functions
• Provenance and Context
• Physical and conceptual environment
• Important for use and re-use
needed to understand data
• Discovery including
• Citation to enable proper credit, authority and identification
• Access and use restrictions
• Preservation / Archiving
• Facilitating utility of data over time
• Project Documentation
• Accumulation of important facts, guidelines, explanations about
the project
Metadata requirements for climate (1)
STATION IDENTIFIERS: name, aliases, station number or code in other
networks, opening/closing dates, type of station and station information contact
GEOGRAPHICAL DATA: latitude and longitude, elevation above mean sea level
and relocations
LOCAL ENVIRONMENT: at different scales, it is recommended to keep track
of several attributes as proximity and size of large water surfaces, urbanized
areas and mountain ranges. At toposcale terrain slope, both steepness and
direction, forests, crops and other roughness, nearby obstacles such as trees or
houses (at airports: airplanes) and proximity to irrigation.
INSTRUMENT EXPOSURE: obstacles and shelter, ground cover (roughness,
albedo, pavement, sand, etc.)
13
Metadata requirements for climate (2)
INSTRUMENTATION: instrument manufacturer, model of instrument,
with size and identification, output type and sensitivity, transducer type,
response time
DATA RECORDING AND TRANSMISSION: units and resolution, range of
recorded variations, response time and/or sampling time, averaging period
if applicable
oOBSERVING
procedures,
PRACTICES:
checking
observer,
instruments,
meteorological
replacing
operational
disposable
items,
housekeeping, observed elements, observing times.
oCORRECTIONS MADE BY OBSERVER: conversion tables or computer
programs should be kept
Metadata requirements for climate (3)
• DATA PROCESSING: the corrections made to the data, the procedures
of quality control applied, the adjustments made to ensure its homogeneity,
and the data estimated and filled in, after applying the interpolation
procedure selected.
•
UNITS: º Celsius o ºFahrenheit
•
SPECIAL CODES: like missing data, wrong value, trace precipitation,
non-precipitation, or accumulated precipitation
•
CALCULATIONS: time averaging (daily, monthly and so on) of
elements
•
QUALITY CONTROL: Gross error checking, Tolerance tests, Internal
consistency check
•
HOMOGENEITY ADJUSTEMENTS: elements, periods,applied test,
etc.
Information sources
WMO
Guide to climatological practices (WMO N° 100)
Guidelines on climate metadata and
homogenisation (WMO/TD-No.1185 )
Climate Database Management Systems
Specification Document
”Creating Documentation and Metadata”.
Nancy J. Hoebelheinrich, Stanford University Libraries
My experience tells me that the basic
metadata are very difficult to obtain. The
technicians are reluctant to write and we
need to specify very in a simple and clear
that metadata are needed before and
during the installation of a station and to
be closely monitored during the operation
thereof.
Thanks