Transcript Applications of BUFR (Why, when and how to use BUFR) Jeff Ator NOAA National Weather Service United States of America [email protected] WORLD METEOROLOGICAL ORGANIZATION RA II/VI Training.

Applications of BUFR
(Why, when and how to use BUFR)
Jeff Ator
NOAA National Weather Service
United States of America
[email protected]
WORLD METEOROLOGICAL ORGANIZATION
RA II/VI Training Seminar on Table-Driven Codes
Muscat, Oman, 10-14 December 2005
Why use BUFR?
• Allow the representation of new data elements
without changing underlying encoder/decoder
software
• Exchange large volumes of data efficiently with
other centres
• Easily include quality control and monitoring
information right alongside the data itself
• Efficiently store data for processing
• Efficiently store data in a local database
Applications – new data
• BUFR can be used to quickly encode and exchange
new types of data as they become available
– New data elements can be included immediately within
BUFR messages (eg. adding soil moisture to surface
observations) since BUFR is self-describing
• If an official element descriptor already exists, then the new BUFR
messages remain fully official for international exchange.
• Otherwise, a new official element descriptor can be developed and
approved through WMO. This process takes some time, but, in the
meantime, a local descriptor can be defined and used.
– No changes to encoder/decoder software are required since
BUFR is table-driven
Applications – data exchange
• BUFR is a very efficient mechanism for the
exchange of large volumes of data
• BUFR is:
– Efficient - uses comparatively little communications
bandwidth or disk space, especially if messages contain
multiple data subsets and the built-in data compression
option is used
– Standardized - readable by any other centre with a
standard BUFR decoder and the proper tables
– Flexible - nearly any meteorological data can be
encoded in BUFR, and BUFR can even be used to
exchange data from other disciplines
Applications – QC and
monitoring
• BUFR has several mechanisms for including
quality control and/or data monitoring information.
• Such information can be included by the originator
of the data, or it can even be added by a separate
processing centre at a later time!
• Some of the mechanisms are fairly complex.
– Refer to the BUFR guide and/or the WMO No. 306
Codes Manual itself for additional information.
Applications – data storage
• The efficiency of BUFR means that
substantial disk space savings are possible
over other data storage alternatives
• It is somewhat dependent on the actual data
in question, but large volume reductions
compared with alphanumeric or other
binary formats are possible
Applications – database
• One of the shortcomings of BUFR is the difficulty of searching
for specific data subsets or elements without first decoding the
entire message.
• One method of minimizing this problem is to store BUFR
messages in a database (eg. as a “BLOB”, thus preserving the
data storage advantages) together with separate indexing
information, enabling relevant data subsets or elements to be
quickly located.
– The optional section 2 of a BUFR message could even be used as a
backup location to store this indexing information, enabling the recreation of such information from the BUFR message itself if the
original database index were to later become corrupted!
– The data category and/or sub-category in section 1 of a BUFR message
can also be used as a quick filter when searching the database for all
messages containing a particular type of data (eg. radiosonde, buoy,
METAR, etc.)
Acknowledgements
Based on:
• Guide to WMO Table-Driven Code Forms
FM94 BUFR and FM95 CREX
(http://www.wmo.int/web/www/WMOCodes/Guides/BUFRCREXPreface_en.html)
Special thanks to:
• Charles Sanders BOM-Australia
• Simon Elliott EUMETSAT
• Joël Martellet WMO