Transcript DATA RESCUE IN AFRICA: SYNERGY WITH NATIONAL AND REGIONAL PROJECTS Climate&Environment Department ACMAD International Workshop on the Rescue and Digitization of Climate Records for countries in.

DATA RESCUE IN AFRICA:
SYNERGY WITH NATIONAL AND
REGIONAL PROJECTS
Climate&Environment Department
ACMAD
International Workshop on the Rescue and Digitization
of Climate Records for countries in West Africa
19-23 November 2012, Accra, Ghana
OUTLINE
-
-
-
-
Historical Background
ACMAD plans for African Data Rescue&
Management
Proposed support to other DARE projects
in Africa
Concluding Remarks
HISTORICAL BACKGROUND
With funding from Belgium, WMO sent experts in 46
African countries to locate and image hydromet data
 Microfiches of over 30 millions old observations
from about 1000 sites were produced with copies to
countries
 Microfiches were handed over to ACMAD around the
mid 90s
 Discussions with IEDRO and recently with IRI at
Columbia University for imaging and digitalization
Call for bid and selection of a firm to procure data
rescue equipments and related training at ACMAD
( Aug-Oct 2012)
ACMAD DATA RESCUE PLANS

Training of ACMAD/country experts on scanning of
microfiches, creation of images library , data keying and
entry into the national and continental databases

Inventory , scan all microfiches ( about 100
thousands) and save as digital images

Key in the data and enter them into the ACMAD/Regional
Climate Centre’s African climate database
ACMAD DATA RESCUE PLANS ( support to country and subregional DARE projects)

Provide on the job training on data rescue at ACMAD
for countries

Countries data Rescue with seconded experts and visiting
scientists programmes at ACMAD
Each country expert will bring back a copy of digitized
Data in a Climsoft managed database interfaced with
application tools ( RCLIMDEX, CPT, Excell, GIS macros…)
for climate monitoring and forecasting products

Organize workshops to produce climate indices, Atlases
and other added value products using rescued data
 DEVELOPMENT AND USE OF THE CONTINENTAL CLIMATE
DATABASE AT ACMAD
Data base design and architecture
Ingestion of Daily Data
Ingestion of Synoptic data
 Curent status of the Data base
DATA BASE DESIGN AND ARCHITECTURE





Data model is the model for CLIMSOFT version 3, which was last updated
in 2010
Data model allows storage of different data formats: numeric, text,
Boolean (Yes/No) and graphical images
Though originally designed in MS Access, the database design has been
ported to MySQL and PostgreSQL. It can be ported to any other RDBMS.
For the ACMAD climatological database, the open source PostgreSQL
was chosen for its efficiency, stability and robustness.
PostgreSQL data storage capacity is on the order of Terabytes (TB).
INGESTION OF DAILY SYNOP DATA



CLIMSOFT ingest GTS data via FTP.
The data are automatically imported into the
CLIMSOFT database at a scheduled time.
This module for the ingestion of synoptic
data became operational in June 2012.
Current Database Status




Historical daily data from the NOAA/NCDC FTP
site date back to 1972.
GTS SYNOP data imported from GTS data
reception system at ACMAD.
Total database size is approximately 24 GB
Very Basic QC is carried out. We are planning to
implement more rigorous Quality checks.
PROVISION OF FORMATED DATA
FOR CLIMATE OPERATIONS




Table of 10-day summaries for selected stations
for all Africa input to dekadal Bulletins
Data output in RClimDex, CPT and Instat inputs
formats.
We have plans to include other outputs in
formats required by any climate application tool



Long-term climatological means generated
for all GTS stations in Africa.
Climatological means available for the period
1981-1990, 1991-2000, 2001-2010, 19812010.
Data products in XML format and can be
viewed with any web browser
RCLIMDEX and INSTAT input data format from Chad climsoft
Database at ACMAD
1600
1400
1200
CUM71-00
1000
CUM91
800
CUM2001
600
CUM2010
400
CUM2011
200
CUM2012
358
341
324
307
290
273
256
239
222
205
188
171
154
137
120
103
86
69
52
35
18
1
0
800
700
600
500
400
300
200
100
0
ANNEE 2012
NORMALE

Multiple key entry for countries data using
on the job training, secondment and visits
on data rescue to reduce possible errors
 Continue
development of CLIMSOFT for
station data by building interfaces with the
web, application tools, other environmental
database management systems as well as
GIS software
 Expand CDMS training across Africa
integrated into workshops for generating
value added information for climate data