Transcript Integrated Data Management System Alexander Besprozvannykh Russian Federation CDMS CliWare line • CliWare Lite (Climate data management system) • CliWare Real-Time (GTS data management system) •

Integrated Data
Management System
Alexander Besprozvannykh
Russian Federation CDMS
1
CliWare line
• CliWare Lite (Climate data management system)
• CliWare Real-Time (GTS data management system)
• CliWare Coral (Integrated data management)
Russian Federation CDMS
2
CliWare Lite
•
•
•
•
•
•
Simple CDBMS
Oriented on Desktop or Small work group
Based on only open source projects
Uses PostgreSQL DBMS
Has all applications, including user tasks
Metadata implementation for climate data sets
Russian Federation CDMS
3
CliWare Real-Time
• Oriented to Internet or Work group
• Oracle, MS Sql Server, SAPDB or PostreSQL DBMS
• Real time GTS data processing with decoding FM
codes: Synop, Ship, Upper air, CLIMAT, Bathy, Tesak,
Buoys, Tracob, Coastal, Grib, Metar, Speci, Taf, Hydro,
Agro
• Process about 170000 observation per day
• Maintains up to two years period of database
• Based on Web technology
Russian Federation CDMS
4
CliWare Integrated data management
Russian Federation CDMS
5
Main tasks
• Data and products collection;
• Producing basic data and products;
• Data and products dissemination;
Russian Federation CDMS
6
Meteo Office today
Data processing technology
Russian Federation CDMS
7
Integration into data processing
Data collection
Data production places
Complete set of the
input data
Complete set of the
output data
Data dissemination
Russian Federation CDMS
8
Data and products collection
•
•
•
•
•
•
•
•
WMO Global Telecommunication System;
Internet request-response facility;
Internet Web push facility;
E-mail facility;
Local file system grabber facility;
Predefined data collection based on timetable.
Quality control of the data
Data collection monitoring;
Russian Federation CDMS
9
Producing basic data and products:
• Providing
basic
calculations
and
summarizing hydro-meteorological data;
• Converting data into web pages, charts,
GIS products
• Statistics.
• Coding and decoding of data.
• Format converting
Russian Federation CDMS
10
Data dissemination:
•
•
•
•
•
•
Global telecommunication system;
Internet request-response facility;
Internet Web push facility;
E-mail facility;
Local file system facility.
Predefined data dissemination
timetable.
Russian Federation CDMS
based
on
11
Metadata implementation:
• Human facility to create and edit metadata;
• Automatic generation of metadata about of real
time data and products
• Implementation of metadata about archives and
offline data
• Search facility.
Russian Federation CDMS
12
Using of Metadata :
• Metadata is uniform description of different types
of data, presented in different data format.
• Metadata describe item of data that can be
maintained by system (observation, data file,
document, product and so on)
Russian Federation CDMS
13
WMO Metadata
Profile
CliWare Metadata
Profile
• Does a dataset on a
• Reference name
specific topic exist
(‘what’)?
(‘what’)?
•
For
a
specific
• For a specific place
place(‘where’)?
(‘where’)?
• For specific date or
• For a specific date or
period (‘when’)?
period (‘when’)?
• A point of contact to
• URL address of contacct
learn more about or
to receive online data
order the dataset
(‘who’)?
(‘who’)?
Russian Federation CDMS
14
WEB MAP Server
Russian Federation CDMS
15
WEB Feature server (GML)
<?xml version="1.0" encoding="windows-1251"?>
<wmo:SynopBulletin xmlns:gml="http://www.opengis.net/gml"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:wmo="http://cliware.meteo.ru/wmo"
xmlns:cw="http://cliware.meteo.ru/cw"
xsi:schemaLocation="http://cliware.meteo.ru/deegreewfs?request=DescribeFeatureType&amp;typename=wmo:Synop">
<wmo:SynopBulletinMember>
<wmo:Synop gml:id="Z_26898_c_obns_20040721060000.gml">
<gml:timeStamp><gml:TimeInstant><gml:timePosition>2004-0721T06:00:00</gml:timePosition></gml:TimeInstant></gml:timeStamp>
<gml:location><gml:Point srsName = "http://www.opengis.net/gml/srs/epsg.xml#4326">
<gml:coordinates>34.19,53.15</gml:coordinates>
</gml:Point></gml:location>
<wmo:Name>BRJANSK</wmo:Name>
<wmo:Station>26898</wmo:Station>
<wmo:Date>2004-07-21T06:00:00</wmo:Date>
<wmo:dd>310</wmo:dd>
<wmo:ff>2</wmo:ff>
<wmo:TTT>19.1</wmo:TTT>
<wmo:TdTdTd>12.9</wmo:TdTdTd>
<wmo:UUU/>
<wmo:P0P0>993.4</wmo:P0P0>
<wmo:PPPP>1018.6</wmo:PPPP>
<wmo:h/>
<wmo:VV/>
<wmo:N/>
<wmo:a>7</wmo:a>
<wmo:ppp>.5</wmo:ppp>
<wmo:RRR/>
<wmo:RRRTime/>
<wmo:ww/>
<wmo:Nh>0</wmo:Nh>
<wmo:CL>0</wmo:CL>
<wmo:CM>0</wmo:CM>
<wmo:CH>9</wmo:CH>
</wmo:Synop>
Russian Federation CDMS
16
</wmo:SynopBulletinMember>
</wmo:SynopBulletin>
HTML Server
Russian Federation CDMS
17
FTP Server
Station;Date;Name;VV ;Q;N ;Q;dd ;Q;TTT ;Q;TdTdTd ;Q;P0P0 ;Q;PPPP ;Q
5593760;2004-09-01 00:00;Moscow;;;0;1;0;1;15.8;1;15.1;1;1002.1;0;1020.6;1
5593760;2004-09-01 03:00;Moscow;95;1;0;1;0;1;15.3;1;15;1;1001.6;0;1020.1;1
5593760;2004-09-01 04:00;Moscow;96;1;6;1;0;1;16;1;15.3;1;;;1020.1;1
5593760;2004-09-01 05:00;Moscow;96;1;3;1;70;1;17.4;1;16.9;1;;;1019.9;1
5593760;2004-09-01 06:00;Moscow;96;1;2;1;600;3;18.5;1;16.5;1;1001.5;0;1019.8;1
5593760;2004-09-01 07:00;Moscow;97;0;0;1;80;1;20.7;1;17.4;1;;;1019.5;1
5593760;2004-09-01 08:00;Moscow;97;0;0;1;70;1;23;1;18.1;1;;;1019.1;1
5593760;2004-09-01 09:00;Moscow;97;0;3;1;70;1;25.2;1;19.5;1;1000.7;0;1018.6;1
5593760;2004-09-01 10:00;Moscow;97;0;5;1;90;1;25.2;1;19.3;1;;;1018.8;1
5593760;2004-09-01 11:00;Moscow;97;0;5;1;90;1;26.3;1;18.8;1;;;1018.3;1
5593760;2004-09-01 12:00;Moscow;97;0;6;1;100;1;26.1;1;19.3;1;1000.1;0;1017.9;1
5593760;2004-09-01 13:00;Moscow;97;1;7;1;0;1;23.8;1;19.7;1;;;1018.9;1
5593760;2004-09-01 14:00;Moscow;97;0;7;1;0;1;24.2;1;21;1;;;1017.5;1
5593760;2004-09-01 15:00;Moscow;97;1;5;1;0;1;22.7;1;19.7;1;999.2;0;1017.2;1
5593760;2004-09-01 16:00;Moscow;97;0;7;1;0;1;22.4;1;19.2;1;;;1017.2;1
5593760;2004-09-01 17:00;Moscow;96;1;3;1;0;1;20.6;1;18.1;1;;;1017.5;1
5593760;2004-09-01 18:00;Moscow;;;4;1;0;1;19.3;1;18.5;1;999.5;0;1017.8;1
5593760;2004-09-01 21:00;Moscow;;;2;1;0;1;17.1;1;16.4;1;999.4;0;1017.8;1
5593760;2004-09-02 00:00;Moscow;;;0;1;0;1;16.4;1;15.7;1;999.3;0;1017.7;1
5593760;2004-09-02 03:00;Moscow;96;1;8;1;0;1;17.8;1;16.3;1;998.7;0;1017.1;1
5593760;2004-09-02 04:00;Moscow;96;1;7;1;60;1;16.9;1;16.2;1;;;1017.2;1
5593760;2004-09-02 05:00;Moscow;96;1;8;1;0;1;17.2;1;16.5;1;;;1017;1
Russian Federation CDMS
18
Chart server
Russian Federation CDMS
19
GTS Data server
FM-12x
ZCZC
SMRS10 RUMS
AAXX 21061
26898 //998
26997 //966
27595 //797
27612 //997
27612 //997
27703 //997
27707 //875
27719 //997
27786 //597
27906 //660
27928 //997
27947 //496
27962 //295
27983 //598
28506 //997
28711 //698
28807 //597
34009 //997
34047 //562
34123 //980
34152 //297
34172 //350
34186 //598
34247 //567
34336 //598
34391 //697
35121 //598
NNNN
161647
33102
22901
31405
63202
63202
23505
43602
33402
61406
33404
63404
83205
80107
71405
00902
21203
41403
23303
82907
53003
82703
82004
71305
83104
72603
71801
51002
10191
10193
10220
10183
10183
10200
10186
10195
10191
10205
10182
10144
10153
10172
10234
10226
10194
10198
10153
10162
10144
10150
10184
10189
10199
10184
10263
20129
20154
20153
20111
20111
20122
20139
20111
20125
20155
20110
20135
20151
20140
20159
20139
20142
20121
20137
20120
20137
20137
20125
20128
20128
20144
20130
39934
39975
3////
39984
39984
3////
39896
39915
39948
39936
39941
39940
3////
39973
3////
3////
30068
39877
39914
39955
39886
39884
39959
30007
30031
30075
39980
40186
40183
40117
40168
40168
40168
40175
40151
40099
40163
40138
40092
40042
40094
40135
40120
40116
40165
40087
40130
40073
40067
40089
40114
40129
40104
40114
Russian Federation CDMS
57005 80009=
57005 80001=
52011 83500=
54000 80001=
54000 80001=
58003 80001=
54000 84030=
58003 80001=
52013 86800=
57007 83500=
57012 69902 84032=
56002 60402 78082 883//=
50005 61602 75185 886//=
58015 87300=
52018=
52004 82300=
52007 84200=
57002 82030=
57011 60202 78082 889//=
57007 83051=
57008 60302 76162 8872/=
53003 61502 78082 883//=
58003 69932 86311=
57005 8257/=
57007 82334=
52007 60202 87240=
53002 81901=
20
Metadata server
Russian Federation CDMS
21
Holding the entire set of data and products
All input data is saving in tables of relational
Database and file system. Period of holding real-time data
is two years. Input flow is around
170 000 messages per day.
Russian Federation CDMS
22
С
б
о
р
д
а
н
н
ы
х
2007-01-15
з
а
о
д
н
и
с
у
т
к
и
2005-02-21
Russian Federation CDMS
23
Мониторинг сбора данных
(Synop)
Russian Federation CDMS
24
Buoy
2005-02-15
2007-01-15
(21290 в день)
Russian Federation CDMS
25
Судовые
наблюдения
(1-10 января)
2005-01
(01-10)
2007-01
(01-10)
Russian Federation CDMS
26
Cruise
Russian Federation CDMS
27