Transcript Cluster Programming Technolgy and its Application in

Cluster Programming Technology
and its Application in Meteorology
Computer and Automation
Research Institute
Hungarian Academy of Sciences
Silicon
Computers Ltd.
Hungarian
Meteorological
Service
Background
• Hungarian Meteorological Service developed
MEANDER (MEsoscale Analysis Nowcasting and DEcision Routines)
– crucial task in the protection of life and property (storm warning
at Lake Balaton, weather warnings for aviation...)
– based on incoming meteorology information and computational
intensive methods
• MTA SZTAKI developed
P-GRADE parallel programming environment
– efficient, graphical support for the entire life cycle of parallel
program development
• Cluster programming technology and its applications in meteorology
– joint project of MTA SZTAKI, Hungarian Meteorological Service
and Silicon Computers Ltd.
– supported by Research & Development Division, Ministry of
Education
MEANDER Program Package
 Goal: Analysis of all the available meteorology information
 producing parameters on a high resolution regular mesh
(10km--> 1km)
 ultra-short range forecast (up to 6 hours)
 Application: Forecasting dangerous weather situations (storms,
fog, etc.)
 Meteorology information: surface level measurements, highaltitude measurements, radar, satellite, lightning, results of
previous computed models, etc.
 Basic parameters: pressure, temperature, humidity, wind, …
 Derived parameters: type of clouds, visibility, …
Structure of MEANDER
First guess data
ALADDIN
SYNOP
data
Satellite
data
Radar
data
Lightning
CANARI
DELTA
analysis
decode
Rainfall
phase
Visibilit
y
Overcas
t
Type of
clouds
“Present
”
weather
Basic fields:
pressure,
temperature,
humidity, wind
Derived fields:
Type of clouds,
visibility, etc.
Radar
to
grid
Satellite
to grid
BASIC GRID
Visualization
For users:
GIF
For
meteorologist
:HAWK
Processing of satellite images
Satellite raw image
36000 km
Receiving
image
Transformation &
Interpolation &
Processing
Transformation
Interpolation to basic GRID
Processing:
altitude of clouds
THE
MEANDER
Sequential
code
PROBLEM
Parallelisation?
Debugging?
C, C++
Fortran
Performance?
MEANDER
Parallel
version
PC claster
SGI Origin 2000
SUN E10000
The solution:
P-GRADE development environment
GRP_LB
load balancer
GRP_MM
migration module
GRP_CHKPT
distributed
checkpointing
tool
GRM
TLC model
PROVE
performance
visualization
tool
GRED
checker
graphical
editor
monitor
Macrostep
debugger
DIWIDE
GRAPNEL graphical language / GRP2C prePVM or MPI message passing library
distributed
compiler debugger
Unix-like operating systems & C, C++, Fortran programming libraries
MEANDER: 3D FIELDS
2D
analysis
Radar
data
Satellite
data
Delta 3D
analysis
...computes the basic
meteorological fields:
pressure, temperature,
humidity, wind velocity
and direction
for a high resolution 3D
mesh (10km -1km)
Results:
temperature and
wind at 850 hPa
level
Fortran
seq. code
Implementation
of
DELTA analysis in
P-GRADE
P-GRADE version of MEANDER
for clusters & supercomputers
25 x
25 x
4x
25 x
10 x
20 x
(5th
Live demo
DataGRID conference)
P-GRADE
PERL-GRID
11/5 Mbit job
Dedicated
ftp.met.hu
netCDF
netCDF
input
512 kbit
Shared
34 Mbit
Shared
job
GRM
TRACE &
Results
PERL-GRID
CONDOR-PVM
Parallel execution and GRM
GRM
TRACE &
Results
PROVE performance visualization
Resource requirements of P-GRADE
Edit,
debugging
Performanceanalysis
Testing,
Execution
Advantages of P-GRADE environment
 Efficient support for each stage of parallel
program development
 Fast parallelisation of existing algorithms
 Reusability of sequential code
 Hiding of low level communication functions
 Unified and integrated graphical concept
 Predefined communication templates
 Support for hierarchical design
 Even non-professional programmers can use it
(steep learning curve)
 Portability from supercomputers to PC clusters
Support for cluster computing
III. Load balancing
& migration
New facilities in P-GRADE
for long-running parallel
applications:
• Distributed checkpointing
• Process migration
• Dynamic load balancing
• Fault-tolerance execution
I. Execution without
load balancing
IV. Result:
Faster execution
II. Checkpointing
More information: www.lpds.sztaki.hu