View
217
Download
4
Category
Tags:
Preview:
Citation preview
Cluster Programming Technology and its Application
in Meteorology
Computer and AutomationResearch InstituteHungarian Academy of Sciences
HungarianMeteorologicalService
Silicon Computers Ltd.
Background• Hungarian Meteorological Service developed
MEANDER (MEsoscale Analysis Nowcasting and DEcision Routines)– crucial task in the protection of life and property (storm warning at
Lake Balaton, weather warnings for aviation...)– based on incoming meteorology information and computational
intensive methods
• MTA SZTAKI developed P-GRADE parallel programming environment – efficient, graphical support for the entire life cycle of parallel program
development
• Cluster programming technology and its applications in meteorology– joint project of MTA SZTAKI, Hungarian Meteorological Service and
Silicon Computers Ltd.– supported by Research & Development Division, Ministry of
Education
Goal: Analysis of all the available meteorology information producing parameters on a high resolution regular mesh
(10km--> 1km) ultra-short range forecast (up to 6 hours)
Application: Forecasting dangerous weather situations (storms, fog, etc.)
Meteorology information: surface level measurements, high-altitude measurements, radar, satellite, lightning, results of previous computed models, etc.
Basic parameters: pressure, temperature, humidity, wind, …
Derived parameters: type of clouds, visibility, …
MEANDER Program Package
Structure of MEANDERFirst guess dataALADDIN
SYNOP data Satellite
dataRadardata
CANARI DELTA analysi
sBasic fields: pressure, temperature, humidity, wind
Derived fields: Type of clouds, visibility, etc.
BASIC GRID
Radar to grid
Satellite to grid
“Present” weather
Type of clouds
Overcast
Visibility
Rainfall phase
Visualization For meteorologist:HAWKFor users:
GIF
Lightning
decode
36000 km
Satellite raw image
Receivingimage
Transformation & Interpolation &
Processing
Processing:altitude of clouds Transformation Interpolation to basic GRID
Processing of satellite images
THE PROBLEM
MEANDER Sequential
code
C, C++ Fortran
MEANDERParallelversion
PC clasterSGI Origin 2000
SUN E10000
Parallelisation?
Debugging?
Performance?
GRAPNELGRAPNEL graphical language / GRP2CGRP2C pre-compiler
Unix-like operating systems & C, C++, Fortran programming libraries
PVM or MPI message passing library
DIWIDEDIWIDE
distributed debugger
GREDGREDgraphical
editor
GRMGRM monitor
ddistributedistributed ccheckpointingheckpointing
tooltool
GRP_CHKPTGRP_CHKPT
GRP_MMGRP_MM mmigration moduleigration module
GRP_LBGRP_LB lload oad bbalanceralancer PROVEPROVEperformancevisualization
tool
TLCTLC modelchecker
MacrostepMacrostepdebugger
The solution: P-GRADE P-GRADE development
environment
Results: temperature and wind at 850 hPa level
Results: temperature and wind at 850 hPa level
MEANDER: 3D FIELDS
2D analysis
Radardata
Satellitedata
Delta 3Danalysis
...computes the basic meteorological fields:
pressure, temperature, humidity, wind velocity and direction
for a high resolution 3D
mesh (10km -1km)
Implementation of
DELTA analysis in
P-GRADE
Fortranseq. code
P-GRADE version of MEANDER for clusters & supercomputers
25 x
4 x
10 x
25 x
25 x
20 x
Live demo (5th DataGRID conference)
34 MbitShared
PERL-GRIDCONDOR-PVM
job
11/5 MbitDedicated
P-GRADEPERL-GRID
jobftp.met.hu
netCDF
512 kbitShared
netCDFinput
Parallel execution and GRM
GRMTRACE &Results
GRMTRACE &Results
PROVE performance visualization
Edit, debugging
Performance-analysis
Testing,Execution
Resource requirements of P-GRADE
Advantages of P-GRADE environment Efficient support for each stage of parallel
program development Fast parallelisation of existing algorithms
Reusability of sequential code Hiding of low level communication functions Unified and integrated graphical concept Predefined communication templates Support for hierarchical design
Even non-professional programmers can use it
(steep learning curve) Portability from supercomputers to PC clusters
Support for cluster computing
More information: www.lpds.sztaki.hu
New facilities in P-GRADEfor long-running parallelapplications:
• Distributed checkpointing• Process migration• Dynamic load balancing• Fault-tolerance execution
I. Execution withoutload balancing
III. Load balancing& migration
IV. Result:Faster execution
II. Checkpointing
Recommended