Upload
miguel-keating
View
218
Download
1
Tags:
Embed Size (px)
Citation preview
CNGrid: Progress, achievements, and prospects
Prof. Qian DepeiDirector, Sino-German Joint Software Institute (JSI)
Beihang [email protected]
Beijing, June 21, 2005
Overview of CNGrid A key project supported by the
National High-Tech R&D Program (the 863 program), continued effort to the previous HPCE project
4-year project 100 million fund from the Chinese
government (MOST) Two to three times more associated
funds from local governments, application organizations, and industry
Major tasks of the project Developing a test-bed for the new information in
frastructure—China National Grid (CNGrid) About 8 grid nodes across the country Interconnected by available networks
Developing a 4TFlops high performance computer for the grid
Developing a set of grid software to support grid applications and grid operation and management
Developing several application grids Productive systems Demonstration of the future grid applications
Fund distribution Testbed: 10 million HPC: 40 million Software: 12 million Application: 27 million Strategic study and integration: 5 million Supporting center: 3 million Other: 3 million
Guidelines of the project Application driven
The most important driving force to the research
Technological leaping Emphasizing patents and
standards
Guidelines and strategies of the project (con.) Multilateral cooperation
Funds from 863 program Local government Application organizations Industry
Cooperation of Research teams Application teams Management people
Focus on the grid Complementary research efforts Integration into the grid testbed
The R&D supported 25 projects are supported so far
The strategic and integration technology study
testbed(9 teams) Grid software (1 team) Application grids (11 teams) High performance computers (2 teams) Testing of software and HPC (2 teams)
Task 1: Grid test-bed development Building the China National Grid (CDGrid) Eight nodes across the country
CAS Network Center (Beijing, major node) Shanghai Supercomputer Center (Shanghai, major node Tsinghua University (Beijing) Institute of Applied Physics and computational science (Beiji
ng) University of Science and Technology of China (Hefei, Anhui) Xi’an Jiaotong University (Xi’an, Shaanxi) NUDT (Changsha) Hong Kong University (Hongkong)
CNGrid operation and supporting center (CAS Network Center)
Resource Requirement Major nodes
>1 Tflops Equipped with the HPC developed by t
he project Ordinary nodes
>100GFlops Machines provided by the organization
supporting the node
grid nodes across China
Hefei
Xi’an
Beijing
Changsha
Shanghai
Hong Kong
CNGrid operation & support center Based on the Computer Network & Infor
mation Center of CAS Major tasks
Operational management of CNGrid User management Security management User training Technical support Software maintenance
Task 2: HPC development Two >4TFlops machines have been
developed Dawning 4000
AMD 64-bit Opteron processor 11 Tflops peak performance
Lenovo DeepComp 6800 Intel Itanium 2 processor 5.324 Tflops peak performance
Lenovo DeepComp 6800 Completed in Nov. 2003 2.6TB momory 81TB disks 4.183 Linpack performance (78.5%
efficiency) Ranked at 14 in Top500 ranking Installed at CAS Network Center in J
an. 2004
Lenovo 6800
Dawning 4000A Completed in Apr. 2004 8TFlops Linpack performance (70% effi
ciency) Installed at the Shanghai Supercomput
er Center in Nov. 2004
HPC measurement & evaluation center
Based on Tsinghua University Major tasks
Study of measurement methodology Benchmark development Services for system evaluation China Top 100 ranking jointly with ano
ther organization
Task 3: Grid software development
Grid system software Interfacing to heterogeneous systems Basic resource management &
scheduling Communication
Grid application development environment
Grid user environment Grid browser GSML
Grid software layerComp
Application
System
Resource
Securityservice
Jobservice
Usermanage
Dataservice
Gridmonitor
Information service
Grid resources (host, programs, instruments, file systems, database) wrapped as WS or GS
Data Transact
Grid S/W
Grid manage
UserEnvir.
Other
Grid software deployment Version 1.0 in Feb. 2004 Starting deployment in March
2004 on the test-bed
Deployment of grid software 1.0
Web 浏览器
GT3 core
网格服务器
Web Portal
GT3 core
网格服务器
网格应用
CNGrid GOS APIs
网格应用服务器
GT3
WS GS
GS
OGSA/OGSIWebSphere,
.Net
GS or wrappers
网格结点 A
本地资源驱动
网格结点 B
本地资源驱动
CNGrid 信息管理和资源管理
资源外包
CNGrid 信息管理和资源管理
Internet
CNGrid GOS APIs
基于网格软件的应用
GS orwrappers
其它APIs
注册
注册
网格操作环境
RB1
RB2
RB3
RA1
RA3
RA2
图7 网格软件部署结构
网格管理工具
Major features of the CNGrid software v2
V 1.1 V 2.0
Architecture
Simple virtualization
Resource routing
Advanced virtualization, grid proc, community
Loosely coupled: easy to change
Lower the appl. programming cost
User man
Life cycle man (reg., login, use, maintain, logout) Supporting roles
Security man.
Single login and authentication
Supporting authorization (community-based multi-grain access control)
Sys. Man.
Monitoring, accounting Monitoring, accounting, log file
Resource man.
Virtual & physical resource man., dynamic deploy
Effective, virtual resource management (strategy-based resource selection)
Data man.
Multi-point (not user transparent), physic view
Unified virtual view, location independent, access control
CNGrid GOS Client API
CNGrid GOS Core API
Grid application logicManagement tools for grid software, grid use environment
DynamicDeployment services
Nameservices
Data services
PKI based grid securitymechanism
GridApplication
Gridaddressbuses
Gridresources
Resource management
Application servers (i.e. Tomcat...)
Grid node hosts and storage resources
Resources encapsulation
Strategies and Access control engine
Grid file management Resource schedule strategy
Authorize strategy
Access control at service operational level
Grid process container and grid process
Securitymechanism of grid App
Access mechanism of general services
Securitymechanism of grid nodes
Grid resources driver
Grid router network
User management
Gridcommunity
Gridprocesscontainer
Community Service
Grid community
grid applications grid process container grid community services Forming Virtual organization Self-organizing and managing
Grid process runtime entity for accessing resources
on behalf of users running in the grid process container. Simplifies client programming
interface Hiding resource access details Result buffering Decouple grid applications from
resources
Grid process container provides execution and
management environment for grid process
Dynamically deployable
Grid address bus
Community is built upon grid address bus.
Grid address implements the binding of community resources and grid resources The location change of the grid
resources will not affect the grid community and grid applications
Grid use environment GSML is the application development language
for the end user XML based Easy to use flexible, robust access to grid resources Interactive GSML page composed of information cells
GSML runtime system GSML browser GSML editor Tools
Roadmap of grid software
Compatible to v1.1: sys.interfaces, Hosting Environment
Time table Feb. 2005: internal testing version May 2005: beta version, support demo applic
ations June 2005: v2.0
Task 4: Application grids development
Applications from selected areas Resource and Environment
GSG, DFG, SeisGrid Research
SDG, BioGrid, DDG, ChinaGrid Services
MSG, ITG Manufacturing
AviGrid, SimGrid
Geological Survey Grid Undertaken by China National
Geologic Survey Bureau Nation-wide productive grid Data exchange and sharing Resource evaluation services
Shorten the evaluation time from 3 years to 3 months
Delivering achievements in North China ground water resources
evaluation Mineral resources evaluation
Grid environment and resource deployment
Digital Forestry Grid Undertaken by the Institute of
Forest Resources Information Support turning the farmland
back to forests potential evaluation project planning project evaluation
Potential Analysis (major rivers and lakes )
Buffer region analysis result(A)
Farmland, unused land, and mountain (B)
Overlap result: A+B = C
Grade >25 ( D )
Second overlap result : B + D = E
The third time overlap: (C + E )
Meteorological Service Grid Undertaken by China National
Meteorological Center Multiple sites to form the research
platform for new weather forecasting model research
Provide new weather forecast services (time and location-specific)
Application development platform
·
CA·Myproxy·MDS GIIS·Globus 2.4
Node A( Oscar cluster )
Node B( Sunway cluster )
Node C( Galuxy III )
Man node
·Globus 2.4·OpenPBS.GRIS
·Globus 2.4·OpenPBS.GRIS
·Globus 2.4·OpenPBS.GRIS
Web server
·
CMAG Portal·Tomcat, Java SOAP·Globus, CoG·Myproxy client
Grid resource
32 dual CPU node 16 dual CPU node
MM5 mode ( 5km ) : simulation of rainfall of 30-36h, simulation result is 12h later than reality.
GRAPES_MESO mode ( 6km ) rain belt, center, and time close to real situation!
Result from GRAPES for forecasting snow (rain) in Dec. 21, 2004 ( grid : 30km )
Scientific Data Grid Undertaken by the Network &
Information Center of CAS Integration of more than 200
scientific databases from more than 40 institutes across the country by means of metadata management
Virtual astronomical observatory
Software module structure
Scientific Data Grid Portal
General metadata management tool
MDIS(LDAP)
interim XML
MD schema
Userpage
Process(Java bean)
XML engine
install & configure
universal, extensible
customizable
-metadata is tree-like and more flexible than fix-column tables, difficult to deal with on web UI
-use xml files to store interim results
New drug discovery grid Undertaken by Shanghai Institute of Mat
eria Medica CAS Compound screening for new drug discov
ery Speed up the process by computer simulatio
n Higher accuracy
Using HPC in P2P mode New drug for diabetes is under developm
ent and will enter clinic testing by the end of 2005
New Drug Discovery Grid Platform
神威机群Shanghai SCC
神威PC机群
PCPC
神威PC机群
曙光4000A
Beijing Medical Institute
Shanghai Institute of Materia Medica CAS
主服务器
Three layers of the grid
Internet
Main server
Sub-server
term term term
Internet
CN
CN
CN
CN
CN
CN
CN
CN
CN
Sub-server Sub-server
DDG Portal
Screening task allocation on different clusters
Bio-informatics Grid Undertaken by Genomics &
Bioinformatics Institute, CAS Provide computing, data, and
information grids for bio-information research in the country
Manufacturing Grid Undertaken by AVIC II of China Support distributed design,
simulation, and manufacturing CAD software sharing Data exchange and sharing Computing resource sharing
哈飞
洪都
北京 628
Manufacturing Grid
Aviation grid platform composed of three clusters in different locations
Manufacturing Grid
WindowsWindows
Windows Windows
Linux
Linux
LinuxWindowsAIX
Automatically locating resource when submitting jobs
网格内的所有结点的硬件共享
WAN
MultiClusterMultiCluster
submit
Licence poolLicence pool
Acquire Token
GLBGLBrun
Manufacturing Grid
Software sharing
After optimizationbefore optimization
Added skin Reduced skin
Manufacturing Grid
How to judge the success of Grid?
Application is the most important criteria Improves the quality? Increase the efficiency? Lower the cost? Reduce the complexity?
International Cooperation MoU with UK e-Science Planned connection with EGEE IBM and Platform participate CNGri
d project Exchange research results with Glo
bus team Looking forward to broad internatio
nal cooperation
What is going on? CNGrid may become part of the CNGI effort National Network Research Environment will be built upon C
NGrid The next five-year R&D plan is being designed. It is very possi
ble that Grid will be one of the key issues. Grid applications are emerging. We need successful applicati
ons to show the feasibility of Grid technology New trends in Grid research is emerging
Ubiquitous and pervasive Grid Mobile grid applications New operating systems, programming models, languages, and t
ools for Grid Business models and economical models for Grid services
Thank you!