PRAGMA: Cyberinfrastructure, Applications, People
Yoshio Tanaka (AIST, Japan)Peter Arzberger (UCSD, USA)
PRAGMA’s Founding Motivations
• The grid is transforming computing and collaboration
• The problem remains that the grid is too hard to use
• Middleware software needs to interoperate
• Science is an intrinsically global activity
IVOA
Overarching GoalsEstablish sustained collaborations
and
Advance the use of the grid technologies for applications
among a community of investigators working with leading institutions around the Pacific
Rim
PRAGMA
Working closely with established activities that promote grid activities or the underlying infrastructure,
both in the Pacific Rim and globally.
Key Activities and Outcomes
• Encourage and conduct joint (multilateral) projects that promote development of grid facilities and technologies
• Share resources to ensure project success• Conduct multi-site training• Exchange researchers
• Advance scientific applications• Create grid testbeds for regional e-science projects• Contribute to the international grid development efforts• Increase interoperability of grid middleware in Pacific
Rim and throughout the world
Act
ivit
ies
Ou
tcom
e s
Contents:2004-2005
• Overview• Accomplishments• PRIME• Working Groups• Institutions• References• Opportunities• Sponsors
http://www.pragma-grid.net
Working Groups: Integrating PRAGMA’s Diversity
• Telescience – including Ecogrid• Biological Sciences:
– Proteome Analysis using iGAP in Gfarm
• Data Computing– Online Data Processing of
KEKB/Belle Experimentation in Gfarm
• Resources– Grid Operations center
A collaborative effort in the fight against bio-terrorism
• Gfarm makes it possible to use iGAP to analyze the complete proteome (available 9/28/04) of the bacteria, Burkholderia mallei, a known biothreat agent, on distributed international resources.
• This is a collaboration under PRAGMA and the data is available through http://eol.sdsc.edu.
Source: Cindy Zheng
PRAGMA Grid Testbed – overview –
• Establish a regional wide testbed– Mainly focusing on computational Grid, later
introduce other resources– Provides an easy-access environment for
researchers, students, vendors, etc.– Testbed for software development and trial to have
evaluation of usability and to archive performance numbers
– Testbed for finding demonstrative / feasible applications
– is going to be a production Grid
PRAGMA Grid Testbed – unique features – • Truly (naturally) multi national/political/institutional VO
beyond boundaries– Not an application-dedicated testbed – general platform– Diversity of languages, culture, policy, interests, …
• Grid BYO – Grass roots approach– Each institution contributes his resources for sharing– Not a single source funded for the development
• Physical resources– Most contributed resources are small-scale clusters– Networking is there, however the bandwidth is not enough
• We can– have experiences on running international VO– verify the feasibility of this approach for the testbed development
言語と文字
Lessons Learned thru developments
• Difficulties are caused by technical problems as well as sociological/political problems
• Each site has its own policy– account management– firewalls– trusted CAs– …
• Differences in interests– Application, middleware, networking, etc.
• Differences in culture, language, etc.– Human interaction is very important
Building a Production Grid Thru Running Applications
• Routine-basis experiments– Ninf-G based TDDFT, http://pragma-
goc.rocksclusters.org/tddft/default.html– BioGrid,
http://pragma-goc.rocksclusters.org/biogrid/default.html
– iGAP over Gfarm
• Learn requirements/issues• Research/implement solutions• Improve middleware integrations
Source: Cindy Zheng
Status of Testbed in May, 2004Computational resource
26 organizations (10 countries)27 clusters (889 CPUs)Network performance is getting better.
Architecture, technologyBased on Globus Toolkit (mostly version 2)Ninf-G (GridRPC programming)Nimrod-G (parametric modeling system)SCMSWeb (resource monitoring)Grid Data FArm (Grid File System), etc.
Operation policyDistributed management (No Grid Operation Center)Volunteer-based administration
Less duty, less formality and less documents
Status of Testbed in May, 2004 (cont’d)
Questions???Ready for real science application?
Easy to use for every user?Reliable environment?Middleware stability?Plenty document?Enough security?
and etc.
Direction of PRAGMA Resource Working GroupDo “Routine-basis Experiments”
Try daily application runs for a long termFind out any problems and difficultyLearn what is necessary for the production grid
Overview of Routine-Basis Exp.Purpose
By daily runs of a sample application on PRAGMA testbed, find out and understand issues of the testbed operation for the real science application
Case study: the 1st applicationApplication
Time-Dependent Density Functional Theory (TDDFT)Software requirements of TDDFT are Ninf-G, Globus and Intel Fortran Compiler.
ScheduleJune 1, 2004 ~ August 31, 2004 (For 3 months)
Participants10 Sites (in 8 countries): AIST, SDSC, KU, KISTI, NCHC, USM, BII, NCSA, TITECH, UNAM193 CPUs (on 106 nodes)
Rough ScheduleMay June July Aug
SC’04
Sep Oct Nov
PRAGMA6
1st App. start
1st App. end
PRAGMA7
2nd App. startSetup Resource Monitor (SCMSWeb)
1. Apply account
2. Deploy application codes
3. Simple test at local site
4. Simple test between 2 sites
Join in the main executions after all’s done
2 sites 5 sites 8 sites 10 sites
2nd user start executions
Summary of the experimentNumber of executions : 43Execution time (Total) : 50.4 days (Max) : 6.8 days (Ave) : 1.2 days
Lessons Learned from the point of view of testbed operation
Need to reduce user’s effortMore documentsNeed common configurations on each clusterShould know the requirements of many applicationsMore collaboration among administrators from each site
Need to monitor site-to-site networkProblem: Very unstable network between AIST and some
sitescollaboration with networking community is important
e.g. APAN NOC
Need to monitor each cluster’s statusProblem: Don’t know maintenance? trouble? network down?
SCMSWeb will help this issue.Periodical status tests
Network Weather Maphttp://mrtg.koganei.itrc.net/mmap/grid.html
Thanks: Dr. Hirabaru and APAN Tokyo NOC team
Summary
• Dream big – build the imagination of those involved
• Start small - build trust• Stay concrete – build infrastructure• Focus on people – build community
• Opportunities: – Networking and Measurements– Testbed: Mutual learning– Applications: Geosciences, Astronomy, Ecology,
Biomedicine, …– Exchange: Undergraduates and Graduates– Participate in Meetings and between meetings
Thank you
http://www.pragma-grid.net
Office of International Science and Engineering Division of Shared Cyberinfrastructure
Division of Biological Infrastructure