Upload
karik
View
46
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Networks, Grids and the Digital Divide in HEP and Global e-Science. Harvey B. Newman ICFA Workshop on HEP Networking, Grids, and Digital Divide Issues for Global e-Science Daegu, May 23 2005. Large Hadron Collider CERN, Geneva: 2007 Start. pp s =14 TeV L=10 34 cm -2 s -1 - PowerPoint PPT Presentation
Citation preview
Networks, Grids and the Digital Divide inNetworks, Grids and the Digital Divide in HEP and Global e-Science HEP and Global e-Science
Harvey B. NewmanHarvey B. Newman ICFA ICFA Workshop on HEP Networking, Grids, Workshop on HEP Networking, Grids, and Digital Divide Issues for Global e-Scienceand Digital Divide Issues for Global e-Science
Daegu, May 23 2005Daegu, May 23 2005
TOTEM pp, general purpose; HI
pp, general purpose; HI
LHCb: B-physics
ALICE : HI
pp s =14 TeV L=1034 cm-2 s-1
27 km Tunnel in Switzerland & France
Large Hadron Collider CERN, Geneva: 2007 Start
Large Hadron Collider CERN, Geneva: 2007 Start
CMS
Atlas
Higgs, SUSY, Extra Dimensions, CP Violation, QG Plasma, … the Unexpected
5000+ Physicists 250+ Institutes 60+ Countries
LHC Data Grid Hierarchy:Developed at Caltech
LHC Data Grid Hierarchy:Developed at Caltech
Tier 1
Tier2 Center
Online System
CERN Center PBs of Disk;
Tape Robot
FNAL CenterIN2P3 Center INFN Center RAL Center
InstituteInstituteInstituteInstitute
Workstations
~150-1500 MBytes/sec
~10 Gbps
1 to 10 Gbps
Tens of Petabytes by 2007-8,at ~100 Sites.
An Exabyte ~5-7 Years later.Physics data cache
~PByte/sec
10 - 40 Gbps
Tier2 CenterTier2 CenterTier2 Center
~1-10 Gbps
Tier 0 +1
Tier 3
Tier 4
Tier2 Center Tier 2
Experiment
CERN/Outside Resource Ratio ~1:2Tier0/( Tier1)/( Tier2) ~1:1:1
Emerging Vision: A Richly Structured, Global Dynamic System
ICFA and Global Networks ICFA and Global Networks for Collaborative Sciencefor Collaborative Science
ICFA and Global Networks ICFA and Global Networks for Collaborative Sciencefor Collaborative Science
National and International Networks, with rapidly National and International Networks, with rapidly increasing capacity and end-to-end capability increasing capacity and end-to-end capability are essential, forare essential, forThe daily conduct of collaborative work in bothThe daily conduct of collaborative work in both
experiment and theory experiment and theoryExperiment development & construction Experiment development & construction
on a global scale on a global scaleGrid systems supporting analysis involving Grid systems supporting analysis involving
physicists in all world regions physicists in all world regions The conception, design and implementation of The conception, design and implementation of
next generation facilities as “global networks” next generation facilities as “global networks” ““Collaborations on this scale would never have been Collaborations on this scale would never have been
attempted, if they could not rely on excellent networks”attempted, if they could not rely on excellent networks”
Challenges of Next Generation Science in the Information AgeChallenges of Next Generation Science in the Information Age
Flagship Applications Flagship Applications High Energy & Nuclear Physics, AstroPhysics Sky Surveys:High Energy & Nuclear Physics, AstroPhysics Sky Surveys:
TByte to PByte “block” transfers at 1-10+ Gbps TByte to PByte “block” transfers at 1-10+ Gbps Fusion Energy:Fusion Energy: Time Critical Burst-Data Distribution; Time Critical Burst-Data Distribution;
Distributed Plasma Simulations, Visualization, Analysis Distributed Plasma Simulations, Visualization, Analysis eVLBI:eVLBI: Many (quasi-) real time data streams at 1-10 Gbps Many (quasi-) real time data streams at 1-10 Gbps BioInformatics, Clinical Imaging:BioInformatics, Clinical Imaging: GByte images on demand GByte images on demand
Advanced integrated Grid applications rely on reliable, Advanced integrated Grid applications rely on reliable, high performance operation of our LANs and WANs high performance operation of our LANs and WANs
Analysis Challenge:Analysis Challenge: Provide results with rapid turnaround, Provide results with rapid turnaround, over networks of varying capability in different world regionsover networks of varying capability in different world regions
Petabytes of complex data explored and analyzed by 100s-1000s Petabytes of complex data explored and analyzed by 100s-1000s of globally dispersed scientists, in 10s-100s of teamsof globally dispersed scientists, in 10s-100s of teams
Huygens Space Probe Lands on Titan - Monitored by 17 telescopes in Au, Jp, CN, US
Huygens Space Probe Lands on Titan - Monitored by 17 telescopes in Au, Jp, CN, US
In October 1997, the Cassini spacecraft left Earth to In October 1997, the Cassini spacecraft left Earth to travel to Saturntravel to Saturn
On Christmas day 2004, the Huygens probe separated On Christmas day 2004, the Huygens probe separated from Cassini from Cassini
On 14 January 2005 it started its descent through the On 14 January 2005 it started its descent through the dense (methane, nitrogen) atmosphere of Titan dense (methane, nitrogen) atmosphere of Titan (speculated to be similar to that of Earth (speculated to be similar to that of Earth billions of years ago) billions of years ago)
The signals sent back from Huygens to Cassini were The signals sent back from Huygens to Cassini were
monitored by 17 telescopes in Australia, China, Japan monitored by 17 telescopes in Australia, China, Japan and the US to accurately position the probe to within a and the US to accurately position the probe to within a kilometre (Titan is ~1.5 billion kilometres from Earth)kilometre (Titan is ~1.5 billion kilometres from Earth)
Courtesy G. McLaughlin
Australian eVLBI data sent over high speed links to the Netherlands
Australian eVLBI data sent over high speed links to the Netherlands
The data from two of the Australian telescopes were The data from two of the Australian telescopes were transferred to the Netherlands over the transferred to the Netherlands over the SXTransport SXTransport and and IEEAF IEEAF links, and links, and CA*net4 CA*net4 using using UCLPUCLP, and were the first to , and were the first to be received by JIVE (Joint Institute for VLBI in Europe), the be received by JIVE (Joint Institute for VLBI in Europe), the correlator sitecorrelator site
The data was transferred at an average rate of 400MbpsThe data was transferred at an average rate of 400Mbps (note 1Gbps was available) (note 1Gbps was available)
The data from these two telescopes were reformatted and The data from these two telescopes were reformatted and correlated within hours of the end of the landingcorrelated within hours of the end of the landing
This early correlation allowed calibration of the data This early correlation allowed calibration of the data processor at JIVE, ready for the data from other telescopes processor at JIVE, ready for the data from other telescopes to be addedto be added
Significant int’l collaborative effort: 9 OrganizationsSignificant int’l collaborative effort: 9 OrganizationsG. McLaughlin, D.
Riley
ICFA Standing Committee on Interregional Connectivity (SCIC)
ICFA Standing Committee on Interregional Connectivity (SCIC)
Created in July 1998 in Vancouver ; Following ICFA-NTFCreated in July 1998 in Vancouver ; Following ICFA-NTFCHARGE: CHARGE: Make recommendations to ICFA concerning the connectivityMake recommendations to ICFA concerning the connectivity
between between the Americasthe Americas, Asia and Europe , Asia and Europe As part of the process of developing theseAs part of the process of developing these
recommendations, the committee should recommendations, the committee should Monitor traffic on the world’s networks Monitor traffic on the world’s networks Keep track of technology developmentsKeep track of technology developments Periodically review forecasts of future Periodically review forecasts of future
bandwidth needs, and bandwidth needs, and Provide early warning of potential problemsProvide early warning of potential problems
Create subcommittees as needed to meet the chargeCreate subcommittees as needed to meet the charge Representatives: Major labs, ECFA, ACFA, North and Representatives: Major labs, ECFA, ACFA, North and
South American Users South American Users Chair of the committee reports to ICFA twice per yearChair of the committee reports to ICFA twice per year
SCIC in 2004-2005 http://cern.ch/icfa-scic SCIC in 2004-2005 http://cern.ch/icfa-scic
Three 2005 Reports, Presented to ICFA Today Three 2005 Reports, Presented to ICFA Today Main Report: “Networking for HENP”Main Report: “Networking for HENP” [H. Newman et al.] [H. Newman et al.]
Includes Updates on the Digital Divide, WorldIncludes Updates on the Digital Divide, WorldNetwork Status; Brief updates on Monitoring and Network Status; Brief updates on Monitoring and Advanced Technologies [*]Advanced Technologies [*]
18 Appendices: A World Network Overview 18 Appendices: A World Network Overview Status and Plans for the Next Few Years of Nat’l & Status and Plans for the Next Few Years of Nat’l & Regional Networks, and Optical Network InitiativesRegional Networks, and Optical Network Initiatives
Monitoring Working Group ReportMonitoring Working Group Report [L. Cottrell] [L. Cottrell]
Also See: Also See: SCIC Digital Divide ReportSCIC Digital Divide Report [A. Santoro et [A. Santoro et
al.]al.] SCIC 2004 Digital Divide in Russia Report [V. Ilyin]SCIC 2004 Digital Divide in Russia Report [V. Ilyin] TERENA (TERENA (www.terena.nlwww.terena.nl) 2004 Compendium ) 2004 Compendium
SCIC Main Conclusion for 2002-5SCIC Main Conclusion for 2002-5
The disparity among regions in HENP could increase The disparity among regions in HENP could increase even more sharply, as we learn to use advanced even more sharply, as we learn to use advanced networks effectively, and we develop dynamic Grid networks effectively, and we develop dynamic Grid systems in the “most favored” regions systems in the “most favored” regions
We must therefore take action, and We must therefore take action, and work work to Close the Digital Divideto Close the Digital Divide To make Scientists in All World Regions Full To make Scientists in All World Regions Full
Partners in the Process of Frontier Discoveries Partners in the Process of Frontier Discoveries This is essential for the health of our global This is essential for the health of our global
experimental collaborations, for our field, experimental collaborations, for our field, and for international collaboration in many and for international collaboration in many fields of science.fields of science.
NEWS: Bulletin: ONE TWOWELCOME BULLETIN General InformationRegistrationTravel Information Hotel RegistrationParticipant List How to Get UERJ/Hotel Computer AccountsUseful Phone Numbers ProgramContact us: Secretariat Chairmen
CLAF CNPQ FAPERJ UERJ
SPONSORS
HEPGRID and Digital Divide Workshop
UERJ, Rio de Janeiro, Feb. 16-20 2004Theme: Global Collaborations, Grids and
Their Relationship to the Digital DivideFor the past three years the SCIC has focused on understanding and seeking the means of reducing or eliminating the Digital Divide. It proposed to ICFA that these issues, as they
affect our field, be brought to our community for discussion. This led to ICFA’s approval, in July 2003, of the Digital Divide and HEP Grid
Workshop. Review of R&E Networks; Major Grid
Projects Perspectives on Digital Divide Issues by
Major HEP Experiments, Regional Representatives Focus on Digital Divide Issues in Latin
America; Relate to Problems in Other Regions
See http://www.lishep.uerj.br
TutorialsC++Grid TechnologiesGrid-Enabled
AnalysisNetworksCollaborative Systems
A. Santoro
International ICFA Workshop onInternational ICFA Workshop on HEP Networking, Grids, and Digital HEP Networking, Grids, and Digital Divide Issues for Global e-ScienceDivide Issues for Global e-Science
May 23-27, 2005Daegu, Korea
Dongchul SonCenter for High Energy Physics
Harvey NewmanCalifornia Institute of
Technology Focus on Asia-Pacific Focus on Asia-Pacific Also Latin America, Middle East, Also Latin America, Middle East,
AfricaAfrica
Approved by ICFA Approved by ICFA August 2004August 2004
고에너지물리연구센터 CENTER FOR HIGH ENERGY PHYSICS
International ICFA Workshop on HEP International ICFA Workshop on HEP Networking, Grids Networking, Grids and Digital Divide Issues for Global e-Scienceand Digital Divide Issues for Global e-Science
Workshop Goals Review the current status, progress and barriers to effective
use of major national, continental and transoceanic networks
Review progress, strengthen opportunities for collaboration, and explore the means to deal with key issues in Grid computing and Grid-enabled data analysis, for high energy physics and other fields of data intensive science
Exchange information and ideas, and formulate plans to develop solutions to specific problems related to the Digital Divide, with a focus on the Asia Pacific region, as well as Latin America, Russia and Africa
Continue to advance a broad program of work on reducing or eliminating the Digital Divide, and ensuring global collaboration, as related to all of the above aspects.
PingER: World View from SLAC, CERN
S.E. Europe, Russia: Catching upLatin Am., China: Keeping upIndia, Mid-East, Africa: Falling Behind
C. Asia, Russia, SE Europe, L. America, M. East, China: 4-7 yrs behindIndia, Africa: 7-8 yrs behind
Latin AmericaLatin America
R. Cottrell
Connectivity to AfricaConnectivity to Africa
Internet Access: More than an order of magnitude lower than the Internet Access: More than an order of magnitude lower than the corresponding percentages in Europe (33%) & N. America (70%). corresponding percentages in Europe (33%) & N. America (70%).
Digital Divide: Lack of Infrastructure, especially interior, high prices Digital Divide: Lack of Infrastructure, especially interior, high prices (e.g. $ 4-10/kbps/mo.); “Gray” satellite bandwidth market(e.g. $ 4-10/kbps/mo.); “Gray” satellite bandwidth market
Intiatives: Intiatives: EUMEDCONNECT (EU-North Africa); GEANT: 155 Mbps to EUMEDCONNECT (EU-North Africa); GEANT: 155 Mbps to S. Africa; Nectarnet (Ga. Tech); IEEAF/I2 NSF-Sponsored InitiativeS. Africa; Nectarnet (Ga. Tech); IEEAF/I2 NSF-Sponsored Initiative
INTERNET USERS AND POPULATION STATISTICS FOR AFRICA
Population( 2004 Est. )
Pop. Pct.
World
Internet Users
Usage Growth(2000-4)
Penetration:Percent of Population
Users: Percentin World
TOTAL for AFRICA
893 M 14.0 % 12.9 M 186.6 % 1.4 % 1.6 %
REST of WORLD
5,497 M 86.0 % 800 M 124.4 % 14.6 % 98.4 %
Sample Bandwidth Costs for African Universities
$0.27
$3.00
$6.77
$9.84
$11.03
$20.00
$0.00 $5.00 $10.00 $15.00 $20.00 $25.00
USA
IBAUD Target
Ghana
Uganda
Average
Nigeria
$/kbps/month
Sample size of 26 universitiesAverage Cost for VSAT service: Quality, CIR, Rx, Tx not distinguished
Bandwidth prices in Africa vary dramatically; are in general many times what they could be if universities purchase in volume
Roy Steiner Internet2 2004 Workshop
Avg. Unit Cost is 40X US Avg.; Cost is Several Hundred Times, Compared to Leading Countries
Asia Pacific Academic Network ConnectivityAsia Pacific Academic Network Connectivity
Access PointExchange PointCurrent Status2004 (plan)
US
Europe
ID
VN
LK
KR
TH
MY
SG
RU
PH
CN
HK
JP2G
12M
34M
45M
20.9G
155M
777M
1.2G
TW
200M
310M
AU
45M`722M
IN
16M
1.5M
1.5M
7.5M2M
2.5M
2M
90M
155M
622M
155M
9.1G
932M
Better Better North/South Linkages within Asia North/South Linkages within Asia NeededNeeded JP- TH link: 2Mbps JP- TH link: 2Mbps 45Mbps in 2004. 45Mbps in 2004.
Connectivity Connectivity to US from JP, to US from JP, KO, AU is KO, AU is Advancing Advancing Rapidly.Rapidly.
Progress in Progress in the Region, the Region, and to Europe and to Europe is Much is Much Slower Slower
APAN Status APAN Status 7/20047/2004
D. Y. Kim
CountriesCountries NetworkNetwork Bandwidth(Mbps)Bandwidth(Mbps) AUP/RemarkAUP/RemarkID-IDID-ID AI3 (ITB-UNIBRAW)AI3 (ITB-UNIBRAW) 0.128/0.1280.128/0.128 R&E + CommodityR&E + CommodityIN-US/UKIN-US/UK ERNETERNET 7070 R&ER&EJP-CNJP-CN NICT-CERNETNICT-CERNET 1 Gbps1 Gbps R&ER&EJP-CNJP-CN NICT-CSTNETNICT-CSTNET 1 Gbps1 Gbps R&ER&EJP-IDJP-ID AI3 (ITB)AI3 (ITB) 0.5/1.5 (From/To JP)0.5/1.5 (From/To JP) R&E + CommodityR&E + CommodityJP-KRJP-KR APIIAPII 2 Gbps2 Gbps R&E R&E JP-LAJP-LA AI3 (NUOL)AI3 (NUOL) 0.1280.128 R&E + CommodityR&E + CommodityJP-LKJP-LK AI3 (IOIT)AI3 (IOIT) 0.5/0.5 (From/To JP)0.5/0.5 (From/To JP) R&E + CommodityR&E + CommodityJP-MYJP-MY AI3 (USM)AI3 (USM) 0.5/0.5 (From/To JP)0.5/0.5 (From/To JP) R&E + CommodityR&E + CommodityJP-PHJP-PH AI3 (ASTI)AI3 (ASTI) 0.5/0.5 (From/To JP)0.5/0.5 (From/To JP) R&E + CommodityR&E + CommodityJP-PHJP-PH MAFFIN (ASTI)MAFFIN (ASTI) 66 R&E R&E JP-SGJP-SG AI3 (TP)AI3 (TP) 0.5/0.5 (From/To JP)0.5/0.5 (From/To JP) R&E + CommodityR&E + CommodityJP-THJP-TH AI3 (AIT)AI3 (AIT) 0.5/1.5 (From/To JP)0.5/1.5 (From/To JP) R&E + CommodityR&E + CommodityJP-THJP-TH SINET (ThaiSarn)SINET (ThaiSarn) 4545 R&E R&E JP-USJP-US IEEAFIEEAF 10 Gbps+62210 Gbps+622 R&E R&E JP-USJP-US JGN2JGN2 10 Gbps10 Gbps R&E R&E JP-USJP-US SINETSINET 10 Gbps10 Gbps R&E R&E
Some APAN Links Some APAN Links
G. McLaughlin
Digital Divide Illustrated by Network Infrastructures: TERENA NREN Core Capacity
Digital Divide Illustrated by Network Infrastructures: TERENA NREN Core Capacity
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Ukraine
Moldova
Azerbaijan
Jordan
Morocco
Iran
Bulgaria
Algeria
Croatia
Romania
Turkey
Serbia/Montenegro
Cyprus
Estonia
Latvia
Malta
Slovenia
Lithuania
Austria
Denmark
Iceland
Ireland
Slovakia
Switzerland
Portugal
Czech Republic
Finland
France
Greece
Hungary
Italy
Norway
Spain
Belgium
Germany
Netherlands
Poland
Sweden
United Kingdom
Core Capacity in Gbps
Current Core Capacity
Expected Increase in two years
Core capacity goes up in Large Steps: 10 to 20 Gbps; 2.5 to 10 Gbps; 0.6-1 to 2.5 Gbps
Current In Two Years
SE Europe, Medit., FSU, Middle East:Less Progress Based on Older
Technologies (Below 0.15, 1.0 Gbps): Digital Divide Will Not Be Closed
Source: TERENA
Long Term Trends in Network Traffic Volumes: 300-1000X/10Yrs
Long Term Trends in Network Traffic Volumes: 300-1000X/10Yrs
SLAC Traffic ~400 Mbps; Growth in Steps (ESNet Limit): ~ 10X/4 Years.
Projected: ~2 Terabits/s by ~2014 July 2005: 2x10 Gbps links: one
for production and one for research
ESnet Accepted Traffic 1990 – 2004Exponential Growth Since ’92;
Annual Rate Increased from 1.7 to 2.0X Per Year In the Last 5 Years
W. Johnston
L. Cottrell
Progress
in Steps
ESnet Monthly Accepted Traffic ThroughNov, 2004
0
50
100
150
200
250
300
350
400
Jan,
94Ju
n, 94
Nov,
94Ap
r, 95
Sep,
95Fe
b, 96
Jul, 9
6De
c, 96
May,
97Oc
t, 97
Mar, 9
8Au
g, 98
Jan,
99Ju
n, 99
Nov,
99Ap
r, 00
Sep,
00Fe
b, 01
Jul,
01De
c, 01
May,0
2Oc
t, 02
Mar, 0
3Au
g, 03
Jan,
04Ju
n, 04
Nov,
04
TByte
/Mon
th
W. Johnston
10 Gbit/s
FNAL: 10 to 20 (+40) Gbps by Fall 2005
CANARIE (Canada) Utilization Trends and UCLPv2
0
5
10
15
20
25
30LightpathsIP PeakIP Average
GbpsNetwork Capacity Limit
Jan 0
5
“Demand for Customer Empowered Nets (CENs) is exceeding our wildest expectations
New version of UCLP will allow easier integration of CENs into E2E nets for specific communities &/or disciplines
UCLPv2 will be based on SOA, web services and workflow to allow easy integration into cyber-infrastructure projects
The Network is no longer a static fixed facility – but can be ‘orchestrated’ with different topologies, routing etc to meet specific needs of end users”
W. St. Arnaud
National Lambda Rail (NLR): www.nlr.netNational Lambda Rail (NLR): www.nlr.net
NLR Initially 4-8 10G
Wavelengths To 40 10G
Waves in Future Ultralight, Internet2
HOPI, Cisco Research & UltraScience NetInitiatives w/HEP
Atlantic & Pacific Wave
Transition beginning now to optical, multi-wavelength Community owned or leased
“dark fiber” networks for R&E
Initiatives in: nl, ca, jp, uk, kr; pl, cz, sk, pt, ei, gr, sb/mn … + 30 US States (Ca, Il, Fl, In, …)
GEANT2 Hybrid ArchitectureGEANT2 Hybrid Architecture
Cooperation of 26 NRENs Implementation on dark fiber, IRU
Asset, Transmission & Switching Equipment Layer 1 & 2 switching, “the Light Path” Point to Point (E2E) Wavelength services LHC in Europe: N X 10G T0-T1 Overlay Net
Global Connectivity
10 Gbps + 3x2.5 Gbps to North America
2.5 Gbps to Japan
622 Mbps to South America
45 Mbps to Mediterranean countries
155 Mbps to South Africa
Will be Improved in GEANT2
H. Doebbling
SXTransport: Au-US 2 X 10GSXTransport: Au-US 2 X 10G
AARNet has dual 10Gbps circuits to the US via Hawaii, dual 622Mbps commodity links G. McLaughlin
JGN2: Japan Gigabit Network (4/04 – 3/08)20 Gbps Backbone, 6 Optical Cross-Connects
Kanazawa Sendai
Sapporo
Nagano
Kochi Nagoya
Fukuoka
Okinawa
Okayama
< 1 G> ・ Teleport Okayama (Okayama) ・ Hiroshima University (Higashi
Hiroshima) < 100 M> ・ Tottori University of
Environmental Studies (Tottori) ・ Techno Ark Shimane (Matsue) ・ New Media Plaza Yamaguchi
(Yamaguchi)
< 10 G> ・ Kyoto University (Kyoto) ・ Osaka University (Ibaraki) < 1 G> ・ NICT Kansai Advanced Research Center (Kobe) < 100 M> ・ Lake Biwa Data Highway AP * (Ohtsu) ・ Nara Prefectural Institute of Industrial
Technology (Nara) ・ Wakayama University (Wakayama) ・ Hyogo Prefecture Nishiharima Technopolis
(Kamigori-cho, Hyogo Prefecture) < 10 G> ・ Kyushu University (Fukuoka) < 100 M> ・ NetCom Saga (Saga) ・ Nagasaki University
(Nagasaki) ・ Kumamoto Prefectural Office
(Kumamoto) ・ Toyonokuni Hyper Network
AP *(Oita) ・ Miyazaki University (Miyazaki) ・ Kagoshima University
(Kagoshima)
< 100 M> ・ Kagawa Prefecture Industry Promotion Center (Takamatsu) ・ Tokushima University (Tokushima) ・ Ehime University (Matsuyama) ・ Kochi University of Technology
(Tosayamada-cho, Kochi Prefecture)
< 100 M> ・ Nagoya University (Nagoya) ・ University of Shizuoka (Shizuoka) ・ Softopia Japan (Ogaki, Gifu Prefecture) ・ Mie Prefectural College of Nursing (Tsu)
< 10 G> ・ Ishikawa Hi-tech Exchange Center
(Tatsunokuchi-machi, Ishikawa Prefecture) < 100 M> ・ Toyama Institute of Information Systems (Toyama) ・ Fukui Prefecture Data Super Highway AP * (Fukui)
< 100 M> ・ Niigata University
(Niigata) ・ Matsumoto Information Creation Center
(Matsumoto, Nagano Prefecture)
< 10 G> ・ Tokyo University
(Bunkyo Ward, Tokyo) ・ NICT Kashima Space Research Center
(Kashima, Ibaraki Prefecture) <1G> ・ Yokosuka Telecom Research Park
(Yokosuka, Kanagawa Prefecture) <100M> ・ Utsunomiya University (Utsunomiya) ・ Gunma Industrial Technology Center
(Maebashi) ・ Reitaku University
(Kashiwa, Chiba Prefecture) ・ NICT Honjo Information and Communications Open Laboratory
(Honjo, Saitama Prefecture) ・ Yamanashi Prefecture Open R&D Center
(Nakakoma-gun, Yamanashi Prefecture)
< 1 G> ・ Tohoku University (Sendai) ・ NICT Iwate IT Open Laboratory
(Takizawa-mura, Iwate Prefecture) < 100 M> ・ Hachinohe Institute of Technology
(Hachinohe, Aomori Prefecture) ・ Akita Regional IX * (Akita) ・ Keio University Tsuruoka Campus
(Tsuruoka, Yamagata Prefecture) ・ Aizu University (Aizu Wakamatsu)
< 100 M> ・ Hokkaido Regional Network Association AP * (Sapporo)
NICT Tsukuba Research Center
20Gbps10Gbps1GbpsOptical testbeds
Core network nodes
Access points
Osaka
Otemachi
USA
NICT Keihannna Human Info-Communications
Research Center
NICT Kita Kyushu IT Open Laboratory
NICT Koganei Headquarters
[Legends ]
*IX:Internet eXchange
AP:Access Point
JGN2 Connection services at
the optical level: 1 GbE and 10GbE
Optical testbeds: e.g.GMPLS Interop. Tests
Y. Karita
APAN-KR : KREONET/KREONet2 IIAPAN-KR : KREONET/KREONet2 II
KREONET 11 Regions, 12 POP Centers Optical 2.5-10G Backbone;
SONET/SDH, POS, ATM National IX Connection
KREONET2 Support for Next Gen. Apps: IPv6, QoS, Multicast;
Bandwidth Alloc. Services StarLight/Abilene Connection
International Links GLORIAD Link to 10G to
Seattle Aug. 1 (MOST) US: 2 X 622 Mbps via
CA*Net; GbE via TransPAC Japan: 2 Gbps TEIN to GEANT: 155 Mbps
SuperSIREN (7 Res. Institutes) Optical 10-40G Backbone Collaborative Environment
Support High Speed Wireless: 1.25 G
D. Son
The Global Lambda Integrated Facility for Research and Education (GLIF)
The Global Lambda Integrated Facility for Research and Education (GLIF)
Architecting an international LambdaGrid infrastructure Architecting an international LambdaGrid infrastructure Virtual organizationVirtual organization supports persistent data-intensive scientific supports persistent data-intensive scientific
research and middleware development on “LambdaGrids”research and middleware development on “LambdaGrids”
Many 2.5 - 10G Links Across the Atlantic and PacificPeerings: Pacific & Atlantic Wave; Seattle, LA, Chicago, NYC, HK
Product of transfer speed and distance Product of transfer speed and distance using standard using standard Internet (TCP/IP) protocols.Internet (TCP/IP) protocols.
Single Stream Single Stream 7.5 Gbps X 16 kkm7.5 Gbps X 16 kkm with with Linux: July 2004Linux: July 2004
IPv4 Multi-stream record with FAST IPv4 Multi-stream record with FAST TCP: TCP: 6.86 Gbps X 27kkm:6.86 Gbps X 27kkm: Nov 2004 Nov 2004
IPv6 record: IPv6 record: 5.11 Gbps5.11 Gbps between between Geneva and Starlight: Jan. 2005Geneva and Starlight: Jan. 2005
Concentrate now on reliable Concentrate now on reliable Terabyte-scale file transfersTerabyte-scale file transfers Disk-to-disk Marks: Disk-to-disk Marks:
536 Mbytes/sec (Windows); 536 Mbytes/sec (Windows); 500 Mbytes/sec (Linux) 500 Mbytes/sec (Linux)
Note Note System Issues:System Issues: PCI-X Bus, PCI-X Bus, Network Interface, Disk I/O Network Interface, Disk I/O Controllers, CPU, DriversControllers, CPU, Drivers
Internet 2 Land Speed Record (LSR)
NB: Computing Manuf.’s Roadmaps for 2006: One Server Pair to One 10G Link
Nov. 2004 Record Network
6.6 Gbps16500km
4.2 Gbps16343km5.6 Gbps
10949km
5.4 Gbps7067km2.5 Gbps
10037km0.9 Gbps10978km0.4 Gbps
12272km
0
20
40
60
80
100
120
140
160
Thro
ughp
ut (G
bps)
Internet2 LSR - Single IPv4 TCP stream 7.21 Gbps20675 kmInternet2 LSRs:
Blue = HEP
7.2G X 20.7 kkm
Th
rou
hg
pu
t (P
eta
bit
-m/s
ec)
S. Ravot
SC2004 Bandwidth Record by HEP: High Speed TeraByte Transfers for Physics
SC2004 Bandwidth Record by HEP: High Speed TeraByte Transfers for Physics
Caltech, CERN Caltech, CERN SLAC, FNAL, UFl, SLAC, FNAL, UFl, FIU, ESNet, UK, FIU, ESNet, UK, Brazil, Korea;Brazil, Korea;NLR, Abilene, NLR, Abilene, LHCNet, TeraGrid;LHCNet, TeraGrid;DOE, NSF, EU, …; DOE, NSF, EU, …; Cisco, Neterion, Cisco, Neterion, HP, NewiSys, …HP, NewiSys, …
Ten 10G Waves, Ten 10G Waves, 80 10GbE Ports, 80 10GbE Ports, 50 10GbE NICs50 10GbE NICs
Aggregate Rate of Aggregate Rate of 101 Gbps101 Gbps
1.6 Gbps to/from 1.6 Gbps to/from KoreaKorea
2.93 Gbps to/from 2.93 Gbps to/from BrazilBrazil
UERJ, USP
Monitoring NLR, Abilene, LHCNet, SCINet, UERJ, USP, Int’l R&E Nets and 9000+ Grid Nodes Simultaneously
I. Legrand
SC2004 KNU Traffic: 1.6 Gbps to/From Pittsburgh Via Transpac (LA) and NLR SC2004 KNU Traffic: 1.6 Gbps to/From Pittsburgh Via Transpac (LA) and NLR
Monitoring in Daegu
Courtesy K. Kwon
SC2004: 2.93 (1.95 + 0.98) Gbps Sao Paulo – Miami – Pittsburgh
(Via Abilene)
Brazilian T2+T3 HEPGrid: Rio + Sao
Paolo
Also 500 Mbps Via Red CLARA, GEANT (Madrid)
Madrid &
GEANT
GEANT (SURFNet
)
J. Ibarra
HENP Bandwidth Roadmap for Major Links (in Gbps)
HENP Bandwidth Roadmap for Major Links (in Gbps)
Year Production Experimental Remarks
2001 0.155 0.622-2.5 SONET/SDH
2002 0.622 2.5 SONET/SDH DWDM; GigE Integ.
2003 2.5 10 DWDM; 1 + 10 GigE Integration
2005 10 2-4 X 10 Switch; Provisioning
2007 2-4 X 10 ~10 X 10; 40 Gbps
1st Gen. Grids
2009 ~10 X 10 or 1-2 X 40
~5 X 40 or ~20-50 X 10
40 Gbps Switching
2011 ~5 X 40 or
~20 X 10
~25 X 40 or ~100 X 10
2nd Gen Grids Terabit Networks
2013 ~Terabit ~MultiTbps ~Fill One Fiber
Continuing Trend: ~1000 Times Bandwidth Growth Per Decade;HEP: Co-Developer as well as Application Driver of Global Nets
Evolving Quantitative Science Requirements for Networks (DOE High Perf. Network Workshop)
Evolving Quantitative Science Requirements for Networks (DOE High Perf. Network Workshop)
Science AreasScience AreasToday Today
End2EndEnd2End ThroughputThroughput
5 years 5 years End2End End2End
ThroughputThroughput
5-10 Years 5-10 Years End2End End2End
ThroughputThroughput
RemarksRemarks
High Energy High Energy PhysicsPhysics
0.5 Gb/s0.5 Gb/s 100 Gb/s100 Gb/s 1000 Gb/s1000 Gb/s High bulk High bulk throughputthroughput
Climate (Data & Climate (Data & Computation)Computation)
0.5 Gb/s0.5 Gb/s 160-200 Gb/s160-200 Gb/s N x 1000 Gb/sN x 1000 Gb/s High bulk High bulk throughputthroughput
SNS SNS NanoScienceNanoScience
Not yetNot yet started started
1 Gb/s1 Gb/s 1000 Gb/s + QoS 1000 Gb/s + QoS for Control for Control
ChannelChannel
Remote Remote control and control and time critical time critical throughputthroughput
Fusion EnergyFusion Energy 0.066 Gb/s0.066 Gb/s(500 MB/s (500 MB/s
burst)burst)
0.198 Gb/s0.198 Gb/s(500MB/(500MB/
20 sec. burst)20 sec. burst)
N x 1000 Gb/sN x 1000 Gb/s Time critical Time critical throughputthroughput
AstrophysicsAstrophysics 0.013 Gb/s0.013 Gb/s(1 TByte/week)(1 TByte/week)
N*N multicastN*N multicast 1000 Gb/s1000 Gb/s Computat’l Computat’l steering and steering and
collaborationscollaborations
Genomics Data Genomics Data & Computation& Computation
0.091 Gb/s0.091 Gb/s(1 TBy/day)(1 TBy/day)
100s of users100s of users 1000 Gb/s + QoS 1000 Gb/s + QoS for Control for Control
ChannelChannel
High High throughput throughput
and steeringand steering
See http://www.doecollaboratory.org/meetings/hpnpw/ W. Johnston
LHCNet , ESnet Plan 2007/2008:40Gbps US-CERN, ESnet MANs, IRNC
LHCNet , ESnet Plan 2007/2008:40Gbps US-CERN, ESnet MANs, IRNC
DENDEN
ELPELP
ALBALBATLATL
Metropolitan Area Rings
Aus.
Europe
SDGSDG
AsiaPacSEASEA
Major DOE Office of Science SitesHigh-speed cross connects with Internet2/Abilene
New ESnet hubsESnet hubs
SNVSNV
Europe
Japan
Science Data Network core, 40-60 Gbps circuit transportLab suppliedMajor international
Production IP ESnet core, 10 Gbps enterprise IP traffic
Japan
Aus.
Metro Rings
ESnet2nd Core:
30-50G
ESnet IP Core(≥10 Gbps)
10Gb/s10Gb/s
30Gb/s2 x 10Gb/s
NYCNYCCHICHI
LHCNetData Network(4 x 10 Gbps US-CERN)
LHCNet Data Network
DCDCGEANT2SURFNetIN2P3
NSF/IRNC circuit; GVA-AMS connection via Surfnet or Geant2
CERN
FNAL
BNL
LHCNet US-CERN: 9/05: 10G CHI + 10G NY
2007: 20G + 20G 2009: ~40G + 40G
S. Ravot
We Need to Work on the Digital Dividefrom Several Perspectives
We Need to Work on the Digital Dividefrom Several Perspectives
Workshops and Tutorials/Training SessionsWorkshops and Tutorials/Training Sessions For Example: ICFA DD Workshops, Rio 2/04, HONET (Pakistan) For Example: ICFA DD Workshops, Rio 2/04, HONET (Pakistan)
12/04; 12/04; Daegu May 2005Daegu May 2005 Share Information: Monitoring, BW Progress; Dark Fiber Projects;Share Information: Monitoring, BW Progress; Dark Fiber Projects;
Prices in different markets Prices in different markets Use Model Cases: Poland, Slovakia, Czech Rep., China, Brazil,…Use Model Cases: Poland, Slovakia, Czech Rep., China, Brazil,…
Encourage, and Work on Inter-Regional Projects Encourage, and Work on Inter-Regional Projects GLORIAD, Russia-China-Korea US Optical RingGLORIAD, Russia-China-Korea US Optical Ring Latin America: CHEPREO/WHREN (US-Brazil); RedCLARALatin America: CHEPREO/WHREN (US-Brazil); RedCLARA
Help with Modernizing the Infrastructure Help with Modernizing the Infrastructure Design, Commissioning, Development Design, Commissioning, Development Provide Tools for Effective Use: Monitoring, CollaborationProvide Tools for Effective Use: Monitoring, Collaboration
Systems; Advanced TCP Stacks, Grid System SoftwareSystems; Advanced TCP Stacks, Grid System Software Work on Policies and/or Pricing: pk, br, cn, SE Europe, in, …Work on Policies and/or Pricing: pk, br, cn, SE Europe, in, …
Encourage Access to Dark FiberEncourage Access to Dark Fiber Raise World Awareness of Problems, & Opportunities for SolutionsRaise World Awareness of Problems, & Opportunities for Solutions
UERJ T2 HEPGRID Inauguration: Dec. 2004: The Team (Santoro et al.)
UERJ Tier2 Now On Grid3 and Open Science Grid
(5/13)
100 Dual Nodes;
UpgradesPlanned
Also Tier3 in Sao Paulo
(Novaes)
Grid3, the Open Science Grid and DISUN
Grid3, the Open Science Grid and DISUN
Grid3: A National Grid Infrastructure
35 sites, 3500 CPUs: Univ. + 4 Nat’l labs
Part of LHC Grid Running since
October 2003 HEP, LIGO,
SDSS, Biology, Computer Sci.
Transition to Open Science Grid (www.openscience.org)
+Brazil (UERJ, USP)
7 US CMS Tier2s; Caltech, Florida, UCSD, UWisc Form DISUN
P. Avery
Science-Driven: HEPGRID (CMS) in BrazilHEPGRID-CMS/BRAZIL is a project to build a Grid thatAt Regional Level will include CBPF,UFRJ,UFRGS,UFBA, UERJ & UNESPAt International Level will be integrated with CMS Grid based at CERN; focal points include Grid3/OSG and bilateral projects with Caltech Group
France Italy USAGermany BRAZIL622 Mbps
UFRGS
UERJUFRJ
T1
IndividualMachines
On line systems
Brazilian HEPGRID
CBPF
UNESP/USPSPRACE-Working
Gigabit
CERN 2.5 - 10 Gbps
UFBA
UERJ RegionalTier2 Ctr
T4
T0 +T1
T2 T1
T3 T2
UERJ: T2T1,
100500 Nodes;
Plus T2s to
100 Nodes
ICFA DD Workshop 2/04; T2 Inauguration
+ GIGA/RNP Agree 12/04
Rio Tier2-SPRACE (Sao Paolo)-Ampath Direct Link at 1 GbpsRio Tier2-SPRACE (Sao Paolo)-Ampath Direct Link at 1 Gbps
AMPATHNAP of Brazil
TerremarkEletropaulo Fiber leased to ANSP
1 Gbps
Iqar
a Fib
er
leas
ed to
ANSP1
Gbps
SPRACE
T2 Rio
CC-USP
Jump CC
Giga Fiber
1 Gbps
Giga Fiber
1 Gbps
1 Gb
ps
1 Gb
ps
GigaRouter
ANSPRouters
UERJ
Caltech/CiscoRoutersL. Lopez
Highest Bandwidth Link in NREN’s Infrastructure, EU & EFTA Countries, & Dark Fiber
Highest Bandwidth Link in NREN’s Infrastructure, EU & EFTA Countries, & Dark Fiber
1
10
100
1000
10000
Hig
hest
Lin
k S
peed
(M
bp
s)
Legend
has dark fiber
will have dark fiber
no dark fiber
Owning (or leasing) dark fiber is an interesting option Owning (or leasing) dark fiber is an interesting option for an NREN; Depends on the national situation. for an NREN; Depends on the national situation.
NRENs that own dark fiber can decide for themselves NRENs that own dark fiber can decide for themselves which technology and what speeds to use on it which technology and what speeds to use on it
10.0G
1.0G
0.1G
0.01G
Source: TERENA
Europe: Core Network Bandwidth Increase for Years 2001-2004 and 2004-2006
Europe: Core Network Bandwidth Increase for Years 2001-2004 and 2004-2006
1
10
100
1000
Incre
ase f
acto
r
Expected increase in two years
Increase factor for 2001-2004
Countries With No Increase Already Had 1-2.5G Backbone in 2001 These are all going to 10G backbones by 2006-7
Countries Showing the Largest Increase Are: PIONIER (Poland) from 155 Mbps to 10 Gbps capacity (64X) SANET (Slovakia) from 4 Mbps to 1 Gbps (250X).
Source: TERENA
1660 km of Dark Fiber CWDM Links, 1 to 4 Gbps (GbE) August 2002: Dark Fiber Link, to Austria April 2003: Dark Fiber Link to Czech Republic 2004: Dark Fiber Link to Poland Planning 10 Gbps Backbone
> 250X: 2002-2005
120km CBDF Cost 4 k
Per Month 1 GE Now;
10G Planned
T. Weis
Dark Fiber in Eastern Europe Poland: PIONIER (10-20G) Network
Dark Fiber in Eastern Europe Poland: PIONIER (10-20G) Network
2763 km Lit Fiber 2763 km Lit Fiber Connects 22 MANs; Connects 22 MANs;
+1286 km (9/05) +1286 km (9/05) + 1159 km (4Q/06)+ 1159 km (4Q/06)
Vision: Support - Vision: Support - Computational Grids;Computational Grids;
Domain-Specific Grids Domain-Specific Grids Digital LibrariesDigital Libraries Interactive TVInteractive TV
Add’l Fibers for Add’l Fibers for e-Regional Initiatives e-Regional Initiatives
10 Gb/s
MetropolitanAreaNetworks
1 Gb/s
10 Gb/s(2 lambdas)
GDAŃSK
POZNAŃ
ZIELONA GÓRA
KATOWICE
KRAKÓW
LUBLIN
WARSZAWA
BYDGOSZCZTORUŃ
CZĘSTOCHOWA
BIAŁYSTOK
OLSZTYN
RZESZÓW
BIELSKO-BIAŁA
GÉANT
KOSZALIN
SZCZECIN
WROCŁAW
ŁÓDŹ
KIELCE PUŁAWYOPOLE
RADOM
BASNET 34 Mb/s
CESNET, SANET
4Q05 Plan: Courtesy 4Q05 Plan: Courtesy M. PrzybylskiM. Przybylski
PIONIER Cross Border Dark Fiber Plan LocationsPIONIER Cross Border Dark Fiber Plan Locations
Single Single GEANT PoPGEANT PoPin Poznanin Poznan
CESNET2 (Czech Republic) Network Topology, Dec. 2004
2500+ km 2500+ km Leased Fibers Leased Fibers (Since 1999) (Since 1999)
2005:2005: 10GE Link Praha-Brno (300km) in Service; 10GE Link Praha-Brno (300km) in Service; Plan to go to 4 X 10G and higher as needed;Plan to go to 4 X 10G and higher as needed;
More 10GE links planned More 10GE links planned J. Gruntorad
APAN China ConsortiumAPAN China Consortium Established in 1999. The China Education and Research Established in 1999. The China Education and Research
Network (Network (CERNETCERNET) and the China Science and Technology ) and the China Science and Technology Network (Network (CSTNETCSTNET) are the main advanced networks. ) are the main advanced networks.
CERNet
2.5 Gbps
CSTnet
CERNET 2000: Own dark fiber crossing 30+
major cities and 30,000 kilometers 2003: 1300+ universities and
institutes, over 15 million users
CERNET 2: Next Generation R&E Net Backbone connects 15-20 Giga-
POPs at 2.5G-10Gbps (I2-like) Connects to 200 Universities and
100+ Research Institutes at 1 Gbps-10 Gbps
Native IPv6 and Lambda Networking
From 6 to 78 Million Internet Users in China from Jan. – July 2004J. P. Wu, H.
Chen
Brazil (RNP2): Rapid Backbone Progressand the GIGA Project
Brazil (RNP2): Rapid Backbone Progressand the GIGA Project
RNPRNP Connects the regional Connects the regional networks in all 26 states of Brazilnetworks in all 26 states of Brazil
Backbone on major links to 155 Backbone on major links to 155 Mbps; 622 Mbps Rio – Sao Paulo.Mbps; 622 Mbps Rio – Sao Paulo.
2.5G to 10G Core in 2005 2.5G to 10G Core in 2005 (300X Improvement in 2 Years)(300X Improvement in 2 Years)
The GIGA ProjectThe GIGA Project – Dark Fiber – Dark Fiber experimental networkexperimental network
700 km of fiber, 7 cities 700 km of fiber, 7 cities and 20 institutions in and 20 institutions in Sao Paolo and Rio Sao Paolo and Rio
GbE to Rio Tier-2, Sao GbE to Rio Tier-2, Sao Paulo Tier-3Paulo Tier-3
RNP & GIGA: RNP & GIGA: Extend GIGA Extend GIGA to the Northeast, with 4000 to the Northeast, with 4000 km of dark fiber by 2008km of dark fiber by 2008
L. Lopez
DFN (Germany): X-WiN-Fiber Network
Faser KPNFaser GLFaser GCFaser vorhanden
GAR
ERL
BAY
MUE
FZJ
AAC BIR
DES
HAM
POTTUB
FZK
GSI
DUI
BRE
HAN
BRA MAGBIE
FRA
HEI
STU
REG
DRE
CHE
ZIB
ILM
KIE
ROS
LEI
JENWEI
ESF
HUB
ADH
Most of the Most of the X-WiNX-WiN core core will be a fibre network, will be a fibre network, (see map), the rest will (see map), the rest will be provided by be provided by wavelengthswavelengths
Several fibre and Several fibre and wavelengths providerswavelengths providers
Fibre is relatively cheap Fibre is relatively cheap – in most cases more – in most cases more economic than (one) economic than (one) wavelengthwavelength
X-Win creates many new X-Win creates many new options besides being options besides being cheaper than the current cheaper than the current G-WiN coreG-WiN core
13.04.2005
K. Schauerhammer
Romania: Inter-City Links were 2 to 6 Mbps in 2002; Improved to 155 Mbps in 2003-2004;
GEANT-Bucharest Link: 155 to 622 Mbps
Romania: Inter-City Links were 2 to 6 Mbps in 2002; Improved to 155 Mbps in 2003-2004;
GEANT-Bucharest Link: 155 to 622 Mbps
RoEduNetJanuary 2005
Plan: 3-4 Centers at 2.5 Gbps; Dark Fiber InterCity Backbone
Compare Pk: 56 univ. share 155 Mbps Internationally
N. Tapus T. Ul Haq
ICFA Report: Networks for HENPICFA Report: Networks for HENPGeneral ConclusionsGeneral Conclusions
ICFA Report: Networks for HENPICFA Report: Networks for HENPGeneral ConclusionsGeneral Conclusions
Reliable high Reliable high End-to-end PerformanceEnd-to-end Performance of networked applications such of networked applications such as Data Grids is required. Achieving this requires:as Data Grids is required. Achieving this requires:A coherent approach to End-to-end monitoring in all regions A coherent approach to End-to-end monitoring in all regions
that allows scientists throughout the world to extract clear that allows scientists throughout the world to extract clear information information
Upgrading campus infrastructures.Upgrading campus infrastructures. To support Gbps flows to HEP centers. To support Gbps flows to HEP centers.
Removing local, last mile, and nat’l and int’l bottlenecks Removing local, last mile, and nat’l and int’l bottlenecks end-to-end, whether technical or political in origin.end-to-end, whether technical or political in origin.
Bandwidth Bandwidth across borders, the countryside or the city isacross borders, the countryside or the city isoften much less than on national backbones and int’l linksoften much less than on national backbones and int’l links
This problem is very widespread in our community: This problem is very widespread in our community: Examples stretching from the Asia Pacific to Latin Examples stretching from the Asia Pacific to Latin America America to the Northeastern US. Root causes vary, from to the Northeastern US. Root causes vary, from lack of lack of local infrastructure, to unfavorable policies and local infrastructure, to unfavorable policies and pricing pricing
Situation of Local Access in Belém in Brazil in 2004
Situation of Local Access in Belém in Brazil in 2004InstitutionInstitution Summary of local network connectionsSummary of local network connections Annual Annual
Cost (US$)Cost (US$)
CEFET CEFET Access to provider at 512 kbpsAccess to provider at 512 kbps 22,20022,200
CESUPA CESUPA (4 campi)(4 campi)
Internal + access to provider at 6 MbpsInternal + access to provider at 6 Mbps 57,80057,800
IEC/MS IEC/MS (2 campi)(2 campi)
Internal at 512 kbps + Access to provider at Internal at 512 kbps + Access to provider at 512 kbps512 kbps
13,30013,300
MPEG MPEG (2 campi)(2 campi)
Internal at 256 kbps; Access at 34 Mbps Internal at 256 kbps; Access at 34 Mbps (radio link)(radio link)
7,6007,600
UEPA UEPA (5 campi)(5 campi)
Internal at 128 kbps; Access at 512 kbpsInternal at 128 kbps; Access at 512 kbps 18,50018,500
UFPA UFPA (4 campi)(4 campi)
Internal at 128 kbps; Provider PoPInternal at 128 kbps; Provider PoP 16,70016,700
UFRAUFRA Access to provider at 1 MbpsAccess to provider at 1 Mbps 16,00016,000
UNAMA UNAMA (4 campi)(4 campi)
Internal wireless links, access at 6 MbpsInternal wireless links, access at 6 Mbps 88,90088,900
Annual telco charges for POOR local access = US$ 241,000
Belém: a Possible Topology(30 km ring)
Belém: a Possible Topology(30 km ring)
Alternative Approach in Brazil – Do It Yourself (DIY) Networking (M. Stanton, RNP)
Alternative Approach in Brazil – Do It Yourself (DIY) Networking (M. Stanton, RNP)
1.1. Form a consortium for joint network provisionForm a consortium for joint network provision
2.2. Build your own optical fibre network to reach ALL the campi Build your own optical fibre network to reach ALL the campi of ALL consortium members of ALL consortium members
3.3. Light it up and go!Light it up and go!
Costs involved:Costs involved: Building out the fibre: using utility poles of electric companyBuilding out the fibre: using utility poles of electric company
US$ 7,000 per kmUS$ 7,000 per km Monthly rental of US $1 per pole (~25 poles per km)Monthly rental of US $1 per pole (~25 poles per km)
Equipment costs: mostly use cheap 2 port GbE switchesEquipment costs: mostly use cheap 2 port GbE switches Operation and maintenanceOperation and maintenance
In Belém for 11 institutions using In Belém for 11 institutions using All GigEAll GigE connections: connections: Capital costs around US $ 500,000Capital costs around US $ 500,000 Running costs around US $ 40,000 p.a.Running costs around US $ 40,000 p.a. Compare with current US $ 240,000 p.a. for traditional telco Compare with current US $ 240,000 p.a. for traditional telco
solution solution [for 0.128 to 6 Mbps: ~1000X less bandwidth[for 0.128 to 6 Mbps: ~1000X less bandwidth]]
Brazil: RNP Nat’l Plan for Optical Metro Nets in 2005-6
Brazil: RNP Nat’l Plan for Optical Metro Nets in 2005-6
In December 2004, RNP signed contracts with In December 2004, RNP signed contracts with
“Finep” (the agency of the Ministry of Science and “Finep” (the agency of the Ministry of Science and
Technology) to Technology) to build optical metro networks in all 27 build optical metro networks in all 27
capital cities in Brazil capital cities in Brazil
Total value of more than US$15 millionsTotal value of more than US$15 millions
Most of this money will be spent in 2005Most of this money will be spent in 2005
M. Stanton
G. Cole
SegmentSegment CurrentCurrent Year 1Year 1 Year 2Year 2 Year 3Year 3 Year 4Year 4 Year Year 55
1 - Trans-1 - Trans-AsiaAsia
155 Mbps155 Mbps 2.5 Gbps (US-2.5 Gbps (US-China),10 Gbps China),10 Gbps (US-Korea-(US-Korea-China)China)
2 x 10 Gbps 2 x 10 Gbps US-China; US-China; US-Korea US-Korea -China -China
2 x 10 2 x 10 GbpsGbps
N x 10 N x 10 GbpsGbps
N x N x 10 10 GbpsGbps
2 - Trans-2 - Trans-ChinaChina
2.5 Gbps (155 2.5 Gbps (155 Mbps, Beijing-Mbps, Beijing-Khabarovsk)Khabarovsk)
2.5 Gbps2.5 Gbps 1 x 10 Gbps1 x 10 Gbps 2 x 10 2 x 10 GbpsGbps
N x 10 N x 10 GbpsGbps
N x N x 10 10 GbpsGbps
3 - Trans-3 - Trans-RussiaRussia
155 Mbps155 Mbps 622 Mbps622 Mbps 2.5 Gbps2.5 Gbps 1 x 10 1 x 10 Gbps Gbps
N x 10 N x 10 GbpsGbps
N x N x 10 10 GbpsGbps
4 - Trans-4 - Trans-EuropeEurope
622 Mbps622 Mbps 622 Mbps622 Mbps 622 Mbps622 Mbps 2 x 10 2 x 10 GbpsGbps
N x 10 N x 10 GbpsGbps
N x N x 10 10 GbpsGbps
5 - Trans-5 - Trans-AtlanticAtlantic
622 Mbps622 Mbps 1 Gbps1 Gbps 1 x 10 Gbps1 x 10 Gbps 2 x 10 2 x 10 GbpsGbps
N x 10 N x 10 GbpsGbps
N x N x 10 10 GbpsGbps
6 - Trans-6 - Trans-North North AmericaAmerica
2.5G (Asia- 2.5G (Asia- Chicago), GbE Chicago), GbE NYC-Chicago NYC-Chicago (via CANARIE)(via CANARIE)
10 Gbps, Seattle-10 Gbps, Seattle-Chicago-NYCChicago-NYC
10 Gbps, 10 Gbps, Seattle-Seattle-Chicago-NYCChicago-NYC
2 x 10 2 x 10 GbpsGbps
N x 10 N x 10 GbpsGbps
N x N x 10 10 GbpsGbps
SeattleChicago
NYC
Amsterdam
Moscow
Novosibirsk Khabarovsk
Beijing
Hong Kong
GLORIAD Topology – Current, Plans for Years 1-5
Pusan
Closing the Digital Divide: R&E Networks in/to Latin America
RNP2 and the GIGA Project RNP2 and the GIGA Project (ANSP)(ANSP)
AmPathAmPath: 45 Mbps Links : 45 Mbps Links to US (2002-5)to US (2002-5)
CHEPREO (NSF, from 2004):CHEPREO (NSF, from 2004): 622 Mbps Sao Paulo – Miami622 Mbps Sao Paulo – Miami
WHREN/LILA (NSF, from 2005)WHREN/LILA (NSF, from 2005)0.6 to 2.5 G Ring by 20060.6 to 2.5 G Ring by 2006Connections to Connections to
Pacific & Atlantic Wave Pacific & Atlantic WaveRedCLARA (EU):RedCLARA (EU): Connects Connects
18 Latin Am. NRENs, Cuba; 18 Latin Am. NRENs, Cuba; 622 Mbps Link to Europe622 Mbps Link to Europe
PacificWave
Sao PauloANSP
AtlanticWave
To GEANT622 Mbps
GOAL: An GOAL: An Information Society:Information Society:“… One in which highly developed “… One in which highly developed networks, equitable and ubiquitous networks, equitable and ubiquitous access to information, appropriate access to information, appropriate content in accessible formats, content in accessible formats, and effective communication can and effective communication can help people achieve their potential”help people achieve their potential”
Role of Science in the Information Society; WSIS 2003-2005
Role of Science in the Information Society; WSIS 2003-2005
HEP Active in WSIS I, GenevaHEP Active in WSIS I, GenevaTheme: Theme: “Creating a “Creating a
Sustainable Process of Sustainable Process of Innovation” Innovation”
CERN RSIS EventCERN RSIS EventSIS Forum & CERN/Caltech SIS Forum & CERN/Caltech
Online Stand at WSIS I (12/03) Online Stand at WSIS I (12/03) > 50 Demos: Advanced Nets > 50 Demos: Advanced Nets & Grids, Global VideoConf., & Grids, Global VideoConf., Telesurgery, “Music Grids”…Telesurgery, “Music Grids”…
Visitors at WSIS I: Visitors at WSIS I: Kofi Annan, UN Sec’y Gen’l Kofi Annan, UN Sec’y Gen’l John H. Marburger, Science John H. Marburger, Science
Adviser to US PresidentAdviser to US PresidentIon Iliescu, President of Ion Iliescu, President of Romania; Romania;
and Dan Nica, and Dan Nica, Minister of ICTMinister of ICT……
World Conference on PhysicsWorld Conference on Physics and Sustainable Developmentand Sustainable Development
Durban, South Africa 10/31-11/2/05Durban, South Africa 10/31-11/2/05
““The World Conference will serve as the first The World Conference will serve as the first global forum to focus the physics global forum to focus the physics
community toward development goals and to community toward development goals and to create new mechanisms of cooperation create new mechanisms of cooperation
toward their achievement.”toward their achievement.”www.saip.org.za/physics2005/WCPSD2005.htmlwww.saip.org.za/physics2005/WCPSD2005.html
WSIS II: TUNIS WSIS II: TUNIS 11/16-11/18/200511/16-11/18/2005www.itu.int/wsiswww.itu.int/wsis
Networks and Grids for HEP and Data Intensive Global Science
Networks and Grids for HEP and Data Intensive Global Science
Networks used by HEP and other fields of DIS are advancing rapidlyNetworks used by HEP and other fields of DIS are advancing rapidly To the 10 G range and now N X 10G; much faster than Moore’s LawTo the 10 G range and now N X 10G; much faster than Moore’s Law New HENP and DOE Roadmaps: Factor ~1000 BW Growth/DecadeNew HENP and DOE Roadmaps: Factor ~1000 BW Growth/Decade
HEP & CS are learning to use long range 10 Gbps networks effectively HEP & CS are learning to use long range 10 Gbps networks effectively 2004-5: 7+ Gbps TCP flows over 20+ kkm; 101 Gbps Record2004-5: 7+ Gbps TCP flows over 20+ kkm; 101 Gbps Record
Transition to community-operated/owned optical R&E networks Transition to community-operated/owned optical R&E networks (us, ca, nl, jp, kr; (us, ca, nl, jp, kr; pl, cz, br, sk, pt, ei, gr, …pl, cz, br, sk, pt, ei, gr, … ) is underway ) is underway
A new era of “hybrid” optical networks & Grid systems is emergingA new era of “hybrid” optical networks & Grid systems is emerging We Must Work to Close to Digital Divide, from Several PerspectivesWe Must Work to Close to Digital Divide, from Several Perspectives
To Allow Scientists in All World Regions to Take Part in DiscoveriesTo Allow Scientists in All World Regions to Take Part in Discoveries Removing Regional, Last Mile, Local Bottlenecks and Removing Regional, Last Mile, Local Bottlenecks and
Compromises in Network Quality are Compromises in Network Quality are On the Critical PathOn the Critical Path Important ExamplesImportant Examples on the Road to Closing the Digital Divide on the Road to Closing the Digital Divide
GLORIAD (US-Russia-China-Korea) Global Optical RingGLORIAD (US-Russia-China-Korea) Global Optical Ring IEEAF “Global Quilt”; NSF: IRNC and IEEAF “Global Quilt”; NSF: IRNC and New Initiative on AfricaNew Initiative on Africa CHEPREO, WHREN and the Brazil HEPGrid in Latin AmericaCHEPREO, WHREN and the Brazil HEPGrid in Latin America Leadership & Outreach: HEP Groups in US, EU, Japan, Korea,Leadership & Outreach: HEP Groups in US, EU, Japan, Korea,
Latin America Latin America
AcknowledgementsAcknowledgements R. AikenR. Aiken A. AliA. Ali S. AltmanovaS. Altmanova P. AveryP. Avery J. BoroumandJ. Boroumand J. BakkenJ. Bakken L. BauerdickL. Bauerdick J. BunnJ. Bunn R. CavanaughR. Cavanaugh H. S. ChenH. S. Chen K. ChoK. Cho G. ColeG. Cole L. CottrellL. Cottrell D. DavidsD. Davids H. DoebblingH. Doebbling J. DolgonasJ. Dolgonas E. FantegrossiE. Fantegrossi D. FosterD. Foster I. FosterI. Foster P. GalvezP. Galvez
J. GruntoradJ. Gruntorad J. IbarraJ. Ibarra V. IlyinV. Ilyin W. JohnstonW. Johnston Y. KaritaY. Karita D. Y. KimD. Y. Kim Y. KimY. Kim K. KwonK. Kwon I. LegrandI. Legrand M. LivnyM. Livny S. LowS. Low O. MartinO. Martin R. MountR. Mount S. McKeeS. McKee G. McLaughlinG. McLaughlin D. Nae D. Nae K. NeggersK. Neggers S. NovaesS. Novaes J. PoolJ. Pool D. PetravickD. Petravick S. RavotS. Ravot
D. ReeseD. Reese D. RileyD. Riley A. SantoroA. Santoro K. SchauerhammerK. Schauerhammer C. SmithC. Smith D. SonD. Son M. StantonM. Stanton C. SteenbergC. Steenberg X. SuX. Su R. SummerhillR. Summerhill M. ThomasM. Thomas F. Van LingenF. Van Lingen E. VelikhovE. Velikhov D. WalstenD. Walsten T. WeisT. Weis T. WestT. West D. WilliamsD. Williams V. WhiteV. White J. P. WuJ. P. Wu F. WuerthweinF. Wuerthwein Y. XiaY. Xia
US DOEUS DOE US NSFUS NSF ESnetESnet EuropeanEuropean
CommissionCommission CERNCERN SLACSLAC FermilabFermilab CACRCACR NLRNLR CENICCENIC Internet2Internet2
/HOPI /HOPI FLRFLR UltraIightUltraIight StarlightStarlight KISTI, KAISTKISTI, KAIST RNP, ANSPRNP, ANSP CiscoCisco NeterionNeterion
Some Extra Slides
Follow
Some Extra Slides
Follow
Int’l Networks BW on Major Links for HENP: US-CERN Example
Int’l Networks BW on Major Links for HENP: US-CERN Example
Rate of Progress >> Moore’s Law (US-CERN Example)Rate of Progress >> Moore’s Law (US-CERN Example) 9.6 kbps Analog9.6 kbps Analog 1985) 1985) 64-256 kbps Digital 64-256 kbps Digital 1989 - 1994) 1989 - 1994) [X 7 – 27][X 7 – 27] 1.5 Mbps Shared 1.5 Mbps Shared 1990-3; IBM) 1990-3; IBM) [X 160][X 160] 2 -4 Mbps2 -4 Mbps 1996-1998) 1996-1998) [X 200-400][X 200-400] 12-20 Mbps 12-20 Mbps 1999-2000) 1999-2000) [X 1.2k-2k][X 1.2k-2k] 155-310 Mbps155-310 Mbps 2001-2) 2001-2) [X 16k – [X 16k –
32k]32k] 622 Mbps622 Mbps 2002-3) 2002-3) [X 65k][X 65k] 2.5 Gbps (2.5 Gbps () ) 2003-4 2003-4 [X 250k][X 250k] 10 Gbps 10 Gbps 2004-5 2004-5 [X 1M] [X 1M] 2 x 10 Gbps 2 x 10 Gbps 2005-6 2005-6 [X 2M][X 2M] 4 4 x 10 Gbps x 10 Gbps ~2007-8~2007-8 [X 4M][X 4M] 8 x 10 Gbps or 2x40Gbps 8 x 10 Gbps or 2x40Gbps ~2009-10~2009-10 [X 8M][X 8M]
A factor of ~1M Bandwidth Improvement over A factor of ~1M Bandwidth Improvement over 1985-2005; A factor of ~5k during 1995-20051985-2005; A factor of ~5k during 1995-2005
HENP has become a leading applications driver, HENP has become a leading applications driver, and also a co-developer of global networks and also a co-developer of global networks
Internet Growth in the World At Large
Internet Growth in the World At Large
Amsterdam Internet Exchange Point Example
20 Gbps
60 Gbps
Average
5 Minute
Max
Some Annual Growth Spurts;
Typically In Summer-Fall
“Acceleration” Last Summer The Rate of HENP Network Usage Growth
(> 100% Per Year) is Not Unlike the World at Large
66 G
bp
s
40 Gbps
Brazil: National backbone in 2003Brazil: National backbone in 2003
RNP2 – May 2003
( 30 Mbps)
M. Stanton
BRAZIL: Nat’l Backbone May 2005BRAZIL: Nat’l Backbone May 2005
RNP – May 2005( 622 Mbps)
BRAZIL: RNPng – 10G & 2.5G Core Network (3Q2005)
BRAZIL: RNPng – 10G & 2.5G Core Network (3Q2005)
Fortaleza
Recife
Salvador
Rio de Janeiro
Belo Horizonte
Brasília
São PauloCuritiba
Florianópolis
Porto Alegre
2.5 Gbps10 Gbps
Core Network Tender
Completed May 12th-13th
The classical Grid architecture had a number of implicit The classical Grid architecture had a number of implicit assumptionsassumptions
The ability to locate and schedule suitable resources,The ability to locate and schedule suitable resources, within a tolerably short time (i.e. resource richness) within a tolerably short time (i.e. resource richness)
Short transactions with relatively simple failure modesShort transactions with relatively simple failure modesHENP Grids are HENP Grids are Data Intensive & Resource-ConstrainedData Intensive & Resource-Constrained
Resource usage governed by local and global policiesResource usage governed by local and global policies Long transactions; some long queuesLong transactions; some long queues
Grid Analysis:Grid Analysis: 1000s of users compete for resources 1000s of users compete for resources at dozens of sites: Complex scheduling; management at dozens of sites: Complex scheduling; management
HENP HENP Stateful, End-to-end Monitored and Tracked ParadigmStateful, End-to-end Monitored and Tracked Paradigm Adopted in OGSA, Now WS Resource FrameworkAdopted in OGSA, Now WS Resource Framework
HENP Data Grids, and Now HENP Data Grids, and Now Services-Oriented GridsServices-Oriented Grids
Grid Analysis: A Real Time SOA Enabling Global Communities
Grid Analysis: A Real Time SOA Enabling Global Communities
Analysis Clients talk standard protocols to the Clarens data/services portal; hides complexity.
Simple Web service API allows diverse Analysis Clients (simple or complex)
Clarens Servers Autodiscover, Auto-connect to form a “Services Fabric”
Key features: Global Scheduler, Catalogs, Monitoring, and Strategic Grid-wide Execution service.
SchedulerCatalogs
Analysis Client
Grid ServicesWeb Server
ExecutionPriority
Manager
Grid WideExecutionService
DataManagement
Fully-ConcretePlanner
Analysis Client
AnalysisClient
Virtual
Data
Replica
ApplicationsMonitoring
Partially-AbstractPlanner
Metadata
HTTP, SOAP, XML-RPC
MonAL
ISA
Clar
ens
ROOT-
Clar
ens/
Cojac/
IGUAN
A
BOSS
RefD
B/MOPD
B
POOL
ORCA
ROOT
FAMOS
Fully-AbstractPlanner
VDT-
Serv
er
Chim
era
MCRun
jobSp
hinx
F. Van Lingen,
M. Thomas
Next generation Information System, with the network as an integrated, Next generation Information System, with the network as an integrated, actively managed subsystem in a global Gridactively managed subsystem in a global Grid
Hybrid network infrastructure: packet-switched + dynamic optical paths Hybrid network infrastructure: packet-switched + dynamic optical paths End-to-end monitoring; Realtime tracking and optimization; End-to-end monitoring; Realtime tracking and optimization;
Dynamic bandwidth provisioning; Dynamic bandwidth provisioning; Agent-based services spanning all layersAgent-based services spanning all layers
http://http://ultralight.caltech.eduultralight.caltech.edu
UERJ, USP
CHEPREO
KNU10
Gbps
Caltech, UF, UMich, FNAL, SLAC, CERN,
UERJ (Rio), USP (Sao Paulo), FIU,KNU (Korea), KEK
NLR, CENIC, UCAID, Translight, UKLight, Netherlight, UvA; UCLondon, Taiwan
Cisco
S. McKee; G. Karmous-Edwards
LISA- Localhost Information Service Agent End To End Monitoring Tool
LISA- Localhost Information Service Agent End To End Monitoring Tool
Easy to deploy & install with any browser; Easy to deploy & install with any browser; user friendly GUI user friendly GUI
Detects system architecture, OS Detects system architecture, OS For all versions of Windows, Linux, Mac.For all versions of Windows, Linux, Mac.
Complete system monitoring of the hostComplete system monitoring of the host CPU, memory, IO, disk, …CPU, memory, IO, disk, …
Hardware detection including Audio, Video Hardware detection including Audio, Video equipment; drivers installed in the system equipment; drivers installed in the system
Provides embedded clients for IPERF Provides embedded clients for IPERF (or other net monitoring tools, e.g. Web 100 ) (or other net monitoring tools, e.g. Web 100 )
LISA and ApMon2:LISA and ApMon2: A basis for strategic, A basis for strategic, end-to-end managed Gridsend-to-end managed Grids
Complete, lightweight monitoring of end user systems & network connectivity. Uses MonALISA framework
to optimize client applications.
Try it: http://monalisa.caltech.edu/lisa/lisa.jnlpTry it: http://monalisa.caltech.edu/lisa/lisa.jnlp I. Legrand
ApMon Grid Analysis Demo: Distributed Processing and Monitoring at SC2004
ApMon Grid Analysis Demo: Distributed Processing and Monitoring at SC2004
Demonstrated Demonstrated how CMS analysis jobs can be submitted to how CMS analysis jobs can be submitted to multiple sites, and monitored from anywhere else on the multiple sites, and monitored from anywhere else on the
gridgrid Three sites: Caltech, Three sites: Caltech, UERJ and USPUERJ and USP Using the “BOSS” job submission and tracking tool,Using the “BOSS” job submission and tracking tool,
running as a “Clarens” Grid service running as a “Clarens” Grid service Job status is monitored by MonALISA, and visualized usingJob status is monitored by MonALISA, and visualized using
(netlogger-style) “lifelines” in real time(netlogger-style) “lifelines” in real time
M. Thomas