Janet Update GridPP Collaboration meeting Sept 2012 David Salmon
Preview:
Citation preview
- Slide 1
- Janet Update GridPP Collaboration meeting Sept 2012 David
Salmon
- Slide 2
- Topics Janet6 Janet & e-Infrastructure Issues &
futures....
- Slide 3
- Janet6 Neil Shewry Rob Evans
- Slide 4
- SuperJANET4/5 Getting us out of the cycle of networks that were
bursting at the seams at the end of their life Dedicated fibre
backbone, allowed us to grow. Subject to contract SJ5 was
originally due to finish October 2011 Extended to October 2013
Still no congestion Originally 10Gbit/s, but have upgraded through
40Gbit/s to 100Gbit/s
- Slide 5
- SuperJANET5 Greater visibility of the optical layer than
previous networks Managed by Verizon Business Janet NOC had
read-only access to (most) transmission equipment Regular meetings
with Verizon on implications of capacity requests Good
communications between Janets senior engineers and Verizons. 40G
and 100G trials before deployment However, still some distance away
from the laser-face Deployment issues that were contractual rather
than technical or operational
- Slide 6
- Janet6: Building up ideas Thinking & reflection as SJ5
progressed Greater visibility of optical layer New features of
transmission systems ROADMs: Colourless; Directionless;
Contentionless (more later) Direct experience of transmission
equipment TVN Developments in Janet and other R&E networks Use
of point-to-point circuits Dynamic provisioning
- Slide 7
- Requirements Gathering
- Slide 8
- Bandwidth, bandwidth, bandwidth... Flexibility Agility Cost
control Service delivery Delivery of third party services Cloud
services Reliability and resilience Partnerships Public/public and
public/private Information assurance Off-net Anytime, anywhere
access Internationalisation of education Management of costs
Funding environment Costs of change
- Slide 9
- Requirements Gathering Functionality Responsiveness to
additional network service requirements Flexibility Protection of
interests of teaching and learning and research sectors
Separability Ability to increase bandwidth at controllable cost
Scalability High reliability by minimising single points of
failure; flexibly coping with major breaks; assisting in resilience
to customer organisations Reliability With agility End-to end
Across JANET Support for the provision of wide range of services:
e.g. High information assurance; Unified comms
- Slide 10
- Procurement Strategy
- Slide 11
- Options appraisal Bespoke arrangement Procure dark fibre
infrastructure Procure optical transmission equipment Management
in-house by the Janet NOC competitive dialogue procedure
- Slide 12
- Procurement Strategy Fibre Procurement Lot 1: Fibre
Infrastructure in the UK Options: Scotland, Wales, Ireland, Aurora,
London, Research Lot 2: Fibre Infrastructure in Ireland Framework
Agreement(s) for 4 years Initial order from 5+2 years to 10+5
years
- Slide 13
- Procurement Strategy Transmission equipment procurement Supply,
design, delivery, installation, commissioning, maintenance,
training, specialist technical support Framework Agreement for 7
years Initial order 3 + 2 + 2 years
- Slide 14
- Project Timetable
- Slide 15
- Slide 16
- Procurement Update:
- Slide 17
- Fibre Procurement Prequalification phase Launched October 2011
13 responses Shortlist 6 bidders
- Slide 18
- Fibre Procurement UK Government approval to proceed Cabinet
Office July 2012 Contract Signed July 2012
- Slide 19
- Transmission Equipment Procurement Prequalification phase
Launched November 2011 17 responses Shortlist 6 bidders
- Slide 20
- Transmission Equipment Procurement Contract signed last week
September 2012
- Slide 21
- Current Status
- Slide 22
- Slide 23
- From requirements to services Capacity Exponential growth
continues Commodity IP networking R&E requirements Must be able
to scale the network Not just more channels, but faster channels
over the networks life 100Gbit/s to start 400Gbit/s? 1Tbit/s?
- Slide 24
- From requirements to services Ubiquitous connectivity Access to
Janet from anywhere Not all in the scope of Janet6 Other activities
in the company Feeds the capacity requirements Feeds the external
connection requirements Feeds the reliability requirements
- Slide 25
- From requirements to services Distributed campuses Inter-site,
intra-organisation connections VPNs Separacy Centralised
firewall
- Slide 26
- From requirements to services Outsourced services Compute, data
storage, applications Low latency for interactive services High
bandwidth for bulk data services Reliability, reliability,
reliability.
- Slide 27
- From requirements to services Research National and global
collaboration Interoperation with GEANT and other R&E networks
worldwide LHC, SKA, ITER, Bioinformatics, Climate Data deluge
Services Dynamic provisioning of layer 2 circuits
- Slide 28
- Similar topology to SJ5. Two north-south paths New east-west
path from Birmingham to Nottingham Erdington to Lowdham No backbone
PoP at Reading Still have a regional PoP STFC RAL as a backbone PoP
New PoP in Acton, West London Janet6: Core Fibre and Backbone
Resilience
- Slide 29
- Illustrative only The details are subject to change Similar to
SJ5 Collector arcs Spurs Regional networks connect to two different
routers Over 70 optical nodes Amplifier sites, add/drop sites, and
router sites Janet6: Regional fibre
- Slide 30
- Lighting the fibre DWDM transmission equipment Still some way
to go in the procurement Theoretically 80 channels of 100Gbit/s per
channel Roadmap for 400Gbit/s and 1Tbit/s with better spectral
density ROADMs Native capacity on the core to be 100Gbit/s 10Gbit/s
multiplexed over that where needed Muxponders or OTN switching
Coherent technology required for 100Gbit/s doesnt sit well with
dispersion compensation required for 10Gbit/s RNEP fibre may still
carry 10G native
- Slide 31
- 10G Circuits: muxponders or switched? Muxponders Switching
- Slide 32
- ROADMs Will be more reconfigurable than SJ5 One of the
collector arcs in Scotland required 13 engineers to be in dotted
along it at the same time to commission Get away from the air gap
between RNEP fibre and backbone fibre that exists on SJ5. Different
transmission systems on collector arcs and backbone Requires two
client cards and a patch cable to bridge a circuit from one to the
other
- Slide 33
- Building the network and the NOC What will this give us? Janet
NOC will have a view from the fibre up the stack to the routers We
will see when fibres/lasers/receivers degrade Lower signal levels
Increase in error rate Forward Error Correction (FEC) working
harder We will see where faults are Fewer administrative boundaries
to cross Directly translate the communitys requirements into
engineering
- Slide 34
- IP service As you were Finish moving away from SDH access
towards ethernet 802.1ad bonding for 20Gbit/s, 30Gbit/s etc. 100GE
if you can afford the optics (have seen a dramatic fall) Still
IPv4, IPv6, unicast, multicast In London, retaining Telehouse and
Telecity Harbour Exchange as major points of external connection
Continuing to expand Telecity Manchester Content providers and
content delivery networks IX Manchester operated by LINX
- Slide 35
- VPNs VPNs over primary access Inter-site connectivity Low
bandwidth layer 2 circuits Other services? Statically configured
VLANs (site/RN boundary, perhaps RN/core) BGP signalled VPNs
(RN/core, perhaps some sites) Bandwidth management between IP and
VPNs These are layer 2/3 VPNs Not IPsec If you want to be sure it
is encrypted, best to do that yourself Would you trust this man
with your encryption keys?
- Slide 36
- Lightpath service Good question Ethernet over MPLS network will
remain Upgrade the MX960s to 100GE interconnects Move 10Gbit/s
circuits from dedicated wavelengths to EoMPLS Investigating OTN
switching as part of transmission procurement May be too early to
do much especially as well also have MX960 platform Dont need
guaranteed bandwidth? See VPN service
- Slide 37
- Migration 66.666666666666667% (ish) chance well be moving core
PoPs Moving routers from PoP to PoP overnight would be challenging
Requires all regional networks connected to a PoP to move
simultaneously (no link between old PoP & new PoP) Cant move
anything until all the regional network fibres to a PoP are ready
One delay holds up everything Remember the newts! Need to upgrade
the switching fabric anyway T640 to T1600 T1600 to T4000
Investigating options & costs
- Slide 38
- Migration Aiming to have the network installed and ready by the
end of March 2013 Migration between March and August Post-Clearing
until end of October is contingency only End of October is a hard
cut-off.
- Slide 39
- Janet6 summary Extensive fibre footprint Direct management of
optical transmission On-schedule Very capable strategic foundation
10 years+
- Slide 40
- e-Infrastructure Networking David Salmon
- Slide 41
- Topics e-Infrastructure funding What has Janet been doing ?
Emerging Issues
- Slide 42
- Janet e-Infrastructure networking 26MWith HEFCE Breakdown
10MContribution to Janet6 backbone 12M Network provision within
classic Janet community, including key locations which may need
additional fibre access 4M Industry connection
- Slide 43
- e-Infrastructure programme support CTO Bob Day on ELC Paul
Lewis secondment to BIS programme Office 8 months STFC Michael
Wilson secondment just started funded from Janets e-Infrastructure
allocation
- Slide 44
- e-Infrastructure Strategic Sites Potential fibre / capacity
requirements Work with organisations to understand requirements
Build cases for provision If approved, integrate provision with
deployment of Janet6
- Slide 45
- Patterns of use Data movement working patterns How much data
volume ? Beware Bytes vs. bits ! How quickly ? How often ? Where to
? National & International Network engineers need bit-rates !
Working with research communities to understand emerging/evolving
requirements 45
- Slide 46
- Norwich Bioinformatics Cluster The Genome Analysis Centre -
TGAC The Sainsbury Laboratory John Innes Centre Institute for Food
Research University of East Anglia Norwich Research Park
- Slide 47
- Hinxton Genome Campus European Molecular Biology Laboratory -
European Bioinformatics Institute The Sanger Centre Babraham
Institute
- Slide 48
- Exeter & locale Met Office University of Exeter Weather
& Climate research Monsoon HPC NERC & M.O.
- Slide 49
- Broaden scope of Janets fibre infrastructure Complementary
& integrated with Janet6
- Slide 50
- e-Infrastructure projects funded HPC 5 regional consortia
National facility Hector / Archer Dirac Particle physics NERC STFC
RAL Daresbury will engage to discuss requirements &
evolution
- Slide 51
- Outcome Outline proposals made Approvals discussions with
Research Councils Must converge soon to optimise Janet6 fibre
layout Some budget remains Likely to be a further proposals
Academic & Industry locations ?
- Slide 52
- Some implications 52
- Slide 53
- Throughput challenges Infrastructures can deliver very high
capacities Realising high-throughput end-to-end will take some
effort Cant just take systems out of the box and expect to get full
line-rate data-transfers This needs to be communicated &
understood more widely Janet & NREN community needs to work
more closely with research communities to understand and solve
issues. 53
- Slide 54
- Achievable Throughput What is realistic ? Challenge: 1Gbit/s,
24hr ~= 10TBytes Holistic approach End systems - Hardware &
Software - Both ends ! Local networks Wide area networks Highlight
successful examples Technical case studies Janet role ?
Co-ordination/collaboration Advice & troubleshooting support
?
- Slide 55
- Network structure implications ? Large research communities
International/global overlays or private networks Particle physics
- LHC OPN (LHCONE) Radio Astronomy eVLBI, LOFAR, (SKA...) Future
Bioinformatics ?? ELIXIR Campus implications ? point to point
services (lightpaths) more complex multi-homing routing/switching
issues
- Slide 56
- Industry ? Existing Janet policy Supporting collaborative
R&D Business and Community Engagement (BCE) Evolution - more
needed ? e-Infrastructure Open and accessible Janet
- Slide 57
- Connecting Industry meaning ? Network connections ? Access to
community systems & resources connected to Janet ? Access to
Janet community knowledge & expertise ? Mix of all three... BIS
& HPC Consortia views ?
- Slide 58
- Issues & futures Good planning & dialogue between Janet
& GridPP continue ! understand evolving requirements address
these as they emerge Involved in LHCOPN & LHCONE meetings Rob
Evans (Janet) Technical input Balancing UK requirements, GEANT
& other NRENs in Europe, and US & global NRENs
- Slide 59
- GidPP networking archaeology....(2000)
- Slide 60
- Conclusion Janet6 on-track Very capable strategic platform
supporting UK Research and Education for 10 years+ e-Infrastructure
Approvals soon hopefully Challenge of delivery & deployment
Industry ? Using the network Achieving higher data throughput
- Slide 61
- ?
- Slide 62
- Janet Aurora
- Slide 63
- Last presented at CEF in 2010 Very good results produced High
profile papers at conferences But, funding issues have held back
full development
- Slide 64
- Context - Funding History e-Science and JISC Funding 4.35M
2007-2012 Fibre infrastructure only Fibre access (single pair) ILA
rental space for researchers equipment 1 rack basic telecomms for
management access Tax lit fibres are subject to business rates in
UK NOT Research equipment Staff effort
- Slide 65
- JANET Aurora Intermediate equipment co-location point
University / JANET access point Fibre spans International Lightpath
access to other NRENs via JANET & GEANT JANET Lightpath access
to other UK Locations University of Cambridge University of Essex
UCL University of Southampton Telehouse London 550km of fibre-pairs
VM Brentford PoP VM Camberley PoP VM Crawley Court PoP VM
Chelmsford PoP VM Enfield PoP VM Ipswich PoP VM Bury St Edmonds PoP
31km 52km 58km 43km CAL483331-1 (28km) 30km 55km 57km 54km 50km
72km
- Slide 66
- Current position Janet will maintain Aurora in current form
until Sept/Oct 2013 Aurora / Dark Fibre facility WILL need to
evolve over next 12-18 months Can not extend contracts for current
infrastructure any further (procurement law) The Essex research
group is moving to Bristol (Prof. Dimitra Simeonidou) Research
Council (EPSRC) funding for a Dark-Fibre Facility is still being
considered Equipment and staff effort to exploit dark-fibre as a
test platform
- Slide 67
- e-Infrastructure New funding form UK government 100M+ into HPC
National Regional 26M to Janet to continue and extend capability
Extend Janet6 to key locations Biosciences Climate and Weather
Research HPC Dark-Fibre facility / Aurora MAY also be possible
Exploring with Research Councils
- Slide 68
- Recent Results ECOC next week Dimitra Simeonidou &
colleagues will present interesting new results Collaborations with
NPL continue Ultra-stable lasers NPL and UCL Optical comb transfer
NPL and Southampton Janet in discussion with NPL about
participation in Aurora NPL has independent funding, and plans to
deploy fibre for comb- transfers Following slides from Radan Slavik
UFE, Prague ORC, Southampton
- Slide 69
- JANET Aurora Dark Fiber Link Intermediate equipment co-location
point University / JANET access point Fibre spans International
Lightpath access to other NRENs via JANET & GEANT JANET
Lightpath access to other UK Locations University of Cambridge
University of Essex Aston University UCL University of Southampton
Telehouse London 850km of fibre-pairs University of Southampton
Telehouse London 40 km 60 km 40 km Southampton to London part of
the link: 200 km, dispersion-compensated, 6 double in-line
flat-gain CLA EDFAs with maximum input/output powers of -5/15 dBm.
Experiments: (A) Southampton-London roundtrip for phase
regeneration; (B) Southampton-Essex roundtrip for optical
wavelength conversion.
- Slide 70
- Phase and amplitude all-optical regenerator All-optical
wavelength conversion Taken from University of Southampton, U.K.
(V. Rancano; results under progress) CLA amplifiers used Taken from
University of Southampton, U.K. (R. Slavk et al, Nature Photonics
2010 and JLT 2011)