SuperJANET4/5 Getting us out of the cycle of networks that were
bursting at the seams at the end of their life Dedicated fibre
backbone, allowed us to grow. Subject to contract SJ5 was
originally due to finish October 2011 Extended to October 2013
Still no congestion Originally 10Gbit/s, but have upgraded through
40Gbit/s to 100Gbit/s
Slide 5
SuperJANET5 Greater visibility of the optical layer than
previous networks Managed by Verizon Business Janet NOC had
read-only access to (most) transmission equipment Regular meetings
with Verizon on implications of capacity requests Good
communications between Janets senior engineers and Verizons. 40G
and 100G trials before deployment However, still some distance away
from the laser-face Deployment issues that were contractual rather
than technical or operational
Slide 6
Janet6: Building up ideas Thinking & reflection as SJ5
progressed Greater visibility of optical layer New features of
transmission systems ROADMs: Colourless; Directionless;
Contentionless (more later) Direct experience of transmission
equipment TVN Developments in Janet and other R&E networks Use
of point-to-point circuits Dynamic provisioning
Slide 7
Requirements Gathering
Slide 8
Bandwidth, bandwidth, bandwidth... Flexibility Agility Cost
control Service delivery Delivery of third party services Cloud
services Reliability and resilience Partnerships Public/public and
public/private Information assurance Off-net Anytime, anywhere
access Internationalisation of education Management of costs
Funding environment Costs of change
Slide 9
Requirements Gathering Functionality Responsiveness to
additional network service requirements Flexibility Protection of
interests of teaching and learning and research sectors
Separability Ability to increase bandwidth at controllable cost
Scalability High reliability by minimising single points of
failure; flexibly coping with major breaks; assisting in resilience
to customer organisations Reliability With agility End-to end
Across JANET Support for the provision of wide range of services:
e.g. High information assurance; Unified comms
Slide 10
Procurement Strategy
Slide 11
Options appraisal Bespoke arrangement Procure dark fibre
infrastructure Procure optical transmission equipment Management
in-house by the Janet NOC competitive dialogue procedure
Slide 12
Procurement Strategy Fibre Procurement Lot 1: Fibre
Infrastructure in the UK Options: Scotland, Wales, Ireland, Aurora,
London, Research Lot 2: Fibre Infrastructure in Ireland Framework
Agreement(s) for 4 years Initial order from 5+2 years to 10+5
years
Slide 13
Procurement Strategy Transmission equipment procurement Supply,
design, delivery, installation, commissioning, maintenance,
training, specialist technical support Framework Agreement for 7
years Initial order 3 + 2 + 2 years
Transmission Equipment Procurement Contract signed last week
September 2012
Slide 21
Current Status
Slide 22
Slide 23
From requirements to services Capacity Exponential growth
continues Commodity IP networking R&E requirements Must be able
to scale the network Not just more channels, but faster channels
over the networks life 100Gbit/s to start 400Gbit/s? 1Tbit/s?
Slide 24
From requirements to services Ubiquitous connectivity Access to
Janet from anywhere Not all in the scope of Janet6 Other activities
in the company Feeds the capacity requirements Feeds the external
connection requirements Feeds the reliability requirements
Slide 25
From requirements to services Distributed campuses Inter-site,
intra-organisation connections VPNs Separacy Centralised
firewall
Slide 26
From requirements to services Outsourced services Compute, data
storage, applications Low latency for interactive services High
bandwidth for bulk data services Reliability, reliability,
reliability.
Slide 27
From requirements to services Research National and global
collaboration Interoperation with GEANT and other R&E networks
worldwide LHC, SKA, ITER, Bioinformatics, Climate Data deluge
Services Dynamic provisioning of layer 2 circuits
Slide 28
Similar topology to SJ5. Two north-south paths New east-west
path from Birmingham to Nottingham Erdington to Lowdham No backbone
PoP at Reading Still have a regional PoP STFC RAL as a backbone PoP
New PoP in Acton, West London Janet6: Core Fibre and Backbone
Resilience
Slide 29
Illustrative only The details are subject to change Similar to
SJ5 Collector arcs Spurs Regional networks connect to two different
routers Over 70 optical nodes Amplifier sites, add/drop sites, and
router sites Janet6: Regional fibre
Slide 30
Lighting the fibre DWDM transmission equipment Still some way
to go in the procurement Theoretically 80 channels of 100Gbit/s per
channel Roadmap for 400Gbit/s and 1Tbit/s with better spectral
density ROADMs Native capacity on the core to be 100Gbit/s 10Gbit/s
multiplexed over that where needed Muxponders or OTN switching
Coherent technology required for 100Gbit/s doesnt sit well with
dispersion compensation required for 10Gbit/s RNEP fibre may still
carry 10G native
Slide 31
10G Circuits: muxponders or switched? Muxponders Switching
Slide 32
ROADMs Will be more reconfigurable than SJ5 One of the
collector arcs in Scotland required 13 engineers to be in dotted
along it at the same time to commission Get away from the air gap
between RNEP fibre and backbone fibre that exists on SJ5. Different
transmission systems on collector arcs and backbone Requires two
client cards and a patch cable to bridge a circuit from one to the
other
Slide 33
Building the network and the NOC What will this give us? Janet
NOC will have a view from the fibre up the stack to the routers We
will see when fibres/lasers/receivers degrade Lower signal levels
Increase in error rate Forward Error Correction (FEC) working
harder We will see where faults are Fewer administrative boundaries
to cross Directly translate the communitys requirements into
engineering
Slide 34
IP service As you were Finish moving away from SDH access
towards ethernet 802.1ad bonding for 20Gbit/s, 30Gbit/s etc. 100GE
if you can afford the optics (have seen a dramatic fall) Still
IPv4, IPv6, unicast, multicast In London, retaining Telehouse and
Telecity Harbour Exchange as major points of external connection
Continuing to expand Telecity Manchester Content providers and
content delivery networks IX Manchester operated by LINX
Slide 35
VPNs VPNs over primary access Inter-site connectivity Low
bandwidth layer 2 circuits Other services? Statically configured
VLANs (site/RN boundary, perhaps RN/core) BGP signalled VPNs
(RN/core, perhaps some sites) Bandwidth management between IP and
VPNs These are layer 2/3 VPNs Not IPsec If you want to be sure it
is encrypted, best to do that yourself Would you trust this man
with your encryption keys?
Slide 36
Lightpath service Good question Ethernet over MPLS network will
remain Upgrade the MX960s to 100GE interconnects Move 10Gbit/s
circuits from dedicated wavelengths to EoMPLS Investigating OTN
switching as part of transmission procurement May be too early to
do much especially as well also have MX960 platform Dont need
guaranteed bandwidth? See VPN service
Slide 37
Migration 66.666666666666667% (ish) chance well be moving core
PoPs Moving routers from PoP to PoP overnight would be challenging
Requires all regional networks connected to a PoP to move
simultaneously (no link between old PoP & new PoP) Cant move
anything until all the regional network fibres to a PoP are ready
One delay holds up everything Remember the newts! Need to upgrade
the switching fabric anyway T640 to T1600 T1600 to T4000
Investigating options & costs
Slide 38
Migration Aiming to have the network installed and ready by the
end of March 2013 Migration between March and August Post-Clearing
until end of October is contingency only End of October is a hard
cut-off.
Slide 39
Janet6 summary Extensive fibre footprint Direct management of
optical transmission On-schedule Very capable strategic foundation
10 years+
Slide 40
e-Infrastructure Networking David Salmon
Slide 41
Topics e-Infrastructure funding What has Janet been doing ?
Emerging Issues
Slide 42
Janet e-Infrastructure networking 26MWith HEFCE Breakdown
10MContribution to Janet6 backbone 12M Network provision within
classic Janet community, including key locations which may need
additional fibre access 4M Industry connection
Slide 43
e-Infrastructure programme support CTO Bob Day on ELC Paul
Lewis secondment to BIS programme Office 8 months STFC Michael
Wilson secondment just started funded from Janets e-Infrastructure
allocation
Slide 44
e-Infrastructure Strategic Sites Potential fibre / capacity
requirements Work with organisations to understand requirements
Build cases for provision If approved, integrate provision with
deployment of Janet6
Slide 45
Patterns of use Data movement working patterns How much data
volume ? Beware Bytes vs. bits ! How quickly ? How often ? Where to
? National & International Network engineers need bit-rates !
Working with research communities to understand emerging/evolving
requirements 45
Slide 46
Norwich Bioinformatics Cluster The Genome Analysis Centre -
TGAC The Sainsbury Laboratory John Innes Centre Institute for Food
Research University of East Anglia Norwich Research Park
Slide 47
Hinxton Genome Campus European Molecular Biology Laboratory -
European Bioinformatics Institute The Sanger Centre Babraham
Institute
Slide 48
Exeter & locale Met Office University of Exeter Weather
& Climate research Monsoon HPC NERC & M.O.
Slide 49
Broaden scope of Janets fibre infrastructure Complementary
& integrated with Janet6
Slide 50
e-Infrastructure projects funded HPC 5 regional consortia
National facility Hector / Archer Dirac Particle physics NERC STFC
RAL Daresbury will engage to discuss requirements &
evolution
Slide 51
Outcome Outline proposals made Approvals discussions with
Research Councils Must converge soon to optimise Janet6 fibre
layout Some budget remains Likely to be a further proposals
Academic & Industry locations ?
Slide 52
Some implications 52
Slide 53
Throughput challenges Infrastructures can deliver very high
capacities Realising high-throughput end-to-end will take some
effort Cant just take systems out of the box and expect to get full
line-rate data-transfers This needs to be communicated &
understood more widely Janet & NREN community needs to work
more closely with research communities to understand and solve
issues. 53
Slide 54
Achievable Throughput What is realistic ? Challenge: 1Gbit/s,
24hr ~= 10TBytes Holistic approach End systems - Hardware &
Software - Both ends ! Local networks Wide area networks Highlight
successful examples Technical case studies Janet role ?
Co-ordination/collaboration Advice & troubleshooting support
?
Slide 55
Network structure implications ? Large research communities
International/global overlays or private networks Particle physics
- LHC OPN (LHCONE) Radio Astronomy eVLBI, LOFAR, (SKA...) Future
Bioinformatics ?? ELIXIR Campus implications ? point to point
services (lightpaths) more complex multi-homing routing/switching
issues
Slide 56
Industry ? Existing Janet policy Supporting collaborative
R&D Business and Community Engagement (BCE) Evolution - more
needed ? e-Infrastructure Open and accessible Janet
Slide 57
Connecting Industry meaning ? Network connections ? Access to
community systems & resources connected to Janet ? Access to
Janet community knowledge & expertise ? Mix of all three... BIS
& HPC Consortia views ?
Slide 58
Issues & futures Good planning & dialogue between Janet
& GridPP continue ! understand evolving requirements address
these as they emerge Involved in LHCOPN & LHCONE meetings Rob
Evans (Janet) Technical input Balancing UK requirements, GEANT
& other NRENs in Europe, and US & global NRENs
Slide 59
GidPP networking archaeology....(2000)
Slide 60
Conclusion Janet6 on-track Very capable strategic platform
supporting UK Research and Education for 10 years+ e-Infrastructure
Approvals soon hopefully Challenge of delivery & deployment
Industry ? Using the network Achieving higher data throughput
Slide 61
?
Slide 62
Janet Aurora
Slide 63
Last presented at CEF in 2010 Very good results produced High
profile papers at conferences But, funding issues have held back
full development
Slide 64
Context - Funding History e-Science and JISC Funding 4.35M
2007-2012 Fibre infrastructure only Fibre access (single pair) ILA
rental space for researchers equipment 1 rack basic telecomms for
management access Tax lit fibres are subject to business rates in
UK NOT Research equipment Staff effort
Slide 65
JANET Aurora Intermediate equipment co-location point
University / JANET access point Fibre spans International Lightpath
access to other NRENs via JANET & GEANT JANET Lightpath access
to other UK Locations University of Cambridge University of Essex
UCL University of Southampton Telehouse London 550km of fibre-pairs
VM Brentford PoP VM Camberley PoP VM Crawley Court PoP VM
Chelmsford PoP VM Enfield PoP VM Ipswich PoP VM Bury St Edmonds PoP
31km 52km 58km 43km CAL483331-1 (28km) 30km 55km 57km 54km 50km
72km
Slide 66
Current position Janet will maintain Aurora in current form
until Sept/Oct 2013 Aurora / Dark Fibre facility WILL need to
evolve over next 12-18 months Can not extend contracts for current
infrastructure any further (procurement law) The Essex research
group is moving to Bristol (Prof. Dimitra Simeonidou) Research
Council (EPSRC) funding for a Dark-Fibre Facility is still being
considered Equipment and staff effort to exploit dark-fibre as a
test platform
Slide 67
e-Infrastructure New funding form UK government 100M+ into HPC
National Regional 26M to Janet to continue and extend capability
Extend Janet6 to key locations Biosciences Climate and Weather
Research HPC Dark-Fibre facility / Aurora MAY also be possible
Exploring with Research Councils
Slide 68
Recent Results ECOC next week Dimitra Simeonidou &
colleagues will present interesting new results Collaborations with
NPL continue Ultra-stable lasers NPL and UCL Optical comb transfer
NPL and Southampton Janet in discussion with NPL about
participation in Aurora NPL has independent funding, and plans to
deploy fibre for comb- transfers Following slides from Radan Slavik
UFE, Prague ORC, Southampton
Slide 69
JANET Aurora Dark Fiber Link Intermediate equipment co-location
point University / JANET access point Fibre spans International
Lightpath access to other NRENs via JANET & GEANT JANET
Lightpath access to other UK Locations University of Cambridge
University of Essex Aston University UCL University of Southampton
Telehouse London 850km of fibre-pairs University of Southampton
Telehouse London 40 km 60 km 40 km Southampton to London part of
the link: 200 km, dispersion-compensated, 6 double in-line
flat-gain CLA EDFAs with maximum input/output powers of -5/15 dBm.
Experiments: (A) Southampton-London roundtrip for phase
regeneration; (B) Southampton-Essex roundtrip for optical
wavelength conversion.
Slide 70
Phase and amplitude all-optical regenerator All-optical
wavelength conversion Taken from University of Southampton, U.K.
(V. Rancano; results under progress) CLA amplifiers used Taken from
University of Southampton, U.K. (R. Slavk et al, Nature Photonics
2010 and JLT 2011)