Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
Network Update, Building Tomorrow’s Big Science NetworksChris Tracy
Network Planning, ESnet
Lawrence Berkeley National Laboratory
Tech Exchange
San Francisco, CA
October 16, 2017
Mission: To Enable and Accelerate Scientific Discovery by Delivering Unparalleled Network Infrastructure, Capabilities, and Tools
2
Bulk Data Movement
Global Connectivity
Potential network service requirements to support tomorrow’s scientific collaborations include:
Remote Control Applications
Deadline SchedulingTele-Presence
Real Time Data Streaming
Network Content CachingNetwork Security Services
Virtual Private Networks
Superfacility Model
Named Data Networking
Virtual Private CloudsApplication-Network Interaction
Distributed Workflow Integration
ESnet Terrestrial Circuit Updates● Additional redundancy between ESnet and BTAA (aka OmniPoP)
– Starlight to BTAA (100G)– 600 W. Chicago to BTAA (100G)
● Nashville to McLean (100G Express Wave)● NCSA (new 100G Connection)● Equinix 100G upgrades
– High-speed access to shared fabric and cloud providers– Ashburn & Chicago: 100G to fabric, 200G to backbone– San Jose: 100G Upgrade in progress
● Chicago to McLean (100G direct + 100G through Equinix Chicago/Ashburn)● ESnet Measurement Boxes (Disk PTs aka DTNs)
– BNL, ANL, and LBL DTNs → moving to hubs (NEWY, STAR, and SUNN)
3
ESnet Transatlantic Circuit Updates
4
ESnet Transatlantic Circuit Updates
5
Integrate NSF-funded IRNC NEAAR circuit (completed)
ESnet6
Building Tomorrow’s Big Science Networks
6
Our Next-Generation Network will be ESnet6Project Mission Need
● Exponential growth in network CAPACITY needs– 72% year-on-year traffic growth since 1990– Cost effective solution to increase capacity as needed
● Network Life Cycle: Improve RELIABILITY– Replace aging infrastructure– Increase the cyber-resiliency of the network
● Network FLEXIBILITY– Support increasingly complex workflow models– Flexibility at all layers of the network is needed to support wide
spectrum of science requirements
7
ESnet6 Rough Project Timeline
CY2017 CY2018 CY2019 CY2020 CY2021 CY2022
8
May: R&D Conceptual-Arch Selection
Sept: In Process Design Review
More Reviews
Jan: R&D Architecture Review
Project Close-out
Network Build
ESnet6 Service Starts
ESnet5 DecommissioningStarts
Project Funding
ESnet6 Input Requirements
9
1. Capacity• Predict usage• Determine overheads (e.g. burst multiplier, resiliency requirements, short-term
growth trends)2. Services
• Document workflows• Develop service portfolio*
*NB: Service Portfolio in conjunction with architecture design drives the technical requirements
ESnet6 Capacity Planning Process
10
1. Determine predicted baseline usage (for 2020, 2025, and 2030)1. Perform best-fit growth curve of ingress traffic per router2. Adjust individual router predictions such that total of all router ingress traffic
matches ESnet’s 25+ year total traffic growth curve3. Using historical flow data and predicted ingress traffic data, perform full mesh
path computation to determine per link utilization from PE-to-PE2. Strategic capacity planning (for 2020 and 2025)
1. Add burst overhead bandwidth per link based on historical knowledge2. Add additional bandwidth to paths based on resiliency strategy3. Keep in view: new projects on the horizon, unpredictable events
11
LHCONE ramps up From 1.7 PB in December 2014~10x in 8 months To 18.4 PB in July 2015
Long term modeling and capacity prediction continues to be a challengeESnet6 Predicted Usage Map in Jan 2030
100+Tbps speeds at long-haul distances on a single fiber pair is outside the existing optical technology envelope
12
What does capacity really look like in the future?Jan 2021 Bandwidth Capacity Planning Predictions
ESnet6 Services Definition Process
13
• Determine workflows based on requirements workshops– 6 DOE Office of Science program offices:
- Advanced Scientific Computing Research (ASCR) - Basic Energy Sciences (BES)- Biological and Environment Research (BER) - Fusion Energy Sciences (FES)- High Energy Physics (HEP) - Nuclear Physics (NP)
– Two requirements workshops a year– 3 years to rotate through all 6 programs
• ESnet testbed research activities
ESnet SDN Testbed
ESnet6 Workflows, Services, and Technical Requirements
14
Input on requirements are documented as workflows, which are then formalized as services, driving the technical requirements.
Workflows
Example workflow:
Use of Cloud compute resources as an extension of the site’s resources.
Services
Example Service:
Virtual Private Cloud
• Service Description
• Service Attributes– Scale– Scope– Demarcation
TechnicalRequirementsExample Requirements:
• L2VPN– EVPN– MPLS– ISIS-SR– BGP-SR-TE
• e/iBGP
R&D Phase: Architecture and Technologies Matrix
15
Orchestrators
Packet Optical Integration Traditional Routed Software Defined Networking
– Architecture A&B (Packet Optical Integration)• Potentially highly scalable, cost compelling, power & space efficient• All-in-one box requires very careful resiliency planning• Separate transponder and amplifier chain (vendor) domains require some custom integration for optical end-to-end management
ESnet6 R&D Technology Findings in a Nutshell
16
– Architecture C&D (Traditional Routed)• Clean layer separation provides simpler resiliency planning• Cost challenged, potentially unsustainable power and space requirements moving towards 2025
– Architecture E&F (Software Defined Networking)• Highly flexible for resource slicing and service creations (i.e. NFV)• Potentially highly scalable, cost compelling• Non-trivial design complexity (e.g. distinct requirements and designs for management, control, and data planes)• Potential production support complications (e.g. troubleshooting, multi-vendor components, etc)• Lack of maturity for WAN scale solutions
Conclusion: ESnet6 will be a hybrid, containing certain aspects of each architecture and balancing the benefits and trade-offs.
ESnet6 (“Hollow-Core”) Conceptual Architecture Overview
Services Edge• Primary function is to provide customer service handoff• Instantiate services locally at point-of-use (where possible, and using the
Core only for connectivity to other service edges)• Coordination with Core for edge-to-edge services with TE constraints• Reactive functions performed locally with proactive functions orchestrated
centrally• Highly programmable data and control plane Network Elements (NEs)
leveraging SDN concepts to dynamic instantiate (new) services as needed
“Hollow” CoreProgrammable, Scalable, Resilient
17
Services EdgeProgrammable, Flexible, Dynamic
“Hollow” Core• Primary function is to maintain connectivity between Service Edge
nodes• No per-hop L3 routing within the Core, packets will be (label) switched• High capacity bandwidth paths with optical express and line sub-rate
support• Protection and restoration for (Service) Edge-to-Edge connections• Dynamically provisionable bandwidth paths• Centralized intelligence for traffic engineering paths• Low cost to add capacity as needed
ESnet6 Functional Architecture (Optical Core)
18
Packet CoreOptical Core
Open Line System
Colorless and directionless (not contention-less) programmable wavelength switching[Capacity/Flexibility]
Flexible grid system[Capacity]
Flexibility to support 3rd party transponders[Capacity/Flexibility]
Router or switch transponder interface[Capacity]
Dedicated transponder shelf with client Ethernet interfaces[Capacity]
Integrated management system to manage open line system and 3rd party transponders[Reliability]
“Hollow” Core
Services Edge
ESnet6 Functional Architecture (Packet Core)
19
Packet CoreOptical Core
Open Line System
Deep buffers for traffic aggregation and speed mismatches[Capacity]
Traffic engineering and resiliency functions to facilitate per service (instance) design engineering[Resiliency/Flexibility]
“Hollow” Core
Services Edge
Differentiated class of service for packet traffic[Flexibility]
ESnet6 Functional Architecture Services Edge
20
“Hollow” Core
Customer facing protocols[Resiliency]
Protocols to instantiate the services with WAN[Resiliency/Flexibility] Highly programmable and
flexible edge Network Elements (NE)[Flexibility]
Compute resources for Network Function Virtualization (NFV) and Virtual Network Functions (VNFs)[Flexibility]
Support for commercial network functions[Flexibility]Network
Appliance
Services Edge
ESnet6 Deployment Considerations
21
Packet CoreOptical Core
Open Line System
“Hollow” Core
Services Edge
Network Appliance
Single NE serving both Service Edge and Core (packet) functions
Distinct Service Edge NE with programmable control and data-plane (e.g. FPGA or NPU) to support custom encapsulations
Service Edge NE will be programmable control-plane (via an API), but with “fixed” data-plane functions (e.g. Broadcom Jericho)
Components in Service Edge node deployments may vary across customer/peer locations based on required functionality and expenditure
Orchestration to automate service provisioning, common configuration APIs to NEs is desired
Orchestration Monitoring and Measurement
Leverage standards based monitoring and measurements mechanisms and protocols
Dedicated testbed resources for internal/external research, and prototyping new services
• Integrated or turn-key solutions- Pros: - Cons: - Simplified deployment - High CAPEX/OPEX - Well defined support model - Flexibility may be constrained
• Disaggregated or Open Platform solutions- Pros: - Cons - Lower CAPEX/OPEX - Integration work needed - Minimal vendor lock-in - Support complexity
• Custom in-house solution- Pros: - Cons: - Highly customizable - Significant resources needed - Huge potential cost savings
Design Implementation Options
22
Disaggregated and Open Platform solutions have been progressing at a very rapid rate in the last few years and show great promise. ESnet6’s implementation solution may encompass components from all 3 options.
Your Favorite Vendor Name
Here
ESnet6 and Beyond: Build a Smarter Network (Breaking the 50% utilization ceiling)
23
• Informed: Understand past and present conditions– Fully instrumented– Highly granular telemetry information– Assimilates external information (e.g. network black-lists, attack signatures, etc)
• Proactive: Recognizes trends (resource optimizations) and patterns (threat mitigation)– Deep learning– Anomaly detection
• Adaptable: Reconfigure the network or services to optimize for efficiency, effectiveness, and security
All the above requires research, development, and integration to realize!!!!
Looking Towards ESnet7: Assimilates, Anticipates, Adapts
24
Level 0:No Automation
Now
Level 1:Function-specific
AutomationNow
Level 2:Combined Function
Automation2013+
Level 3:Limited Self-Driving
Automation2020+ ?
Level 4:Full Self-Driving
Automation2025+ ?
No Automation
Drive in complete and sole control at all times
Involves 1 or more specific control functions
(e.g. stability control, pre-charged brakes)
Involves automation of at least 2 primary control functions working in unison (e.g. adaptive
cruise control in combination with lane
centering)
Enables all safety-critical functions to be automated
(incl. steering, throttle, brake). The vehicle
monitor any changes in conditions that require a transition back to driver
control
Vehicle is designed to perform all safety-critical
driving functions and monitor road conditions
for an entire trip (includes both occupied and
unoccupied vehicles)
Drive can regain control or stop faster than if driving without the
special function
Driver is temporarily relieved of these driving
functions
Driver not expected to take control at any time
Driver is temporarily relieved of these driving
functions
Source: NHTSA (Modified)
Vehic
le >
< Dr
iver
Level of Driving Automation [NHTSA]Vehicle can:
Assimilates internal and external information (e.g. usage, maintenance schedules, component MTF, driver’s schedule, etc.)
Anticipates trips based on routines and disruptions (e.g. scheduled maintenance) Adapts route and departure time due to road, (current and expected) traffic, and weather conditions
Level “5”:Full-service
Transportation Orchestration and
Automation
• Networks of tomorrow will be cognitive (e.g. Level “5”)• Networks today are primarily directive (e.g. Level 3-4)
25
Thank You and Questions?