Upload
devops4networks
View
121
Download
0
Tags:
Embed Size (px)
DESCRIPTION
What If Your Network Was Smarter Than You?
Citation preview
1
Jeremy Pollard
What If Your Network Was Smarter Than You?
2
Who Am I?• Jeremy Pollard
• Network Engineer @ Box.com
• SIGGRAPH2015 GraphicsNet Committee Chair
• Automator
• Lindy-Hop and Blues Dancer
3
Complete Network OverhaulNetworks that grow organically don’t scale, news to no one.
4
Network Overhaul
• Old design grew as needed‒ Need a switch? Add a switch.‒ Flat layer 2 design.‒ Did not Scale.
• New Design‒ Greenfield!‒ New hardware!‒ New design!‒ New Datacenter!
5
“Let’s build a smarter network.
Said everyone, everywhere.
6
How do we do this?What are we trying to solve?
7
We’re Network Engineers…
8
And We Like…
• Standards
• Specifications
• Designing with scalability in mind
• Repeatable patterns
9
And Yet We Still Have To Answer Questions Like…
• Which IP address should I use?
• Where is this host located?
• Do you know how this device is supposed to be cabled?
• Which port should I use?
• Did you configure that new switch?
10
Boring
11
Error Prone
12
A Waste Of Time
13
Cost The Company $$$
14
How Did Box Approach This?By thinking outside the Box… HA! Get it?!
*crickets*
15
New Network Design
• Core / Agg / ToR model
• Fully routed to the ToR
• Two ToRs per cabinet
• Pattern based port assignment
• Mathematically generated ‒ IP addresses‒ Hostnames‒ VLANs
• ID numbers to indicate Datacenter, Pod, Cabinet‒ More on this later!
In 30 seconds or less
16
For Every Pair of ToRs
• Over 300 pieces of unique information‒ IP addresses/subnets‒ Pinned routes‒ Radius / Logging / NTP / etc servers‒ Interface descriptions
• ~180 DNS records
• Cabling instructions‒ 8 upstream port assignments‒ 2 Serial consoles‒ 2 management ports
17
Highly Complex
18
Highly Automatable
19
Time to build a smarter network
20
The Infrastructure API
21
Infrastructure API
• HTTP based REST API
• All things IP / Network / Datacenter
• Single source of truth
22
It’s our design specification
23
It’s our design specification
Implemented in code
24
Infrastructure API
• IP address management for network devices and hosts‒ In-band and Out-of-Band
• Hostname generation
• DNS registration
• Generates all 300 unique pieces of info for ToR provisioning
• Generates physical cable mappings and port assignments
• Host to Security zone mapping
• Provide network information for a given IP
• Provide physical location for a given IP
25
Infrastructure API
• Returns JSON objects
• Easily integrates into token-based templates‒ Full text configuration‒ Cabling instructions
• Can be easily integrated into other services
26
How Does It Work?
27
Fundamentals First
• Procedurally Generated
• Single Seed
• Remember the IDs?‒ Datacenter‒ Pod‒ Cabinet‒ Host Type (Production side only)‒ Rack-u (Out-of-Band side only)
Static HostDatacenter Pod Type
0001010.10101000.10100001.00010100Cab
28
Seeds
• IP - > Datacenter / Pod / Cabinet / Type IDs
• IDs - > Everything Else‒ $cab_count = ($MAX_POD_SIZE * $pod_id - 1 ) + $cab_id‒ $hostname = sprintf(‘tsw%02d’, $cab_count)‒ $serial_server_number = $cab_count / 32 + 7($pod_id - 1) + 4‒ $serial_port_number = 33 + (($cab_count - 1) % 32) / 2
• And so on…
29
New Switch ProvisioningA Use Case
30
In The Datacenter
• DC Tech enters rack information to get cabling specifications for the cabinet
31
Once Racking and Cabling is Complete:
• Manually Configure the management IP address‒ This will be our seed!‒ We’re working on DHCP…
• Download provision.sh to the switch and execute.‒ Downloads latest EOS‒ Detects management IP‒ API Call: device_config with management IP as the argument
‒ Infrastructure API generates the config‒ Config is then saved to startup-config
‒ API Call: register_dns with management IP as the argument‒ Infrastructure API calls our DNS API to register all records
‒ Download first_boot.sh‒ Reboot device
32
After Reboot
• first_boot.sh executed 2 minutes after boot
• API Call: inventory_update‒ Inventory API scans the device collecting:
‒ Hostname‒ Serial Numbers‒ Interface IP Addresses‒ Interface States
• Success!!‒ Switch successfully provisioned‒ Automatically added to monitoring
33
Other Uses?
34
Other uses?
• Core / Datacenter teams host provisioning‒ Host IP address assignment‒ Hostname generation / DNS registration
• Hadoop rack awareness
• Assists in automating inventory audits‒ Physical / logical mappings‒ Host locating
• If you build it, they will come.
35
Humans are still needed… Right?Right?!
36
You Bet!
• All those IDs need to be defined
– Thankfully it’s crazy easy!
• YAML based data structure
• Datacenters are assigned pods
• Pods exist in cages
• Pods are assigned Cabs
• Etc…
37
We’re just not answering these questions anymore…
• Which IP address should I use?
• Where is this host located?
• Do you know how this device is supposed to be cabled?
• Which port should I use?
• Did you configure that new switch?
38
“This sounds great! But what are the potential problems?
- Said anyone still paying attention
39
Problems…
• Screw up ID allocation
• DC Tech cabled devices incorrectly or incorrect physical location
• Need to move an existing cab to another pod
• Bugs!
40
What’s Next?To the future!!
41
Yet To Come
• Get DHCP working for management addresses
• Dynamically generate topology diagrams‒ Graphviz‒ D3‒ Take your pick
• Automated validation of link health‒ Up / Down‒ Light levels‒ Db loss
42
Thanks!