SDN AND OPENFLOW
THE HYPE AND THE HARSH REALITY
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page ii
SDN AND OPENFLOW
THE HYPE AND THE HARSH REALITY
Ivan Pepelnjak, CCIE#1354 Emeritus
Copyright 2014 ipSpace.net AG
WARNING AND DISCLAIMER
This book is a collection of blog posts written between March 2011 and the book publication date,
providing independent information about Software Defined Networking and OpenFlow. Every effort
has been made to make this book as complete and as accurate as possible, but no warranty or
fitness is implied. Read the introductory paragraphs before the blog post headings to understand the
context in which the blog posts have been written, and make sure you read the Introduction section.
The information is provided on an as is basis. The authors, and ipSpace.net shall have neither
liability nor responsibility to any person or entity with respect to any loss or damages arising from
the information contained in this book.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page iii
CONTENT AT A GLANCE
FOREWORD ............................................................................................................................ IV
INTRODUCTION ..................................................................................................................... VI
1 THE INITIAL HYPE ...................................................................................................... 1-1
2 SOFTWARE DEFINED NETWORKING 101 ................................................................ 2-1
3 OPENFLOW BASICS ................................................................................................... 3-1
4 OPENFLOW IMPLEMENTATION NOTES .................................................................... 4-1
5 OPENFLOW SCALABILITY CHALLENGES .................................................................. 5-1
6 OPENFLOW AND SDN USE CASES .......................................................................... 6-1
7 SDN BEYOND OPENFLOW ...................................................................................... 7-1
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page iv
FOREWORD
Ivan asked me to write the intro for his latest book on Software Defined Networking and I'm a bit
mystified why. Granted, he's like the control plane to my forwarding plane. The brilliant technical
insights I've gathered from Ivan's web site and webinars have provided me with valuable content
and creative inspiration ever since I first discovered it. In fact, I almost feel like I'm cheating at my
job. Every time I clarify SDN in a conversation with, "It's the decoupling of the logical from the
physical," I want to insert a footnote referencing him.
I remember the first time I heard him on a podcast, I thought to myself "This guy must be super
smart, because he sounds like a Bond villain and I can only grasp 50% of what he's saying." I
started telling colleagues about him, "Hey, check this guy out. His webinars will make your brain
bleed out of your ears!" Trust me, in my circle that's a HUGE compliment.
When I was chosen to attend my first Tech Field Day event, I was most excited because I would
finally get to meet Ivan in person. All my engineering friends were jealous and I was almost
apoplectic when the moment finally arrived, fearful I would do something foolish like confuse SMTP
and SNMP. This is when I discovered a really wonderful aspect to Ivan, if you're ever lucky enough
to interact with him personally (stalking doesn't count), you'll find him to be witty, friendly,
generous and gracious. He never makes you feel stupid for not understanding a protocol, the details
of an RFC or an IEEE standard.
He's the consummate educator and a giving mentor to almost anyone who asks. The more I know
him, the more I admire and respect his dedication to engineering. It truly is a vocation for him.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page v
I guess I need to say something about SDN now, so here goes. While it could be the idea that finally
revolutionizes networking, data centers and even security, I advise caution. Vendors will latch onto
this new buzzword like a pitbull and promote it like the industry's new secret sauce. With this book,
you'll be able to separate facts from hype and make some educated decisions regarding your own
infrastructure.
Michele Chubirka
Security architect, analyst, writer and podcaster
December 2013
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page vi
INTRODUCTION
OpenFlow and Software Defined Networks (SDN) entered mainstream awareness in March 2011
when several large cloud providers and Internet Service Providers formed Open Networking
Foundation.
More than three years later, the media still doesnt understand the basics of SDN, and many
networking engineers feel threatened by what they see as a fundamental shift in the way they do
their jobs.
In the meantime, I published over a hundred blog posts on ipSpace.net trying to debunk the myths,
explain how SDN and OpenFlow work, and what their advantages and limitations are. Most of the
posts were responses to external triggers false claims, vendor launches, or questions I received
from my readers.
This book contains a collection of the most relevant blog posts describing the concepts of SDN and
OpenFlow. I cleaned up the blog posts and corrected obvious errors and omissions, but also tried to
leave most of the content intact. The commentaries between the individual blog posts will help you
understand the timeline or the context in which a particular blog post was written.
The book covers these topics:
The debunking of the initial hype surrounding OpenFlow public launch and the most blatant
misconceptions (Chapter 1);
Overview of what SDN is, what it benefits might be, and deliberations whether or not it makes
sense (Chapter 2);
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page vii
Introduction to OpenFlow, from architectural basics to protocol details, and deployment and
forwarding models (Chapter 3);
OpenFlow implementation notes, describing the peculiarities of hardware and software
implementations of OpenFlow switches (Chapter 4);
OpenFlow scalability challenges, from control-plane complexity to packet punting and limitations
of flow table updates (Chapter 5);
OpenFlow use cases, from production deployment @ Google to interesting ready-to-use
architectures and musings on potential future uses (Chapter 6);
SDN beyond OpenFlow (Chapter 7), covering BGP-based SDN, NETCONF, I2RS, Ciscos OnePK
and Plexxis controller-based data center fabrics.
Youll find additional SDN- and OpenFlow-related information on ipSpace.net web site:
Start with the SDN, OpenFlow and NFV Resources page;
Numerous ipSpace.net webinars describe SDN, network programmability and automation, and
OpenFlow (some of them are freely available thanks to industry sponsors);
2-day SDN, NFV and SDDC workshop will help you figure out how to use SDN, network function
virtualization and SDDC technologies in your network;
Finally, Im always available for short online or on-site consulting engagements.
As always, please do feel free to send me any questions you might have the best way to reach me
is to use the contact form on my web site (www.ipSpace.net).
Happy reading!
Ivan Pepelnjak
July 2014
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
1 THE INITIAL HYPE
Academic researchers were working on OpenFlow concepts (distributed data plane with centralized
controller) for years, but in early 2011 a fundamental marketing shift happened: major cloud
providers (Google) and Internet Service Providers (Deutsche Telekom) created Open Networking
Foundation (ONF) to push forward commercial adoption of OpenFlow and Software Defined
Networking (SDN) or at least their definition of it.
Since then, every single vendor started offering SDN products. Almost none of them come even
close to the (narrow) vision promoted by the Open Networking Foundation (centralized control plane
with distributed data plane), NECs ProgrammableFlow being a notable exception.
Most vendors decided to SDN-wash their existing products, branding their existing APIs Open, and
claiming they have SDN-enabled products.
MORE INFORMATION
Youll find additional SDN- and OpenFlow-related information on ipSpace.net web site:
Start with the SDN, OpenFlow and NFV Resources page;
Read SDN- and OpenFlow-related blog posts and listen to the Software Gone Wild podcast;
Numerous ipSpace.net webinars describe SDN, network programmability and automation, and
OpenFlow (some of them are freely available thanks to industry sponsors);
2-day SDN, NFV and SDDC workshop will help you figure out how to use SDN, network function
virtualization and SDDC technologies in your network;
Finally, Im always available for short online or on-site consulting engagements.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 1-2
As usual, the industry media didnt help they enthusiastically jumped onto the OpenFlow/SDN
bandwagon and started propagating myths. More than two years later they still dont understand the
fundamentals of SDN, and tend to focus exclusively on how SDN is supposed to hurt Cisco (or not).
IN THIS CHAPTER:
OPEN NETWORKING FOUNDATION FABRIC CRAZINESS REACHES NEW HEIGHTS
OPENFLOW FAQ: WILL THE HYPE EVER STOP?
OPENFLOW IS LIKE IPV6
FOR THE RECORD: I AM NOT AGAINST OPENFLOW
NETWORK FIELD DAY FIRST IMPRESSIONS
I APOLOGIZE, BUT IM EXCITED
THE REALITY TWO YEARS LATER
CONTROL AND DATA PLANE SEPARATION THREE YEARS LATER
TWO AND A HALF YEARS AFTER OPENFLOW DEBUT, THE MEDIA REMAINS CLUELESS
WHERES THE REVOLUTIONARY NETWORKING INNOVATION?
FALLACIES OF GUI
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 1-3
In March 2011, industry media quickly picked up the buzz created by the Open Networking
Foundation (ONF) press releases and started exaggerating the already extravagant claims made by
ONF, prompting me to write the following blog post.
OPEN NETWORKING FOUNDATION FABRIC
CRAZINESS REACHES NEW HEIGHTS
Some of the biggest buyers of the networking gear have decided to squeeze some extra discount
out of the networking vendors and threatened them with open-source alternative, hoping to repeat
the Linux/Apache/MySQL/PHP saga that made it possible to build server farms out of low-cost
commodity gear with almost zero licensing costs. They formed the Open Networking Foundation,
found a convenient technology (OpenFlow) and launched another major entrant in the Buzzword
Bingo Software-Defined Networking (SDN).
Networking vendors, either trying to protect their margins by stalling the progress of this initiative,
or stampeding into another Wild West Gold Rush (hoping to unseat their bigger competitors with
low-cost standard-based alternatives) have joined the foundation in hordes; the list of initial
members reads like Whos Who in Networking.
Now, lets try to figure out what SDN might be all about. The ONF Mission Statement (on the first
page) says SDN allows owners and operators of networks to control and manage their networks to
best serve their needs. Are the founding members of ONF trying to tell us they have no control over
their networks and lack network management systems? It must be something else. How about this
one (from the same paragraph): OpenFlow seeks to increase network functionality while lowering
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 1-4
the cost associated with operating networks. Now were getting somewhere I told you it was all
about reducing costs (starting with the networking vendors margins).
(Some of) the industry media happily joined the craze, parroting meaningless phrases from various
press releases. Consider, for example, this article from IT World Canada.
SDN would give network operators the ability to virtualize network resources, being able to
dynamically improve latency or security on demand If you want to do it, you can do it today, using
dynamic routing protocols or QoS (latency), vShield/VSG (on-demand security) or a number of
virtualized networking appliances.
Also, protocols like RSVP to signal per-session bandwidth needs have been around for more than a
decade, but somehow never caught on. Must be the fault of those stupid networking vendors.
Sites like Facebook, Google or Yahoo would be able to tailor their networks so searches would be
blindingly fast I never realized the main search problem was network bandwidth. I always somehow
thought it was related to large datasets, CPU, database indices ... Anyhow, if the network bandwidth
is the bottleneck, why dont they upgrade to the next-generation Ethernet (10G/40G). Ah, yes, it
might be expensive. How about deploying Clos network architecture? Ouch, might be a nightmare to
configure and manage. How exactly will SDN solve this problem?
Stock exchanges could assure brokerage customers on the other side of the globe theyd get
financial data as fast as a dealer beside the exchange. Will SDN manage to flatten & shrink the
earth, will it change the speed of light, or will it use large-scale quantum entanglement?
It could be programmed to order certain routers to be powered down during off-peak power
periods. What stops you from doing that today?
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 1-5
Dont get me wrong OpenFlow might be a good idea and it will probably lead to interesting new
opportunities (assuming they can solve the scalability and resilience issues) ... and Im absolutely
looking forward to the podcast were recording later today (available on Packet Pushers web site).
However, there are plenty of open standards in the networking industry (including XML-based
network configuration and management) waiting to be used. There are also (existing, standard)
technologies that you can use to solve most of the problems these people are complaining about.
The problem is that these standards and technologies are not used by operating systems or
applications (when was the last time youve deployed a server running OSPF to have seamless
multihoming?)
The main problems were facing today arise primarily from non-scalable application architectures
and broken TCP/IP stack. In a world with scale-out applications you dont need fancy combinations
of routing, bridging and whatever-else; you just need fast L3 transport between endpoints. In an
Internet with decent session layer or a multipath transport layer (be it SCTP, Multipath TCP or
something else) you dont need load balancers, BGP sessions with end-customers to support
multihoming, or LISP. All these kludges were invented to support OS/App people firmly believing in
fallacies of distributed computing. How is SDN supposed to change that? Im anxiously waiting to
see an answer beyond marketing/positioning/negotiating bullshit bingo.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 1-6
Not surprisingly, the OpenFlow hype did not subside, and totally inaccurate articles started
appearing in industry press, prompting me to write yet another rant in April 2011.
OPENFLOW FAQ: WILL THE HYPE EVER STOP?
Network World has published another masterpiece last week: FAQ: What is OpenFlow and why is it
needed? Following the physics-changing promises made during the Open Network Foundation
launch, one would hope to get some straight facts; obviously things dont work that way. Lets walk
through some of the points. While most of them might not be too incorrect from an oversimplified
perspective, they do over-hype a potentially useful technology way out of proportions.
NW: OpenFlow is a programmable network protocol designed to manage and direct traffic among
routers and switches from various vendors. This one is just a tad misleading. OpenFlow is actually a
protocol that allows a controller to download forwarding tables into one or more switches. Whether
that manages or directs traffic depends on what controller is programmed to do.
NW: The technology consists of three parts: [...] and a proprietary OpenFlow protocol for the
controller to talk securely with switches. Please do decide what you think proprietary means. All
parts of the OpenFlow technology are defined in publicly available documents under BSD-like
license.
NW: OpenFlow is designed to provide consistency in traffic management and engineering by
making this control function independent of the hardware it's intended to control. How can a low-
level flow-table-control API provide what this statement claims it does? It all depends on the
controller implementation.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 1-7
NW: The programmability of the MPLS capabilities of a particular vendor's platform is specific to
that vendor. And the OpenFlow-related capabilities of individual switches will depend on specific
implementations by specific vendors. Likewise, the capabilities of an OpenFlow controller will be
specific to that vendor. What exactly is the fundamental change?
NW: MPLS is a Layer 3 technique while OpenFlow is a Layer 2 method Do I need to elaborate on
this gem? Lets just point out that OpenFlow works with MAC addresses, IP subnets, IP flow 5-
tuples, VLANs or MPLS labels. Whatever a switch can do, OpenFlow can control it.
But wait ... OpenFlow has no provision for IPv6 at all. Maybe Network World is so futuristic they
consider a technology without IPv6 support a layer-2 technology.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 1-8
In another blog post, I compared OpenFlow to IPv6 the evangelists of both technologies promised
way more than the technologies were ever capable of delivering.
OPENFLOW IS LIKE IPV6
Frequent eruptions of OpenFlow-related hype (a recent one caused by Brocade Technology Day
Summit; Im positive Interop will not lag behind) call for a continuous myth-busting efforts. Lets
start with a widely-quoted (and immediately glossed-over) fact from Professor Scott Shenker, a
founding board member of the ONF: [OpenFlow] doesn't let you do anything you couldn't do on a
network before.
To understand his statement, remember that OpenFlow is nothing more than a standardized version
of communication protocol between control and data plane. It does not define a radically new
architecture, it does not solve distributed or virtualized networking challenges and it does not create
new APIs that the applications could use. The only thing it provides is the exchange of TCAM (flow)
data between a controller and one or more switches.
Cold fusion-like claims are nothing new in the IT industry. More than a decade ago another group of
people tried to persuade us that changing the network layer address length from 32 bits to 128 bits
and writing it in hex instead of decimal solves global routing and multihoming and improves QoS,
security and mobility. After the reality distortion field collapsed, we were left with the same set of
problems exacerbated by the purist approach of the original IPv6 architects.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 1-9
Learn from the past bubble bursts. Whenever someone makes an extraordinary claim about
OpenFlow, remember the it cant do anything you couldnt do before fact and ask yourself:
Did we have a similar functionality in the past? If not, why not? Was there no need or were the
vendors too lazy to implement it (don't forget they usually follow the money)?
Did it work? If not, why not?
If it did - do we really need a new technology to replace a working solution?
Did it get used? If not, why not? What were the roadblocks? Why would OpenFlow remove them?
Repeat this exercise regularly and youll probably discover the new emperors clothes arent nearly
as shiny as some people would make you believe.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 1-10
The OpenFlow pundits quickly labeled me as an OpenFlow hater, but I was just my grumpy old self
;) Heres the blog post (from May 2011) that tried to set the record straight (not that such things
would ever work).
FOR THE RECORD: I AM NOT AGAINST OPENFLOW
... as some of its supporters seem to believe every now and then (I do get severe allergic reaction
when someone claims it will change the laws of physics or when Im faced with technical
inaccuracies not to mention knee-jerking financial experts). Even more, assuming it can cross the
adoption gap, it could fundamentally change the business models of networking vendors (maybe not
in the way youd like them to be changed). You can read more about my OpenFlow views in the
article I wrote for SearchNetworking.
On the more technological front, I still dont expect to see miracles. Most OpenFlow-related ideas
Ive heard about have been tried (and failed) before. I fail to see why things would be different just
because we use a different protocol to program the forwarding tables.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 1-11
In just a few months, everyone was talking about OpenFlow and SDN, and Stephen Foskett, the
mastermind behind GestaltIT, decided to organize the first ever OpenFlow symposium in September
2011.
The vendor and user presentations weve seen at that symposium, combined with the vendor
presentations weve attended during the Networking Tech Field Day 2 seemed very promising
everyone was talking about the right topics and tried to address real-life scalability concerns.
NETWORK FIELD DAY FIRST IMPRESSIONS
We finished a fantastic Network Field Day (second edition) yesterday. While it will take me a while
(and 20+ blog posts) to recover from the information blast I received during the last two days, here
are the first impressions:
Explosion of innovation and its not just OpenFlow and/or SDN. Last year weve seen some great
products and a few good ideas (earning me the grumpy old man thats hard to make smile fame),
this year almost every vendor had something that excited me.
If you were watching the video stream, you probably got sick and tired of my wow, thats cool
comments. I apologize, but thats how I felt.
Everyone gets the problem ... and some of the vendors were trying to tell us what the problem is in
an CIO-level pitch. Not a good idea. However, its refreshing to see that everyone identified the
same problem (large-scale data centers, VM mobility ...), that its the problem were all familiar
with, and that its actually getting solved.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 1-12
Most vendors have sensible answers. They are addressing different parts of the big problem, they
talk about different technologies, but the answers arent bad. For example, every time I spotted a
scalability issue, they were aware of it and/or had good answers (if not a solution).
Layer-2 is fading away (again). While every switching vendor will tell you how you can build large L2
domains with their fabric, nobody is actually pushing them anymore. And the only time layer-2 Data
Center Interconnect (DCI) appeared on a slide, there was a unicorn image next to it. Even more,
two vendors actually said they think long-distance VM mobility is not a good idea (youll have to
watch the videos to figure out who they were).
Were cutting through the hype. Even the OpenFlow symposium was hypeless. Its so nice being able
to spend three days with highly intelligent people who are excited about the next great thing
(whatever it is), while being perfectly realistic about its current state and its limitations.
Youll see lots of new things in the future. Even if youre working in an SMB environment, you might
get exposed to OpenFlow in the not-too-distant future (more about that in an upcoming post).
Get ready for a bumpy ride. Lots of exciting technologies are being developed. Some of them make
perfect sense, some others less so. Some of them might work, some might fade away (not because
they would be inherently bad, but because of bad execution). Now is the time to jump on those
bandwagons get involved (hint: you just might start with IPv6), build a test lab, kick the tires,
figure out whether the new technologies might be a good fit for your environment when they
become stable.
Disclosure: vendors mentioned in this post indirectly covered my travel expenses. Read the full
disclosure (or a more precise one by Tony Bourke).
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 1-13
Even more, the real-life approach of numerous vendors Ive seen during those two events made me
overly optimistic I thought we just might be able to get to real-life OpenFlow and SDN use cases
without the usual vendor jousting and get-rich-quick startup mentality. This is what I wrote in
October 2011:
I APOLOGIZE, BUT IM EXCITED
The last few days were exquisite fun: it was great meeting so many people focusing on a single
technology (OpenFlow) and concept (Software-Defined Networking, whatever that means) that just
might overcome some of the old obstacles (and introduce new ones). You should be at least a bit
curious what this is all about, and even if you dont see yourself ever using OpenFlow or any other
incarnation of SDN in your network, it never hurts to enhance your resume with another technology
(as long as its relevant; dont put CICS programmer at the top of it).
Watching the presentations from the OpenFlow symposium is a great starting point. I would start
with the ones from Igor Gashinsky (Yahoo!) and Ed Crabbe (Google) they succinctly explained the
problems theyre facing in their networks and how they feel OpenFlow could solve them. If youre an
IaaS cloud provider, this is the time to start thinking about potentials OpenFlow could bring to your
network, and if youre not talking to NEC, BigSwitch or Nicira, youre missing out. I would also talk
with Juniper (more about that later).
Next step: watch the vendor presentations from the OpenFlow symposium. Kyle Forster presented a
high-level overview of Big Switch architecture, Curt Beckmann from Brocade added a healthy dose
of reality check (highly appreciated), David Meyer (Cisco) presented an interesting perspective on
robustness and complexity (and several OpenFlow use cases), Don Clark from NEC talked about
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 1-14
their OpenFlow products (watch the video, PDF is not online) and finally David Ward from Juniper
presented the hybrid approach: use OpenFlow in combination (not as a replacement) with existing
technologies.
The afternoon technical Q&A panel just confirmed that numerous vendors well understand the
challenges associated with OpenFlow deployments outside of small lab setups, and that theyre
actively working on solving those problems and making OpenFlow a viable technology.
Two vendors expanded their coverage of OpenFlow during the Network Field Day: David Ward from
Juniper did a technical deep dive (dont skip the Junos automation part at the beginning of the
video, its interesting ... and you just might spot the VRF Smurf) and NEC even showed us a demo
of their OpenFlow-based switched network.
Luckily there are still some coolheaded people around (read Ethan Banks OpenFlow State of the
Union and Derick Winkworths More Open Flow Symposium Notes), but I cant help myself. The
grumpy old man from L3 ivory tower is excited (listen to PacketPushers OpenFlow/SDN podcast if
you dont believe me), and not just about OpenFlow. I still cant believe that I stumbled upon so
many interesting or cool technologies or solutions in the last few days. Could be that its just
vendors adapting to the blogging audience, or there actually might be something fundamentally new
coming to light like MPLS (then known as tag switching) was in the late 1990s.
Disclosure: vendors mentioned in this post indirectly covered my travel expenses. Read the full
disclosure (or a more precise one by Tony Bourke).
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 1-15
The hard reality of intervening two years has crushed all my high hopes. This is the reality of
OpenFlow and SDN as I see it in November 2013:
THE REALITY TWO YEARS LATER
Major vendors (with the exception of NEC) havent made any progress. Juniper still hasnt delivered
on its promises. Cisco still hasnt shipped an OpenFlow switch or an SDN controller (although theyve
announced both months ago). Brocade supposedly has OpenFlow on their high-end routers and
Arista supports OpenFlow on its old high-end switch (but not in GA EOS release).
Every major vendor is talking about SDN, but its mostly SDN-washing (aka CLI-in-API-disguise).
Cisco is talking about OnePK, and has shipping early adopter SDK kit, but it will take a while before
we see OnePK in GA code on a widespread platform.
Startups arent doing any better. Big Switch is treading water and trying to find a useful use case for
their controller. Nicira was acquired by VMware and is moving away from OpenFlow. Contrail was
acquired by Juniper and recently shipped its product (which has nothing to do with OpenFlow and
not much with SDN). LineRate Systems was acquired by F5 and disappeared.
We havent seen customer deployments either. Facebook is doing interesting things (but from what
Ive heard theyre not OpenFlow-based), Google has an OpenFlow/SDN deployment, but they could
have done the exact same thing with classical routers and PCEP, Microsofts SDN is based on BGP
(and works fine).
It seems like the reality hit OpenFlow and it was a very hard hit and according to Gartner we
havent reached the trough of disillusionment yet.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 1-16
In January 2014 I took another look at what the Open Networking Foundation founding members
managed to achieve between March 2011 (the beginning of OpenFlow/SDN hype) and early 2014.
The only one that made significant progress on the centralized control plane front was Google.
Since I wrote this blog post, Facebook launched their own switch operating system, which seems to
be working along the same lines as classical network operating systems (one device, one control
plane).
CONTROL AND DATA PLANE SEPARATION THREE
YEARS LATER
Almost three years ago the OpenFlow/SDN hype exploded and the Open Networking Foundation
started promoting the concept of physically separate control and data planes. Lets see how far its
founding members got in the meantime:
Google implemented their inter-DC WAN network with switches that use OpenFlow within a
switching fabric and BGP/IS-IS and something akin to PCEP between sites;
Facebook is working on the networking platform for their Open Compute Project. It seems
theyve got to switch hardware specs; I havent heard about software running on those switches
yet or maybe theyll go down the same path as Google (We got cheap switches, and we have
our own software. Goodbye and thank you!)
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 1-17
Yahoo! was talking about custom changes to standard networking protocols. Havent heard about
their progress since the first OpenFlow Symposium; the April 2012 presentation from Igor
Gashinsky still concluded with Wheres My Pony?
Deutsche Telekom is still using traditional routers and a great NFV platform.
Microsoft implemented SDN using BGP, using a central controller, but not a centralized control
plane.
I have no idea what Verizon is doing.
In the networking vendor world, NEC seems to be the only company with a mature commercial
product that matches the ONF definition of SDN. Cisco has just shipped the initial version of their
controller, as did HP, and those products seem pretty limited at the moment.
Wondering why I didnt include Big Switch Networks in the above list? My definition of shipping
includes publicly available product documentation, or (at the very minimum) something resembling
a data sheet with feature description, system requirements and maximum limits. I couldnt find
either on Big Switch web site.
On the other hand, the virtual networking world was always full of solutions with separate control
and data planes, starting with the venerable VMware Distributed vSwitch and Nexus 1000V, and
continuing with newer entrants, from Hyper-V extensible switch and VMware NSX to Juniper Contrail
and IBMs 5000V and DOVE. Some of these solutions were used years before the explosion of
OpenFlow/SDN hype (only we didnt know we should call them SDN).
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 1-18
In the meantime, the industry media still hasnt grasped the basics of SDN. Heres my response to a
particularly misleading article written in November 2013:
TWO AND A HALF YEARS AFTER OPENFLOW DEBUT,
THE MEDIA REMAINS CLUELESS
If you repeat something often enough, it becomes a fact (or an urban myth). SDN is no exception;
industry press loves to explain SDN like this:
[SDN] takes the high-end features built into routers and switches and puts them into
software that can run on cheaper hardware. Corporations still need to buy routers and
switches, but they can buy fewer of them and cheaper ones.
That nice soundbite contains at least one stupidity per sentence:
SDN cannot move hardware features into software. If a device relies on hardware forwarding,
you cannot move the same feature into software without significantly impacting the forwarding
performance.
SDN software runs on cheaper hardware. Ignoring the intricacies of custom ASICs and
merchant silicon (and the fact that Cisco produces more custom ASICs than all merchant silicon
vendors combined), complexity and economies of scale dictate the hardware costs. Its pretty hard
to make cheaper hardware with the same performance and feature set.
However, all networking vendors bundle the software with the hardware devices and expense R&D
costs (instead of including them in COGS) to boost their perceived margins.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 1-19
Does the above paragraph sound like Latin to you? Dont worry just keep in mind that software
usually costs about as much (or more) as the hardware it runs on, but you dont see that.
Corporations can buy fewer routers and switches. It cant get any better than this. If you need
100 10GE ports, you need 100 10GE ports. If you need two devices for two WAN uplinks (for
redundancy), you need two devices. SDN wont change the port count, redundancy requirements, or
laws of physics.
Corporations can buy cheaper [routers and switches]. Guess what you still need the
software to run them, and until we see price tags of SDN controllers, and do a TCO calculation,
claims like this one remain wishful thinking (you did notice Im extremely diplomatic today, didnt
you?).
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 1-20
Finally, numerous marketers and SDN/OpenFlow pundits keep repeating how theyll save the
(networking) world and bring true nirvana to the network operations with their flashy new gadgets.
Nothing can be further from the truth because we cannot get rid of the legacy permeating the whole
TCP/IP stack, as I explained in this post written in July 2013:
WHERES THE REVOLUTIONARY NETWORKING
INNOVATION?
In his recent blog post Joe Onisick wrote What network virtualization doesnt provide, in any form,
is a change to the model we use to deploy networks and support applications. [...] All of the same
broken or misused methodologies are carried forward. [...] Faithful replication of todays networking
challenges as virtual machines with encapsulation tunnels doesnt move the bar for deploying
applications.
Much as I agree with him, we cant change much on planet Earth due to the fact that VMs use
Ethernet NICs (so we need some form of VLANs to cater to infinite creativity of some people), IP
addresses (so we need L3 forwarding), broken TCP stack (requiring load balancers to fix it), and
obviously cant be relied upon to be sufficiently protected (so we need external firewalls).
Furthermore, unless we manage to stop shifting the problems around, the networking as a whole
wont get simpler.
What overlay network virtualization does bring us is a decoupling that makes physical infrastructure
less complex so it can focus on packet forwarding instead of zillions of customer-specific features
preferably baked in custom ASICs. Obviously thats not a good thing for everyone out there.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 1-21
The final bit of hype I want to dispel is the misleading focus on CLI that we use to configure
networking devices. CLI is not the problem, and GUI will not save the world.
FALLACIES OF GUI
I love Greg Ferros characterization of CLI:
We need to realise that the CLI is a power tools for specialist tradespeople and not a
knife and fork for everyday use.
However, you do know that most devices GUI offers nothing more than what CLI does, dont you?
Wheres the catch?
For whatever reason, people find colorful screens full of clickable items less intimidating than a
blinking cursor on black background. Makes sense after all, you can see all the options you have;
you can try pulling down things to explore possible values, and commit the changes once you think
you enabled the right set of options. Does that make a product easier to use? Probably. Will it result
in better-performing product? Hardly.
Have you ever tried to configure OSPF through GUI? How about trying to configure usernames and
passwords for individual wireless users? In both cases youre left with the same options youd have
in CLI (because most vendors implement GUI as eye candy in front of the CLI or API). If you know
how to configure OSPF or RADIUS server, GUI helps you break the language barrier (example:
moving from Cisco IOS to Junos), if you dont know what OSPF is, GUI still wont save the day ... or
it might, if you try clicking all the possible options until you get one that seems to work (expect a
few meltdowns on the way if youre practicing your clicking skills on a live network).
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 1-22
What the casual network admins need are GUI wizards a tool that helps you achieve a goal while
keeping your involvement to a minimum. For example: I need IP routing between these three
boxes. Go do it! should translate into Configure OSPF in area 0 on all transit interfaces. When you
see a GUI offering this level of abstraction please let me know. In the meantime, Im positive that
the engineers who have to get a job done quickly prefer using CLI over clickety-click GUI (and Im
not the only one), regardless of whether they have to configure a network device, Linux server,
Apache, MySQL, MongoDB or a zillion other products. Why do you think Microsoft invested so heavily
in PowerShell
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
2 SOFTWARE DEFINED NETWORKING 101
Open Networking Foundation (ONF) launched in March 2011 quickly defined Software Defined
Networking (SDN) as architecture with centralized control plane that controls multiple physically
distinct devices.
That definition definitely suits one of the ONF founding members (Google), but is it relevant to the
networking community at large? Or does it make more sense to focus on network programmability,
or using existing protocols (BGP) in novel ways?
This chapter contains my introductory posts on the SDN-related topics, musings on what makes
sense, and a few thoughts on career changes we might experience in the upcoming years. Youll find
more details in subsequent chapters, including an overview of OpenFlow, in-depth analysis of
OpenFlow-based architectures, some real-life OpenFlow and SDN deployments, and alternate
approaches to SDN.
MORE INFORMATION
Youll find additional SDN- and OpenFlow-related information on ipSpace.net web site:
Start with the SDN, OpenFlow and NFV Resources page;
Read SDN- and OpenFlow-related blog posts and listen to the Software Gone Wild podcast;
Numerous ipSpace.net webinars describe SDN, network programmability and automation, and
OpenFlow (some of them are freely available thanks to industry sponsors);
2-day SDN, NFV and SDDC workshop will help you figure out how to use SDN, network function
virtualization and SDDC technologies in your network;
Finally, Im always available for short online or on-site consulting engagements.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 2-2
IN THIS CHAPTER:
WHAT EXACTLY IS SDN (AND DOES IT MAKE SENSE)?
BENEFITS OF SDN
DOES CENTRALIZED CONTROL PLANE MAKE SENSE?
HOW DID SOFTWARE DEFINED NETWORKING START?
WE HAD SDN IN 1993 AND DIDNT KNOW IT
STILL WAITING FOR THE STUPID NETWORK
IS CLI IN MY WAY OR IS IT JUST A SYMPTOM OF A BIGGER PROBLEM?
OPENFLOW AND SDN DO YOU WANT TO BUILD YOUR OWN RACING CAR?
SDN, WINDOWS AND FRUITY ALTERNATIVES
SDN, CAREER CHOICES AND MAGIC GRAPHS
RESPONSE: SDNS CASUALTIES
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 2-3
The very strict definition of SDN as understood by Open Networking Foundation promotes an
architecture with strict separation between a controller and totally dumb devices that cannot do
more than forward packets based on forwarding rules downloaded from the controller. Does that
definition make sense? This is what I wrote in January 2014:
WHAT EXACTLY IS SDN (AND DOES IT MAKE SENSE)?
When Open Networking Foundation claimed ownership of Software-Defined Networking, they defined
it as separation of control and data plane:
[SDN is] The physical separation of the network control plane from the forwarding plane,
and where a control plane controls several devices.
Does this definition make sense or is it too limiting? Is there more to SDN? Would a broader scope
make more sense?
A BIT OF A HISTORY
Its worth looking at the founding members of ONF and their interests: most of them are large cloud
providers looking for cheapest possible hardware, preferably using a standard API so it can be
sourced from multiple suppliers, driving the prices even lower. Most of them are big enough to write
their own control plane software (and Google already did).
A separation of control plane (running their own software) and data plane (implemented in a low-
cost white-label switches) was exactly what they wanted to see, and the Stanford team working on
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 2-4
OpenFlow provided the architectural framework they could use. No wonder ONF pushes this
particular definition of SDN.
MEANWHILE DEEP BELOW THE CLOUDY HEIGHTS
I have yet to meet a customer (academics might be an exception) that would consider writing their
own control-plane software; most of my customers arent anywhere close to writing an SDN
application on top of a controller framework (Open Daylight, Cisco XNC or HP VAN SDN controller).
Buying a shrink-wrapped application bundled with commercial support might be a different
story but then nobody really cares whether such a solution uses OpenFlow or RFC 2549;
the protocols and encapsulation mechanisms used within a controller-based network
solution are often proprietary and thus impossible to troubleshoot anyway.
On the other hand, I keep hearing about common themes:
The need for faster, more standardized, and automated provisioning;
The need for programmable network elements and vendor-neutral programming mechanisms
(Im looking at you, netmod working group);
Centralized policies and decision making based on end-to-end visibility;
Easier integration of network elements with orchestration and provisioning systems.
Will physical separation of control and forward plane solve any of these? It might, but there are
numerous tools out there that can do the same without overhauling everything weve been doing in
the last 30 years.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 2-5
We dont need the physical separation of control plane to solve our problems (although the ability to
control individual forwarding entries does help) and it will probably take a decade before we
glimpse the promised savings of white-label switches and open-source software (even Greg Ferro
stopped believing that).
NOW WHAT?
Does it make sense to accept the definition of SDN that makes sense to ONF founding members but
not to your environment? Shall we strive for a different definition of SDN or just move on, declare it
as meaningless as the clouds, and focus on solving our problems? Would it be better to talk about
NetOps?
Maybe we should stop talking and start doing there are plenty of things you can do within existing
networks using existing protocols.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 2-6
Every new networking technology is supposed to solve most of our headaches. SDN is no exception.
The reality might be a bit different.
BENEFITS OF SDN
Paul Stewart wrote a fantastic blog post in May 2014 listing the potential business benefits of SDN
(as promoted by SDN evangelists and SDN-washing vendors).
Heres his list:
Abstracted Control Plane for a Central Point of Management
Granular Control of Flows (as required/desired)
Network Function Virtualization and Service Chaining
Decreased dependence on devices like load balancers
Facilitation of system orchestration
Easier troubleshooting/visibility
Platform for chargeback/showback
Decreased complexity and cost
Increased ability to utilize hardware and interconnections
DevOps friendly architecture
I have just one problem with this list Ive seen a similar list of benefits of IPv6:
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 2-7
Figure 2-1: IPv6 myths
Unfortunately, the reality of IT in general and IPv6 in particular is a bit different. The overly hyped
IPv6 benefits remain myths and legends; all we got were longer addresses, incompatible protocols
(OSPFv3 anyone), and half-thought-out implementations (example: DNS autoconfiguration) ridden
with religious wars (try to ask why dont we have first-hop router in DHCPv6 on any IPv6 mailing
list ;).
For more information, watch the fantastically cynical presentation Enno Rey had @ Troopers 2014
IPv6 Security summit, or my IPv6 resources.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 2-8
With Open Networking Foundation adamantly promoting their definition of SDN, and based on
experiences with previous (now mostly extinct) centralized architectures, one has to ask a simple
question: does it make sense? Heres what I thought in May 2014:
DOES CENTRALIZED CONTROL PLANE MAKE SENSE?
A friend of mine sent me a challenging question:
You've stated a couple of times that you don't favor the OpenFlow version of SDN due to
a variety of problems like scaling and latency. What model/mechanism do you like?
Hybrid? Something else?
Before answering the question, lets step back and ask another one: Does centralized control plane,
as evangelized by ONF, make sense?
A BIT OF HISTORY
As always, lets start with one of the greatest teachers: history. Weve had centralized architectures
for decades, from SNA to various WAN technologies (SDH/SONET, Frame Relay and ATM). They all
share a common problem: when the network partitions, the nodes cut off from the central
intelligence stop functioning (in SNA case) or remain in a frozen state (WAN technologies).
One might be tempted to conclude that the ONF version of SDN wont fare any better than the
switched WAN technologies. Reality is far worse:
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 2-9
WAN technologies had little control-plane interaction with the outside world (example: Frame
Relay LMI), and those interactions were run by the local devices, not from the centralized control
plane;
WAN devices (SONET/SDH multiplexers, or ATM and Frame Relay switches) had local OAM
functionality that allowed them to detect link or node failures and reroute around them using
preconfigured backup paths. One could argue that those devices had local control plane,
although it was never as independent as control planes used in todays routers.
Interestingly, MPLS-TP wants to reinvent the glorious past and re-introduce centralized path
management, yet again proving RFC 1925 section 2.11.
The last architecture (that I remember) that used truly centralized control plane was SNA, and if
youre old enough you know how well that ended.
WOULD CENTRAL CONTROL PLANE MAKE SENSE IN LIMITED
DEPLOYMENTS?
Central control plane is obviously a single point of failure, and network partitioning is a nightmare if
you have a central point of control. Large-scale deployments of ONF variant of SDN are thus out of
question. But does it make sense to deploy centralized control plane in smaller independent islands
(campus networks, data center availability zones)?
Interestingly, numerous data center architectures already use centralized control plane, so we can
analyze how well they perform:
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 2-10
Juniper XRE can control up to four EX8200 switches, or a total of 512 10GE ports;
Nexus 7700 can control 64 fabric extenders with 3072 ports, plus a few hundred directly
attached 10GE ports;
HP IRF can bind together two 12916 switches for a total of 1536 10GE ports;
QFabric Network Node Group could control eight nodes, for a total of 384 10GE ports.
NEC ProgrammableFlow seems to be an outlier they can control up to 200 switches, for a total of
over 9000 GE (not 10GE) ports but they dont run any control-plane protocol (apart from ARP and
dynamic MAC learning) with the outside world. No STP, LACP, LLDP, BFD or routing protocols.
One could argue that we could get an order of magnitude beyond those numbers if only we were
using proper control plane hardware (Xeon CPUs, for example). I dont buy that argument till I
actually see a production deployment, and do keep in mind that NEC ProgrammableFlow Controller
uses decent Intel-based hardware. Real-time distributed systems with fast feedback loops are way
more complex than most people looking from the outside realize (see also RFC 1925, section 2.4).
DOES CENTRAL CONTROL PLANE MAKE SENSE?
It does in certain smaller-scale environments (see above) as long as you can guarantee redundant
connectivity between then controller and controlled devices, or dont care what happens after link
loss (see also wireless access points). Does it make sense to generate a huge hoopla while
reinventing this particular wheel? I would spend my energy doing something else.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 2-11
I absolutely understand why NEC went down this path they did something extraordinary
to differentiate themselves in a very crowded market. I also understand why Google decided
to use this approach, and why they evangelize it as much as they do. Im just saying that it
doesnt make that much sense for the rest of us.
Finally, do keep in mind that the whole world of IT is moving toward scale-out architectures. Netflix
& Co are already there, and the enterprise world is grudgingly doing the first steps. In the
meantime, OpenFlow evangelists talk about the immeasurable revolutionary merits of centralized
scale-up architecture. They must be living on a different planet.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 2-12
Just in case youre wondering how the OpenFlow/SDN movement started, heres a bit of pre-2011
history.
HOW DID SOFTWARE DEFINED NETWORKING START?
Software-Defined Networking is clearly a tautological term after all, software defined networking
device behavior ever since we stopped using Token Ring MAUs and unmanaged hubs. Open
Networking Foundation claims it owns the definition of the term (which makes approximately as
much sense as someone claiming they own the definition of red-colored clouds), but I was always
wondering who coined the term in the first place.
I finally found the answer in a fantastic overview of technologies and ideas that led to OpenFlow and
SDN published in December 2013 issue of acmqueue. According to that article, SDN first appeared in
an article published by MIT Technology Review that explains how Nick McKeown and his team at
Stanford use OpenFlow:
Frustrated by this inability to fiddle with Internet routing in the real world, Stanford
computer scientist Nick McKeown and colleagues developed a standard called OpenFlow
that essentially opens up the Internet to researchers, allowing them to define data flows
using software--a sort of "software-defined networking."
You did notice the a sort of classification and quotes around SDN, didnt you? Its pretty obvious
how the article uses software-defined networking to illustrate the point but once marketing took
over all hope for reasonable discussion was lost, and SDN became even more meaningless as cloud.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 2-13
Assuming we forget the ONF-promoted definition of SDN and define SDN as network programmed
from a central controller, its obvious we had SDN for at least 20 years.
WE HAD SDN IN 1993 AND DIDNT KNOW IT
I had three SDN 101 presentations during my 2013 visit to South Africa and had tried really hard to
overcome my grumpy skeptic self and find the essence of SDN while preparing for them. As Ive
been thinking about controllers, central visibility and network device programmability, it struck me:
we already had SDN in 1993.
In 1993 we were (among other things) an Internet Service Provider offering dial-up and leased line
Internet access. Being somewhat lazy, we hated typing the same commands in every time we had
to provision a new user (in pre-TACACS+ days we had to use local authentication to have
autocommand capability for dial-up users) and developed a solution that automatically changed
the router configurations after we added a new user. Heres a high-level diagram of what we did:
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 2-14
Figure 2-2: Simple router provisioning system built in 1993
HTML user interface (written in Perl) gave the operators easy access to user database (probably
implemented as a text file we were true believers in NoSQL movement in those days), and a back-
end Perl script generated router configuration commands from the user definitions and downloaded
them (probably through rcp the details are a bit sketchy) to the dial-up access servers.
Next revision of the software included support for leased line users the script generated interface
configurations and static routes for our core router (it was actually an MGS, but I found no good
MGS images on the Internet) or one of the access server (for users using asynchronous modems).
How is that different from all the shiny new stuff vendors are excitedly talking about? Beats me, I
cant figure it out ;) and as I said before, you dont always need new protocols to solve old
problems.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 2-15
While were happily arguing the merits of reinvented architectures, we keep forgetting that the
basics of sound network architecture were known for over a decade and we still havent made any
progress getting closer to them.
STILL WAITING FOR THE STUPID NETWORK
More than 15 years ago the cover story of ACM netWorker magazine discussed the dawn of the
stupid network an architecture with smart edge nodes and simple packet forwarding code.
Obviously we learned nothing in all those years were still having the same discussions.
Here are a few juicy quotes from that article (taken completely out of context solely for your
enjoyment).
The telcos seemed to "fall asleep at the switch" at the core of their network.
"Keep it simple, stupid," or KISS, is an engineering virtue. The Intelligent Network,
however, is anything but simple; it is a marketing concept for scarce, complicated, high-
priced services.
The Intelligent Network impedes innovation. Existing features are integrally spaghetti-
coded into the guts of the network, and new features must intertwine with the old.
Infrastructure improvements are rapidly making the telcos' Intelligent Network a
distinctly second-rate choice. The bottom line, though, is not the infrastructure; it is the
innovation that the Stupid Network unleashes.
The whole article is well worth reading, more so considering its over 15 years old and still spot-on.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 2-16
Some SDN proponents claim that the way we configure networking devices (using CLI) is the biggest
networking problem were facing today. They also conveniently forget that every scalable IT solution
uses automation, text files and CLI because they work, and allow experienced operators to work
faster.
IS CLI IN MY WAY OR IS IT JUST A SYMPTOM OF A
BIGGER PROBLEM?
My good friend Ethan published a blog post in February 2014 rightfully complaining how various
vendor CLIs hamper our productivity. Hes absolutely correct from the productivity standpoint, and I
agree with his conclusions (we need a layer of abstraction), but theres more behind the scenes.
Were all sick of CLI. I dont think anyone would disagree. However, CLI is not our biggest
problem. We happen to be exposed to the CLI on a daily basis due to lack of automation tools and
lack of abstraction layer; occasional fights with the usual brown substance flowing down the
application stack dont help either.
The CLI problem is mostly hype. The we need to replace CLI with (insert-your-favorite-gizmo)
hype was generated by SDN startups (one in particular) that want to sell their disruptive way of
doing things to the venture capitalists. BTW, the best way to configure their tools is through CLI.
CLI is still the most effective way of doing things ask any really proficient sysadmin, web
server admin or database admin how they manage their environment. Its not through point-and-
click GUI, its through automation tools coupled with simple CLI commands (because automation
tools dont work that well when they have to simulate mouse clicks).
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 2-17
CLI generates vendor lock-in. Another pile of startup hype in this case coming from startups
that want to replace the network device lock-in with controller lock-in (heres a similar story).
WERE NOT UNIQUE
Startups and pundits would like to persuade you how broken traditional networking is, but every
other field in IT has to deal with the same problems just try to manage Windows server with Linux
commands, or create tables on Microsoft SQL server with MySQL or Oracle syntax even Linux
distributions dont have the same command set.
The true difference between other IT fields and networking is that the other people did something to
solve their problems while we keep complaining. Networking is no worse than any other IT
discipline; we just have to start moving forward, create community tools, and vote with our wallets.
Whenever you have a choice between two comparable products from different vendors, buy the one
that offers greater flexibility and programmability. Dont know what to look for? Talk with your
server- and virtualization buddies (I hope youre on speaking term with them, or its high time you
buy them a beer or two). If they happen to use Puppet or Chef to manage servers, you might try to
use the same tools to manage your routers and switches. Your favorite boxes dont support the tools
used by the rest of your IT? Maybe its time to change the vendor.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 2-18
Its reasonably easy to add automation and orchestration on top of existing network implementation.
Throwing away decades of field experience and replacing existing solutions with an OpenFlow-based
controller is a totally different story as I explained in May 2013:
OPENFLOW AND SDN DO YOU WANT TO BUILD
YOUR OWN RACING CAR?
The OpenFlow zealots are quick to point out the beauties of the centralized control plane, and the
huge savings you can expect from using commodity hardware and open-source software. What they
usually forget to tell you is that you also have to reinvent all the wheels the networking industry has
invented in the last 30 years.
Imagine you want to build your own F1 racing car... but the only component you got is a super-
duper racing engine from Mercedes Benz. You're left with the "easy" task of designing the car body,
suspension, gears, wheels, brakes and a few other choice bits and pieces. You can definitely do all
that if you're Google or McLaren team, but not if you're a Sunday hobbyist mechanic. No wonder
some open-source OpenFlow controllers look like Red Bull Flugtag contestants.
Does that mean we should ignore OpenFlow? Absolutely not, but unless you want to become really
fluent in real-time event-driven programming (which might look great on your resume), you should
join me watching from the sidelines until there's a solid controller (maybe we'll get it with Daylight,
Floodlight definitely doesn't fit the bill) and some application architecture blueprints.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 2-19
Till then, it might make sense to focus on more down-to-earth technologies; after all, you don't
exactly need OpenFlow and a central controller to solve real-life problems, like Tail-f clearly
demonstrated with their NCS software.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 2-20
Openness (for whatever value of Open) is another perceived benefit of SDN. In reality, youre
trading hardware vendor lock-in for controller vendor lock-in.
SDN, WINDOWS AND FRUITY ALTERNATIVES
Brad Hedlund made a pretty valid comment to my NEC Launched a Virtual OpenFlow Switch blog
post: On the other hand, it's NEC end-to-end or no dice, implicating the ultimate vendor lock-in.
Of course hes right and while, as Bob Plankers explains, you can never escape some lock-in (part 1,
response from Greg Ferro, part 2 all definitely worth reading), you do have to ask yourself am I
looking for Windows or Mac?
There are all sorts of arguments one hears from Mac fanboys (heres a networking related one) but
regardless of what you think of Mac and OSX, theres the undisputable truth: compared to reloadful
experience we get on most Windows-based boxes, Macs and OSX are rock solid; I have to reboot
my Macbook every other blue moon. Even Windows is stable when running on a Macbook (apart
from upgrade-induced reboots).
Before you start praising Steve Jobs and blaming Bill Gates and Microsoft at large, consider a simple
fact: OSX runs on a tightly controller hardware platform built with stability and reliability in mind.
Windows has to run on every possible underperforming concoction a hardware vendor throws at you
(example: my high-end laptop cannot record system audio because the 6-letter hardware vendor
wanted to save $0.02 on the sound chipset and chose the cheapest possible one), and has to deal
with all sort of crap third-party device drivers loaded straight into the operating system kernel.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 2-21
Now, what do you want to have in your mission-critical SDN/OpenFlow data center networking
infrastructure: a Mac-like tightly controlled and vendor-tested mix of equipment and associated
controller, or a Windows-like hodgepodge of boxes from numerous vendors, controlled by third-
party software that might have never encountered the exact mix of the equipment you have.
If youre young and brazen (like I was two decades ago), go ahead and be your own system
integrator. If youre too old and covered with vendor-inflicted scars, you might prefer a tested end-
to-end solution regardless of what Gartner says in vendor-sponsored reports (and even solutions
that vendor X claims were tested dont always work). Just dont forget to consider the cost of
downtime in your total-cost-of-ownership calculations.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 2-22
SDN controllers will replace networking engineers, at least if you believe what the SDN or
virtualization vendors are telling you. I dont think we have to worry about that happening in
foreseeable future (and nothing changed since I wrote the following blog post in late 2012).
SDN, CAREER CHOICES AND MAGIC GRAPHS
The current explosion of SDN hype (further fueled by recent VMworld announcement of Software-
Defined Data Centers) made some networking engineers understandably nervous. This is the
question I got from one of them:
I have 8 plus years in Cisco, have recently passed my CCIE RS theory, and was looking
forward to complete the lab test when this SDN thing hit me hard. Do you suggest
completing the CCIE lab looking at this new future of Networking?
Short answer: the sky is not falling, CCIE still makes sense, and IT will still need networking people.
However, as I recently collected a few magic graphs for a short keynote speech, let me reuse them
to illustrate this particular challenge were all facing. Starting with the obvious, heres the legendary
Diffusion of Innovations: every idea is first adopted by a few early adopters, followed by early and
late majority.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 2-23
Figure 2-3: Diffusion of ideas (source: Wikipedia)
Networking in general is clearly in the late majority/laggards phase. Whats important for our
discussion is the destruction of value-add through the diffusion process. Oh my, I sound like a
freshly-baked MBA whiz-kid, lets reword it: as a technology gets adopted, more people understand
it, the job market competition increases, and thus its harder to get a well-paying job in that
particular technology area. Supporting Windows desktops might be a good example.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 2-24
As a successful technology matures, it moves through the four parts of another magic matrix (this
one from Boston Consulting Group).
Figure 2-4: Boston Consulting Group matrix
Initially every new idea is a great unknown, with only a few people brave enough to invest time in it
(CCIE R&S before Cisco made it mandatory for Silver/Gold partner status). After a while, the
successful ideas explode into stars with huge opportunities and fat margins (example: CCIE R&S a
decade ago, Nicira-style SDN today at least for Niciras founders), degenerates into a cash cow as
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 2-25
the market slowly gets saturated (CCIE R&S is probably at this stage by now) and finally (when
everyone starts doing it) becomes an old dog not worth bothering with.
Does it make sense to invest into something thats probably in a cash cow stage? The theory says
as much as needed to keep it alive, but dont forget that CCIE R&S will likely remain very relevant
a long time:
The protocol stacks were using havent changed in the last three decades (apart from extending
the address field from 32 to 128 bits), and although people are working on proposals like MP-
TCP, those proposals are still in experimental stage;
Regardless of all the SDN hoopla, neither OpenFlow nor other SDN technologies address the real
problems were facing today: lack of session layer in TCP and the use of IP addresses in
application layer. They just give you different tools to implement todays kludges.
Cisco is doing constant refreshes of its CCIE programs to keep them in the early adopters or
early majority technology space, so the CCIE certification is not getting commoditized.
If you approach the networking certifications the right way, youll learn a lot about the principles
and fundamentals, and youll need that knowledge regardless of the daily hype.
Now that Ive mentioned experimental technologies dont forget that not all of them get adopted
(even by early adopters). Geoffrey Moore made millions writing a book that pointed out that obvious
fact. Of course he was smart enough to invent a great-looking wrapper he called it Crossing the
Chasm.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 2-26
Figure 2-5: The chasm before the mainstream market adoption (source: Crossing the Chasm & Inside the Tornado)
The crossing the chasm dilemma is best illustrated with Gartner Hype Cycles. After all the initial
hype (that weve seen with OpenFlow and SDN) resulting in peak of inflated expectations, theres
the ubiquitous through of disillusionment. Some technologies die in that quagmire; in other more
successful cases we eventually figure out how to use them (slope of enlightenment).
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 2-27
Figure 2-6: Gartner hype cycle (source: Wikipedia)
We still dont know how well SDN will be doing crossing the chasm (according to the latest Gartners
charts, OpenFlow still hasnt reached the hype peak - I dread what's still lying ahead of us); weve
seen only a few commercial products and none of them has anything close to widespread adoption
(not to mention the reality of three IT geographies).
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 2-28
Anyhow, since youve decided you want to work in networking, one thing is certain: technology will
change (whatever the change will be), and it will happen with or without you. At every point in your
career you have to invest some of your time into learning something new. Some of those new things
will be duds; others might turn into stars. See also Private Clouds Will Change IT Jobs, Not Eliminate
Them by Mike Fratto.
Finally, dont ask me for what will the next big thing be advice. Browse through the six years of
my blog posts. You might notice a clear shift in focus; its there for a reason.
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 2-29
Finally, heres a response to an industry press gem I wrote in 2013:
RESPONSE: SDNS CASUALTIES
An individual focused more on sensationalism than content deemed it appropriate to publish an
article declaring networking engineers an endangered species on an industry press web site that I
considered somewhat reliable in the past.
The resulting flurry of expected blog posts included an interesting one from Steven Iveson in which
he made a good point: its easy for the cream-of-the-crop not to be concerned, but what about
others lower down the pile. As always, it makes sense to do a bit of reality check.
While everyone talks about SDN, the products are scarce, and it will take years before theyll
appear in a typical enterprise network. Apart from NECs Programmable Flow and overlay
networks, most other SDN-washed things Ive seen are still point products.
Overlay virtual networks seem to be the killer app of the moment. They are extremely useful and
versatile ... if youre not bound to VLANs by physical appliances. Well have to wait for at least
another refresh cycle before we get rid of them.
Data center networking is hot and sexy, but its only a part of what networking is. I havent seen
a commercial SDN app for enterprise WAN, campus or wireless (Im positive Im wrong write a
comment to correct me), because thats not where the VCs are looking at the moment.
Also, consider that the my job will be lost to technology sentiments started approximately 200 years
ago and yet the population has increased by almost an order of magnitude in the meantime, there
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
Copyright ipSpace.net 2014 Page 2-30
are obviously way more jobs now (in absolute terms) than there were in those days, and nobody in
his right mind wants to do the menial chores that the technology took over.
Obviously you should be worried if youre a VLAN provisioning technician. However, with everyone
writing about SDN you know whats coming down the pipe, and you have a few years to adapt,
expand the scope of your knowledge, and figure out where it makes sense to move (and dont forget
to focus on where you can add value, not what job openings you see today). If you dont do any of
the above, dont blame SDN when the VLANs (finally) join the dinosaurs and you have nothing left to
configure.
Finally, Im positive there will be places using VLANs 20 years from now. After all, AS/400s and
APPN are still kicking and people are still fixing COBOL apps (that IBM just made sexier with XML
and Java support).
This material is copyrighted and licensed for the sole use by Srdjan Milenkovic ([email protected] [109.121.110.253]). More information at http://www.ipSpace.net/Webinars
3 OPENFLOW BASICS
Based on exorbitant claims made by the industry press you might have concluded there must be
some revolutionary concepts in the OpenFlow technology. Nothing could be further from the truth
OpenFlow is a very simple technology that allows a controller to program forwarding entries in a
networking device.
Did you ever encounter Catalyst 5000 with Route Switch Module (RSM), or a combination of Catalyst
5000 and an external router, using Multilayer Switching (MLS)? Those products used architecture
identical to OpenFlow almost 20 years ago, the only difference being the relative openness of
OpenFlow protocol.
This chapter will answer a number of basic OpenFlow questions, including:
MORE INFORMATION
Youll find additional SDN- and OpenFlow-related information on ipSpace.net web site:
Start with the SDN, OpenFlow and NFV Resources page;
Read SDN- and OpenFlow-related blog posts and listen to the Software Gone Wild podcast;
Numerous ipSpace.net webinars describe SDN, network programmability and automation, and
OpenFlow (some of them are freely available thanks to industry sponsors);
2-day