Linux and VXLAN 2013

8/10/2019 Linux and VXLAN 2013

1/30

2009 IBM Corporation

Software Defined Networking using VXLAN

!o"as #i$!ter

!o"as #i$!ter% IBM #esear$! and De&e'op"ent( Linu) e$!no'og* CenterLinu)Con +din,urg! 2-./$t.20-


2/30

2 Software Defined Networking using VXLAN( !o"as #i$!ter 1t"ri$!tde3i,"3$o"4( Linu)Con 20-

Agenda

V)'an% I+5 Draft% VXLAN 5eatures in Linu) 6erne' 37 1D/V+ +)tension4

8rin$ip'e of /peration

% VM Creation( Migration( #e"o&a'

Ad&an$ed sage

% Mu'ti$ast( Broad$ast( VM Dete$tion

Manage"ent oo's

#e'ated and 5uture :ork

IBM 8resentation e"p'ate 5u'' Version


3/30

Software Defined Networking using VXLAN( !o"as #i$!ter 1t"ri$!tde3i,"3$o"4( Linu)Con 20-

Virtua'i;ation in Data Center

Data $enters !ost "u'tip'e $usto"ers

Custo"ers re Logi$a' network on top of e)isting network infrastru$ture

argets% Centra' "anage"ent and $ontro'

% #e'ia,i'it*% Co&er 'ong distan$e ,etween data $enters% Define optiona' po'i$ies 1$o"pression( en$r*ption( 3334


Intranet?InternetVirtua' Bridge

VM VM VM

@ost A

NIC

Swit$!

@ost B

NIC

VM VM

App App


4/30


VXLAN 1I+5 Draft4


Virtua' eXtensi,'e Lo$a' Area Network

% +n$apsu'ates data pa$kets% Conne$tion ,etween end points 1V+84% V+8 $onne$tion &ia e)isting I8 infrastru$ture

8ro&ides% 2 ,it network identifier 1VNI = defines VXLAN seg"ent4

% VM to VM $o""uni$ation on'* wit!in t!e sa"e VXLAN seg"ent

% VMs $an use t!e sa"e MAC?I8 addresses in different VXLAN seg"ents% VM unaware of en$apsu'ation

!is a'k% +)p'ains re$ent e)tensions and t*pi$a' traffi$ f'ow s$enarios% Mapping of VM addresses to V+8% Manage"ent of V+8


5/30


VXLAN Detai's


V)'an de&i$e% Network de&i$e wit! I8( MAC address and VNI

# ip link add vxlan0 type vxlan id 42 group 239.1.1.1 dev eth0

# ip link set vxlan0 address 54:8:20:0:0:{!"

# ip address add 20.0.0.{!"$8 dev vxlan0

# ip link set up vxlan0

% Creates and $onne$ts to D8 so$ket endpoint 1port 79?72 1&)'an44

% Eoins "u'ti$ast group% +n$apsu'ates a'' traffi$ wit! VXLAN !eader% ses D8 to forward traffi$ &ia et!0

-923-F73-003B

Swit$!

@ost B

et!0

&)'an0&)'an0

@ostA

et!0

2030303B

-923-F73-003A

2030303A

+GI8 !dr udp &)'an +GI8 !dr

V)'an pa*'oad

+t!0 pa$ket 1outer4

i$"p

V)'an0 pa$ket 1inner4

IANA 6erne'


6/30

F Software Defined Networking using VXLAN( !o"as #i$!ter 1t"ri$!tde3i,"3$o"4( Linu)Con 20-

VXLAN Detai's 124

@ost.A H ping 2030303B

5ind &)'an0 interfa$e and send out A#8 re


7/30 Software Defined Networking using VXLAN( !o"as #i$!ter 1t"ri$!tde3i,"3$o"4( Linu)Con 20-

New VXLAN 5eatures for /&er'a* Networks

Draw,a$ks% Missing $ontro' p'ane 1no $entra' $ontro' of V+8s and ta,'e "anage"ent4% Depends on "u'ti$ast routing support a&ai'a,i'it* 1wide area( routing ta,'e si;e4

% Mapping VNI to "u'ti$ast address

V)'an 5eatures re'eased into Linu) 6erne' 37 1D/V+ e)tensions4% LMISS Destination VM I8 address not in Neig!,or ta,'e

> rigger net'ink "essage to user spa$e> +)pe$t net'ink rep'* to add dst VM I8 address into Neig!,or ta,'e

% L2MISS MAC address not in VXLAN 5DB

> Do not ,road$ast to an* V+8 1"u'ti$ast4> rigger net'ink "essage to user spa$e> +)pe$t net'ink rep'* to add MAC address into VXLAN 5DB

% N/L+A#NINK Disa,'e snooping of in$o"ing pa$kets

> No entr* of MAC and destination V+8 address to VXLAN 5DB% /pti"i;ation 1for &irtua' ,ridges4

> 8#/XJ #ep'* on Neig!,or re



VXLAN 5orwarding Data,ase 15DB4

Maps destination VM MAC to V+8 I8% @as!ed( ke* is MAC address% Si;e 'i"itation possi,'e

Contains destination

% I8 Address% VNI port nu",er% /t!ers ti"esta"ps( f'ags

% Aging

Mu'tip'e destinations possi,'e

% 5or "u'ti$ast?a'' ;ero MAC address% rans"it to se&era' V+8

% /ne $op* per destination

se iproute2 too' to $reate?de'ete 5DB entries% Co""and %ridge &d% add$del$append$repla(e )

*+,- -xtensions


MAC

I8

VNI( 8ort

#e"ote V+8

MAC

I8

VNI( 8ort

#e"ote V+8

I8

VNI( 8ort

#e"ote V+8

uni$ast "u'ti$ast



VM Creation

Create &irtua' ,ridge wit! VXLAN de&i$e per VNI

Hip link add vxlan0 type vxlan id 1 l2iss l3iss rs( proxy nolearning

Neig!,or 5DB @ost A

A#8 2030303B = B2000B 1LMISS net'ink "essage45DB B2000B = -923-F73-003B 1L2MISS net'ink "essage4

Neig!,or 5DB @ost BA#8 2030303A = A2000A 1LMISS net'ink "essage45DB A2000A = -923-F73-003A 1L2MISS net'ink "essage4

raffi$ f'ow ,etween VM A VM BCan tra&e' a$ross internet


Swit$!

-923-F73-003B

Virtua' Bridge

V)'an

@ost B

NIC

VMB

@ost A

NIC

VMA

-923-F73-003A

2030303AA2000A

2030303BB2000B


10/30-0 Software Defined Networking using VXLAN( !o"as #i$!ter 1t"ri$!tde3i,"3$o"4( Linu)Con 20-

VM Migration

Create Virtua' Bridge wit! VXLAN de&i$e per VNI

@ost A De'ete +ntries( @ost C Add +ntries A#8 2030303B = B2000B 5DB B2000B = -923-F73-003B

@ost B Modif* +ntries A#8 2030303A = A2000A 5DB A2000A = -923-F73-003C Modif* on a'' !osts part of t!e 203)3)3) o&er'a* network

raffi$ f'ow ,etween VM A VM B


Swit$!

-923-F73-003B

Virtua' Bridge

V)'an

@ost B

NIC

VMB

@ost A

NIC

VMA

-923-F73-003A

@ost C

NIC

VMA

-923-F73-003C

2030303AA2000A

2030303BB2000B

2030303AA2000A


11/30

-- Software Defined Networking using VXLAN( !o"as #i$!ter 1t"ri$!tde3i,"3$o"4( Linu)Con 20-

VM #e"o&a'

De'ete Virtua' Bridge wit! VXLAN de&i$e per VNI

@ost C De'ete +ntries A#8 2030303B = B2000B 5DB B2000B = -923-F73-003B

@ost B De'ete +ntries A#8 2030303A = A2000A 5DB A2000A = -923-F73-003C Modif* on a'' !osts part of t!e 203)3)3) o&er'a* network


Swit$!

-923-F73-003B

Virtua' Bridge

V)'an

@ost B

NIC

VMB

@ost C

NIC

VMA

-923-F73-003C

20330303AA2000A

2030303BC2000B


12/30

-2 Software Defined Networking using VXLAN( !o"as #i$!ter 1t"ri$!tde3i,"3$o"4( Linu)Con 20-

VM Broad$ast?Mu'ti$ast

VM- Hping ., 20323232% Destination MAC ffffffffffffff% /ne entr* per V+8

raffi$ f'ow ,etween VM A VM B and VM C


Swit$!

-923-F73-003B

Virtua' Bridge

V)'an

@ost B

NIC

VMB

@ost A

NIC

VMA

-923-F73-003A

@ost C

NIC

VMC

-923-F73-003C

2030303CC2000C

2030303AA2000A

2030303BB2000B2030303A = A2000A

2030303B = B2000B

A#8

A2000A = -923-F73-003AB2000B = -923-F73-003B

5DB

2030303A = A2000A2030303C = C2000C

A#8

A2000A = -923-F73-003AC2000C = -923-F73-003C

5DB

2030303B = B2000B2030303C = C2000C

A#8

B2000B = -923-F73-003BC2000C = -923-F73-003C5555333333355 = -923-F73-003A5555333333355 = -923-F73-003C

5DB


13/30

- Software Defined Networking using VXLAN( !o"as #i$!ter 1t"ri$!tde3i,"3$o"4( Linu)Con 20-

Mu'tip'e Custo"er Setup

Create Virtua' Bridges wit! different VXLAN de&i$es and VNIs

raffi$ f'ow ,etween VM- VM2 and VMX VMJ% Iso'ation of 'ogi$a' networks 1defau't $onfiguration4

Cross 'ogi$a' network traffi$ possi,'e 1do"ain4

% Need $onfiguration% Add target VNI in VXLAN 5DB

B2-00B = -923-F73-003B VNI

Mu'tip'e nets &ia I8 routing VM X VM


VNI

5an out ,ased on VNI in

VXLAN !eaderSwit$!

-923-F73-003B

Virtua' Bridge

V)'an

2030303BB2000B

@ost B

NIC

VM2

2030303AA2000A

@ost A

NIC

VM-

-923-F73-003A

VMX

2030303AA2000A

A#8 2030303B = B2000B5DB B2000B = -923-F73-003B

A#8 2030303B = B2000B5DB B2000B = -923-F73-003BA#8 2-30303B = B2-00B5DB B2-00B = -923-F73-003B VNI 4

2-30303BB2-00B

A#8 2030303A = A2000A5DB A2000A = -923-F73-003A

A#8 2030303A = A2000A

5DB A2000A = -923-F73-003AVNI

VMVMJ

2030303BB2000B


14/30


+)terna' Conne$tions

+)terna' Conne$tions

% Lega$* VM to /&er'a* Network VM% A$$ess to +)terna' Network

Create VM wit! a$$ess to ,ot! networks

% Configure as gatewa*raffi$ f'ow ,etween

% VMX K: VM2?Internet


Swit$!

-923-F73-003B

Virtua' Bridge

V)'an

@ost B

NIC

VM2

@ost A

NIC

VMX

-923-F73-003A

K: VMVMJ

"a$&tap

NIC

Internet


15/30


Contro' 8'ane Neig!,or and 5DB a,'e Manage"ent

Agent runs on ea$! !ost

% Manipu'ates Neig!,or and 5DB entries% Kets VM I8 and MAC address

> D@C8 Snooping?Kratuitous A#8> IKM8 Snooping

% Data +)$!ange wit! AM

% Agent registers for 'i,&irtd "igration e&ents

Agent Manager% Define 'ogi$a' networks% Conne$ts to a'' agents% Mu'tip'e instan$es for re'ia,i'it*% Defines 8o'i$* 1ACL( firewa''( en$r*ption( gatewa*s( O4

% Do"ains 1/ne "ngt for "u'tip'e VNI networks4


Swit$!

-923-F73-003B

Virtua' Bridge

V)'an

@ost B

NIC

@ost A

NIC -923-F73-003A

VMAA#8 2030303B = B2000B5DB B2000B = -923-F73-003B

A#8 2030303A = A2000A5DB A2000A = -923-F73-003A

VMB

AgentAgent

AgentManager

-

2

-4 VM ,oot dete$ted ,* Agent

24Agent forwards I8?MAC to AM

4 C!e$k po'i$* and per"issions

4 Notifies Agents

4 Agents add entries


16/30

-F Software Defined Networking using VXLAN( !o"as #i$!ter 1t"ri$!tde3i,"3$o"4( Linu)Con 20-

#e"arks

Detai's

% 8re&ent frag"entation en route( set D5 ,it on V+8% D8 traffi$ ,etween V+8

Se$urit*% Se$ure $o""uni$ation ,etween Agent and Agent Manager% Agent Manager data ,ase prote$tion% Midd'e ,o)es 1firewa''( &irus s$anner4 "ust ,e VXLAN aware

I8 &F support under work% Mu'ti$ast support "issing

Ipta,'es( e,ta,'es( t$

% A&ai'a,'e on !ost side

A'ternati&es 1VLAN( I+++ 7023 ,g4

% Need !ardware $onfiguration on de&i$es% +)port VM MAC addresses to p!*si$a' network 1ta,'e si;e( S84



17/30


Su""ar*


Lo$ation independent addressing% VM assigned addresses retained w!i'e "o&ed in o&er'a* network

Logi$a' network s$a'ing% Independent of under'*ing p!*si$a' network and proto$o's% se e)isting I8 network infrastru$ture% No VM addresses in e)terna' swit$!es = ta,'e si;e( S8% No VLAN 'i"itation% No "u'ti$ast dependen$*

Address spa$e iso'ation

% Different tenants $an use sa"e addresses


18/30


#e'ated and 5uture :ork

#e'ated :ork

% /&er'a* transport> Si"i'ar $on$ept 1en$apsu'ation( inner and outer !eaders4> NVK#+

% #5C 27 and #5C2790

% K#+ proto$o' 10)F74 o&er I8> S

% Designed for NIC wit! S/( L#/% S proto$o' 1si"i'ar to C84 o&er I8

5uture :ork% Integration into /pen Sta$k 1See #eferen$e Nr3 4 and /pen &Swit$!



19/30


Questions?estions?


Send to t"ri$!tde3i,"3$o"
mailto:[email protected]:[email protected]


20/30


#eferen$es

-4M3 Ma!a'inga"( D3 Dutt et a'( ,/: rae'ork &or +verlaying ,irtualied ayer 2

et'orks over ayer 3 et'orks ,ersion 5! 7.Ma*.20-( !ttp??datatra$ker3ietf3org(Note !is is work in progress

24IBM6"7 * ,- hite aper!Eun.20-(-4!ttp??www.03i,"3$o"?s*ste"s?networking?so'utions?sdn3!t"'

4#a"i Co!en( et a'n intent;%ased approa(h &or net'ork virtualiation! I5I8?I+++Internationa' S*"posiu" on Integrated Network Manage"ent 1IM 20-4( 29.-.Ma*.

20-( pp 2.0

4#a"i Co!en( et a'*istri%uted +verlay ,irtual -thernet *+,- integration 'ith +pensta(k!I5I8?I+++ Internationa' S*"posiu" on Integrated Network Manage"ent 1IM 20-4( 29.-.Ma*.20-( pp -077.-079

4Vi&ek 6as!*ap( et'ork +verlays( Network Virtua'i;ation and Lig!tning a'ks( Linu)8'u",ers Conferen$e( August 29.-( 20-2( San Diego( CA( SA

http://datatracker.ietf.org/http://datatracker.ietf.org/http://datatracker.ietf.org/http://datatracker.ietf.org/


21/30

2- Software Defined Networking using VXLAN( !o"as #i$!ter 1t"ri$!tde3i,"3$o"4( Linu)Con 20-

#eferen$es 124

F4B3 Da&ie( E3 Kross( et a'( tateless -: et'ork ,irtualiation using =eneri( >outing -n(apsulation,ersion 3!7.Aug.20-( !ttp??too's3ietf3org?!t"'?draft.srid!aran.&irtua'i;ation.n&gre.0(Note !is is work in progress

http://tools.ietf.org/html/draft-sridharan-virtualization-nvgre-03http://tools.ietf.org/html/draft-sridharan-virtualization-nvgre-03http://tools.ietf.org/html/draft-sridharan-virtualization-nvgre-03http://tools.ietf.org/html/draft-sridharan-virtualization-nvgre-03http://tools.ietf.org/html/draft-sridharan-virtualization-nvgre-03http://tools.ietf.org/html/draft-sridharan-virtualization-nvgre-03http://tools.ietf.org/html/draft-sridharan-virtualization-nvgre-03http://tools.ietf.org/html/draft-sridharan-virtualization-nvgre-03http://tools.ietf.org/html/draft-sridharan-virtualization-nvgre-03http://tools.ietf.org/html/draft-sridharan-virtualization-nvgre-03http://tools.ietf.org/html/draft-sridharan-virtualization-nvgre-03http://tools.ietf.org/html/draft-sridharan-virtualization-nvgre-03


22/30


A$know'edg"ents

Vi&ek 6as!*ap( Ker!ard Sten;e'( Dirk @errendPrfer( MiQo Safradin( Srid!ar Su"adra'a( Da&id

Ste&ens( Vinit Eain IBM Linu) e$!no'og* Center( Data Center Networking



23/30


rade"arks

!is work represents t!e &iew of t!e aut!or and does not ne$essari'* represent t!e &iew ofIBM3

IBM is a registered trade"ark of Internationa' Business Ma$!ines Corporation in t!e nitedStates and?or ot!er $ountries3

NIX is a registered trade"ark of !e /pen Kroup in t!e nited States and ot!er $ountries 3

Linu) is a registered trade"ark of Linus or&a'ds in t!e nited States( ot!er $ountries( or,ot!3

/t!er $o"pan*( produ$t( and ser&i$e na"es "a* ,e trade"arks or ser&i$e "arks of ot!ers3



24/30


K'ossar*

Agent App'i$ation to "aintain Neig!,or?5DB ta,'es

AM Agent Manager

D/V+ Distri,uted /&er'a* Virtua' +t!ernet

5DB 5orwarding Data Base

La*er 2 /SI Data Link La*er 1#e'ia,'e Link ,etween dire$t'*$onne$ted nodes4

La*er /SI Network La*er 1I8 addressing4

L2MISS Destination MAC address unknown

LMISS Destination I8 address unknown

L+A#NINK Add new MAC?V+8 address in 5DB

Mu'ti enant Software Instan$e used for se&era' $usto"ers

NVK#+ Network Virtua'i;ation Keneri$ #outing +n$apsu'ation

/SI /pen S*ste"s Inter$onne$tion

/V /&er'a* ransport Virtua'i;ation

#SC #oute S!ort Cir$uit

SDN Software Defined Network


SS State'ess ransport unne'ing

VNI VXLAN Network Identifier or VXLAN Seg"ent Identifier

V+8 Virtua' unne' +nd 8oint

VXLAN Virtua' e)tensi,'e Lo$a' Area Network


25/30


BACKUPACKUP



26/30

2F Software Defined Networking using VXLAN( !o"as #i$!ter 1t"ri$!tde3i,"3$o"4( Linu)Con 20-

#oute S!ort Cir$uit 1#SC4

Neig!,or 5DB @ost A

A#8 20303032 = A20005+ 1-4 2-30303B = B2-00B 12,45DB B2-005+ = -3233 router 12a4

B2-00B = -923-F73-003B 14

-4 Look up router I8 to MAC "apping in neig!,or ta,'e

24 #outerf'ag seta4 #e"ote I8 address in 5DB entr* ignored,4 Look up destination I8 address to MAC "apping in neig!,or ta,'e

4 #ep'a$e destination MAC in inner !eader A20005+ = B2-00-

4 Look up destination MAC in 5DB and trans"it to V+8



Swit$!

-923-F73-003B

Virtua' Bridge

V)'an

@ost B

NIC

VMB

@ost A

NIC

VMA

-923-F73-003A

2030303AA2000A

2-30303BB2-00B

ip r add defau't &ia 20303032 ip r add defau't &ia 2-303032

H ping 2-30303A

Conso'e VM A


27/30


#oute S!ort Cir$uit 2 1Migration VM A to @ost B4

-4 Look up router I8 to MAC "apping in neig!,or ta,'e

24 #outerf'ag set

a4 #e"ote I8 address in 5DB entr* ignored

,4 Look up destination I8 address to MAC "apping in neig!,or ta,'e

4 #ep'a$e destination MAC in inner !eader A20005+ = B2-00-

4 Look up destination MAC in 5DB and feed ,a$k to 'o$a' ,ridge 1destination I8 03030304



Swit$!

-923-F73-003B

Virtua' Bridge

V)'an

@ost B

NIC

VMBVMA

2030303AA2000A

2-30303BB2-00B

ip r add defau't &ia 20303032 ip r add defau't &ia 2-303032

H ping 2-30303A

Conso'e VM A

5DB0-2 = -3233 router 124B2-00B = 0303030 14

A20030A = 0303030

Neig!,or20303032 = 0-2 1-42-303032 = 0-2

2030303A = A2000A

2-30303B = B2-00B 14


28/30


/pen Sta$k Integration

See paper #3 Co!en 1#eferen$es Nr 4

% Map ,ridge na"e to VNI


/pensta$k $o"pute node

No&aAgent

AgentAgentVI5

/pensta$k $o"pute node

No&a

AgentAgent

Agent

VI5/pensta$k $ontro' node

uantu"Mgr

AgentManager

Agent8'ugin

No&aMgr

@ori;on Das!,oard Agent +)t


29/30


VXLAN Standard 124

8ing 20303037 1on !ost F4-4 @ost F A#8 re


30/30

VM Atta$!"ent and Ma$&tap De&i$e /ptions


@ost B

App

"a$&tap0 "a$&tap-

Ma$&tap

Co",ines tun?tap and "a$&'an de&i$es

Modes

1-4Bridged destination MAC address 'ookup on a'' "a$&tap de&i$es defined on NIC

124Veparaffi$ forwarded to e)terna' swit$!

148ri&ate Sa"e as &epa( ,ut ingress traffi$ ,'o$ked

148asst!roug! /n'* - "a$&tap de&i$e a''owed per NIC 1e)$'usi&eT use4

NIC

?de&?tapX I?f to ser Spa$e 1tuntap4

Virtua' I?f wit! new MAC address

App

-

2

Documents

Linux and VXLAN 2013