VLAN Technology

Technology - VLAN is a method used to logical segment a physical network (IEEE 802.1Q standard) •Network separation is achieved by isolation on layer 2 connectivity •Delivery of Ethernet packets is restricted to members that belong to the same VLAN •A VLAN tag information (VID) is added to network frames either by the host or by the switchAdvantages •Provides more flexible network deployment over traditional network technology •Simplifies network administration and configuration •Can improve network performance by separating a network into different segments

VLAN Tagging

data

data

data

data

• Application

• TCPIP (AIX)

• Ethernet Adapter

• Network switch• or AIX VLAN device driver• or power hypervisor

A chunk of data

An IP packet(max.1500 byte)

An untagged Ethernet frame(max. 1516 bytes)

An a tagged Ethernet frame(max. 1532 bytes)

Layer

2La

yer

2La

yer

2

AIX VLAN Configuration (VLAN Unaware Hosts)

•VLAN definition at switch level (standard configuration) - VLANs are defined at switch level by the network administrator (frequently used) - VLAN tag is interpreted by the switch - PVID (Port VLAN ID = default VLAN ID) is added to packets entering switch and removed before packets are delivered to host • VLAN unaware host could be confused by receiving a tagged Ethernet frame (drop and indicate a frame error)

AIX/LinuxVLAN unaware host

NetworkInterfaceLayer3

en 0VLAN n

Frame A

Frame B

Frame B

Frame A

Network switch with PVID Definition

AIXVLAN aware host

ent 1

Frame B

Frame A

AIX VLAN Configuration (VLAN Aware Hosts)

• AIX can define VLANs based on any physical Ethernet adapters - VLANs are defined at AIX level by the system administrator - VID tag is added to Ethernet frames by AIX logical device (VALN device driver) • VLAN tag is interpreted by the logical device - Switch must be configured with tagged port

Network switch with PVID and additional VID Definition

Frame AFrame

B

en 0en 1

Tagged or untagged frames

VLAN n and p

Dynamic VLAN Registration Protocol

• Generic Attribute Registration Protocol (GARP) - Generic link –layer protocol that allows different applications to propagate information between switches - packets sent for GARP protocols are called Bridge Protocol Data Units(BPDU)• GARP VLAN Registration Protocol(GVRP) - GVRP is a GARP application that allows for the dynamic registration of VLANs over networks - Reduce the work required by network administrators • The network automatically learns about the VLAN topology - Makes setup easier - Only the host needs to be configured with VLAN , rather than all switch ports - GVRP support with virtual i/o server version 1.4 (fix pack 9.1)

• GVRP is implemented with in the existing shared Ethernet adapter(SEA) driver code - GVRP announce to the physical network the VLANs that have been statically configure on the SEAs virtual adapter - To use GVRP when creating an SEA •$ mkdev –sea ent0 –vadapter ent1 –default ent1 defaultid 1 –attr gvrp=yes

Physical Network to PowerVM Virtulation • POWER Hypervisor Ethernet switch: Network

virtualization • Interpretation communications • External network communications

AIX LPAR

AIX LPAR

AIX LPAR

AIX LPAR

enx

POWER Hypervisor

POWER system using Physical adapter

Enterprise Network

Physical DevicePCI Eth or LHEA port

VLAN

AIX LPAR

AIX LPAR

AIX LPAR

AIX LPAR

enx

POWER Hypervisor

Enterprise Network

VIOSLPAR

POWER 5 or POWER 6 SYSTEM

External Network Access with or without VIOS

AIX LPAR

TCPIP ROUTING

AIX LPAR

AIX LPAR

AIX LPAR

AIX LPAR

AIX LPAR

AIX LPAR

AIX LPAR

enx en

x

POWER Hypervisor POWER Hypervisor

VIOS LPAR

Physical Devices

Enterprise

Network

Enterprise

Network

Routing with Gateway LPAR

Bridging with SEA

VLANs

Multiple Virtual Switch (POWER6 only)

• Multiple Virtual machines can be defined in IBM System p6 --Vswitches are defined at system level(Hypervisor)• One VSwitch (VSwitch0) is defined by default• Create a switch from HMC --select server->configuration->virtual network management• VLANS are not able to communicate through different Vswitches• No VLAN predefinition• Create a Virtual Ethernet and select a Vswitch --Select the Vswitch and define the PVID and additional V Ds

AIX LPAR AIX

LPARAIX

LPAR

AIX LPAR

AIX LPAR

entx

entx

entx

entx

entx

V switch A V switch B

V switch C

POWER Hypervisor

VLAN X

VLAN X

VLAN X

VIOS LPAR

Power6 Server

Physical Device

Enterprise

Network

Bridging to External Network using SEA Feature

AIX LPAR

en0

(if)

ent0(virt)

• Bridging - VIOS partition uses SEA for external network access - One virtual Ethernet adapter for one or many VLANs on the VIOS

ent1(virt)

VIOS

PHPY

ent2SEA

ent0(phy)

To other switch

PO

WER

6 S

erv

er

AIX LPAR AIX LPAR AIX LPARen0

(if)

en0

(if)

en0

(if)

ent0(virt)

ent0(virt)

ent0(virt)

PVID118

PVID218

PVID318

PHPY

ent1(virt)

ent2SEA

ent0(phy)

VIOS PVID=99Added VLANs VIDs=118,216,318

PO

WER

6 S

erv

er

To other switch

• SEA backing device supports: - PCI Ethernet adapter. - Logical host Ethernet Adapter (LHEA)port with promiscuous mode.• The only logical port configurable is number 1 on the physical port• Corresponding IVE physical port is dedicated for SEA use• All the IVE switch bandwidth is assigned to the SEA

Physical Ethernet Adapter for the SEA backing device

AIX LPAR

PCI adapterOr

LHEA port with Promiscuous mode

en0

(if)

ent0(virt)

ent1(virt)

ent2SEA

VIOS

PO

WER

6 S

erv

er

Switch

PHYP

• Two IVE physical ports set to promiscuous mode are required to improve the network connectivity of the SEA.

• IVE communicates directly to logical partitions (LPARs). -- It is physical device, it must before partition mobility. -- Each IVE physical port is separated network Switch. -- It reduces the interaction with POWER Hypervisor. -- 10Gbps IVE feature has better performance than virtual Ethernet. -- IVE supports a maximum of 16 or 32 LPARs.• IVE reduces the need for physical Ethernet adapters.

Accessing External Network: IVE (Power versus SEA

VIOS AIX AIX LINUX AIX AIX LINUX

SEAVirtual

Ethernet Driver

Virtual Ethernet Driver

Virtual Ethernet Driver

Network Adapter

Virtual Ethernet switch

Hypervisor

LHEADriver

LHEADriver

LHEADriver

Integrated Virtual Ethernet

LAN

Using VIO Shared Ethernet Adapter Using Integrated Virtual Ethernet

Network Availability: Methodology

• Define which level of availability is needed --Accepted network downtime (from millisecond to hour) --Network maintenance timeframe• Find corresponding SPOFs that can be eliminated --Hardware and software single point failure(SPOF)• Identify which type of availability feature could be used --Hardware (physical adaptor or equipment) --Hypervisor(virtual adapters) --AIX, Linux, or VIOS• Implement availability options /features --step-by-step process implementation to minimize down time• Test failover and recovery conditions --Check mechanism used to failover and recover• Verify disk configuration and partition migration dependencies. --Example : Disk MPIO requires Dual VIOS configuration.

Network Feature Used for Availability

• Hardware - Redundant Ethernet adapters - Dual switch configuration• AIX - NIB (Ether Channel backup - Link aggregation (Ether Channel) - TCPIP multipath routing with Dead Gateway detection• IBM POWER Systems - Virtual Ethernet adapter and virtual hypervisor switch - Virtual I/O server or gateway partition - Virtual I/O server used as a backup network access - Dual shared Ethernet adapter with NIB at the client - SEA failover feature

Network availability

=

Mix of several features

Availability Solutions: External Access

• SPOFs/Resilience --Physical Ethernet Adapter --VIO’s partition --External switch port / switch• Solutions for availability --VIOS physical link aggregation --LHEA ports link aggregation --Single VIOS as Network Backup --Dual VIOS •Multipath routing with DGD on client •NIB at client •SEA Failover

POWER Server

AIX Client AIX Client

VirtEnt

VirtEnt

Physical Ent

Shared Ethernetadapter

VIOS 1

Switch

Solutions: VIOS physical Link Aggregation

• Link Aggregation at VIOS - one primary adapter with one backup adapter •Two different switches without extra configuration - Several primary adaptors with one backup •All primary adaptors must be connected to the same physical switch •Extra configuration on switch is needed - Easy to setup and manage - No special configuration on client partition - single or multiple LANs at client partition• Resilience - VIOS partition is a point of Failure• Performance - Network bandwidth is increased

PhysEnt

PhysEnt

switch switch

Solutions: AIX Link Aggregation with LHEA ports

• Link aggregation at AIX VIOS - one primary adapter with one backup adapter • Two different switches without extra configuration - Several primary adapters with one backup • All IVE physical ports corresponding to the primary adapters must be connected - Set up the external address to ping LA parameter - No need for VIOS - Multiple LANs requires VLAN tags at client partition.• Resilience - IVE adapter is a point of failure. • Use multiple IVE adapters(only p57/02CECs)• Performance - Low latency and high network bandwidth

AIX Client

ent

ent

LHEA LHEA LHEA

ent

LINK Agent

backup

LHEA port

LHEA port

LHEA port

PhysPort

PhysPort

PhysPort

Primary

Switch Switch

POWER Server

IVE Adapter

Single VIOS configuration as network Backup

Use a VIOS to provide network Backup path

• Complexity -- Requires configuration on client(NIB) -- Need to ping outside the client initiate NIB failover resilience -- protects against single switch port / switch /Ethernet adapter failure• Throughput / scalability -- High bandwidth applications may benefit from the physical adapter -- Backup performance limited to a single Ethernet adapter and VIOS CPU• Notes -- Useful for multiple LPAR’s configuration -- NIB does not support tagged VLANs on physical LAN

AIX Client AIX Client

NIB NIB

PhysEnt

PhysEnt

VirtEnt

VirtEnt

VIOS 1

PhysEnt

Shared Ethernetadapter

Switch

Switch

POWER Server

Dual VIOS Configurations: Three solutions

• Routing: Two shared Ethernet adapters on different VLANS with multi path routing on client partition

• NIB: Two shared Ethernet adapters in different VLANs with NIB on the client partition• Shared Ethernet adapter failover feature.

Benefits: --Bridging solutions are easy to setup --No specialized configuration on switch component --Needs two physical Ethernet adapters (one per virtual I/O server

Needs additional CPU, RAM, and disk for second VIOS partition

AIX Client

VirtEnt

AIX Client

VirtEnt

VIOS 1 VIOS 2

Shared Ethernetadapter

Shared Ethernetadapter

PhysEnt

PhysEnt

Switch

Switch

POWER Server

• Considerations - No VLAN tagging (only untagged packets) - Need two virtual Ethernet adapters at client partition - Special client partition configuration (one NIB for each AIX network) - Address to ping must be reliable• Benefits: - Allows manual load balancing configuration of client through both VIO servers - No specific configuration needed on external physical switch

Dual VIOS Configuration with NIB at Client

AIX Client AIX Client

NIB NIBVirtEnt

VirtEnt

VirtEnt

VirtEnt

VIOS 1Shared Ethernetadapter

PhysEnt

VIOS 1Shared Ethernetadapter

PhysEnt

Switch

Switch

POWER Server

Dual VIOS with NIB at client: Single LAN (Details)

Clie

nt LPA

R 1

Clie

nt LPA

R 2

en2

(if)ent2NIB

ent0virt

ent1virt

backup

en2

(if)ent2NIB

ent0virt

ent1virt

backup

Dual VIOS configuration with SEA failover Feature : A Best Practice

• Considerations - Is supported at VIOS 1.2 and above - Has no load balancing for single configuration - Supports VLAN tagging• Benefits - Easy client partition configuration - Network configuration simplified compared to the NIB• Remarks - The control channel is critical for VIO server synchronization. only one SEA must be active at a time - Two parameters for the support:• Priority• Control channel

Heart beat

SEA failover Feature: Testing

• Manual failover: - set ha_mode to standby on primary: the SEA is expected to fail over >chdev –dev ent2 –attr ha_mode=standby - Reset ha_mode to auto on primary :the SEA is expected to failover• VIOS shutdown: - Reboot VIOS on primary: the SEA is expected to failover - when primary VIOS comes up again: the SEA is expected to fail back• VIOS error: - Deactivate primary VIOS on HMC: the SEA is expected to failover • Active and boot VIOS: the SEA is expected to fail back• Physical link failure: - Unplug the link of the physical adapter on the primary: the SEA is expected to fail over • Reconnect the link of physical adapter on primary :the SEA is expected to failback• Reserve the boot sequence: - shut down both VIO servers •Boot the standby VIOS : the SEA is expected to become active on standby. •Boot the primary VIOS: the SEA is expected to fail back

Migrating from NIB configuration to SEA failover

• Converting from older NIB method

• Use DLPAR to add required virtual adapters - control channel, trunk - Minimize downtime

• NIB configuration removed at later time - Client’s IP address is associated with the NIB - Downtime when moving IP address to another interface (short planned downtime on the Client)

Multiple LANs in the client partition: Availability solution

Multiple LANs solution is driven by VLAN tagging rules:

• If only untagged frames are propagated to external network infrastructure (VLAN unaware hosts) - No VLAN ID administration at AIX and supervisor level - Need to segment traffic in Hypervisor (thru PVID configuration) - Multiple gateway s or SEAs and multiple LANs in virtual switch - Easy network administration, complex virtual architecture.• If tagged frames are supported by external switch - AIX and Hypervisor VIDs must be defined according enterprise network architecture - No additional gateway or SEA for multiple VLAN’s configuration

Thank you