Unique Access Solutions

OAM and QoS

Presented by:

Yaakov (J) Stein

Chief Scientist

OAMQoS-YJS Slide 2

SERVICE GUARANTEES

OAMQoS-YJS Slide 3

Generally good (and frequently much better than toll quality)voice service is available free of charge (Skype, Fring, Nimbuzz, …)

So why does anyone pay for voice services ?

Similarly, one can get free • (WiFi) Internet access• email boxes• file storage and sharing• web hosting• software services

So why pay ?

Why do we pay for services ?

OAMQoS-YJS Slide 4

The simple answer is that one doesn’t pay for the serviceone pays for Quality of Service guarantees

In our voice model

But what does QoS meanand why are we willing to pay for it ?

To explain, we need to review some history

Paying for QoS

QoS

price

BE

toll qualitywith mobility

OAMQoS-YJS Slide 5

Everyone knows that the father of the telephone wasAlexander Graham Bell (along with his assistant Mr. Watson)

But Bell did not invent the telephone network

Bell and Watson sold pairs of phones to customers

The father of the telephone network wasTheodore Vail

Father of the telephone

OAMQoS-YJS Slide 6

Theodore Vail -

Theodore Who?Son of Alfred Vail (Morse’s coworker)Ex-General Superintendent of US Railway Mail Service First general manager of Bell TelephoneFather of the PSTN

Why is he so important?Organized PSTNEstablished principle of reinvestment in R&DEstablished Bell Telephones IPR divisionExecuted merger with Western Union to form AT&TSolved the main technological problems • use of copper wire• use of twisted pairs

Organized telephony as a service (like the postal service!)

Vailism is the philosophy that public services should be run as closed centralized monopolies for the public good

OAMQoS-YJS Slide 7

What’s the difference ?

In the Bell-Watson modelthe customer pays once, but is responsible for

• installation• wires• wiring

• operations• power• fault repair• performance (distortion and noise)

• infrastructure maintenancewhile the Bell company is responsible only for

providing functioning telephones

In the Vail model the customer pays a monthly feebut the provider assumes responsibility for everythingincluding fault repair and performance maintenance

the telephone company owns the telephone sets and even the wires in the walls !

+

OAMQoS-YJS Slide 8

In order to justify recurring paymentsthe provider agrees to a minimum level of service in an SLA

SLAs should capture Quality of user Experience (QoE)but this is often hard to quantify

So SLAs usually actually detail measurable network parameters that influence QoE, such as :

• availability (e.g., the famous five nines)• time to repair (e.g., the famous 50 ms)• information rate (throughput)• information latency (delay)• allowable defect densities (noise/distortion)

Availability (basic connectivity) always influences QoE

It is hard to predict the effect of the other parameters on QoE even when there is only one application (e.g., voice)

When multiple applications are in use - it may be impossible

Service Level Agreements

OAMQoS-YJS Slide 9

System trafficrouting protocols, DNS, DHCP, time delivery, system update, OAM, tunneling and VPN setup

Business processes database access, backup and data-center, B2B, ERP

Communications - interactivevoice, video conferencing, telepresence, instant messaging,remote desktop, application sharing

Communications – non-interactiveemail, broadcast programming, music

video : progressive download, live streaming, interactive

Information gatheringhttp(s), Web 2.0, file transfer

Recreationalgaming, p2p file transfer

Malicious DoS, malware injection, illicit information retrieval

Some Applications

OAMQoS-YJS Slide 10

Some applications only require availability

Some also require minimum available throughput

Some require delay less then some end-end (or RT) delay

Some require packet loss ratio (PLR) less than some percentageand these parameters are not necessarily independent

For example,

TCP throughput drops with PLR

What do applications need ?

1000 B packets50 ms RTT

OAMQoS-YJS Slide 11

Mission Critical (and life critical) applications require• high availability

If there are any MC applications then system traffic requires high availability too

MC applications do not necessarily require strict throughputbut always indirectly require

• a certain minimal average throughput • bounded delay

If the MC application uses TCP then it requires • low PLR

Real-time applications require• sufficient throughputbut not necessarily low PLR (audio and video codecs have PLC)

Interactive applications require • low RT delayIt may be more scalable for a SP to measure 1-way delays

Some rules of thumb

OAMQoS-YJS Slide 12

OAM

OAMQoS-YJS Slide 13

The Service Provider’s justification for payment is the maintenance of an SLA

To ensure SLA compliance, the SP must : • monitor the SLA parameters• take action if parameter is dropping below compliance levels

But how does the SP verify/ensure that the SLA is being met ?

Monitoring is carried out usingOperations, Administration, Maintenance (OAM)

The customer too may use OAM to see that the SP is compliant !

Technical note:OAM is a user-plane function

but may influence control and management plane operationsfor example• OAM may trigger protection switching, but doesn’t switch• OAM may detect provisioned links, but doesn’t provision them

Monitoring an SLA

OAMQoS-YJS Slide 14

Operations, Administration, Maintenance

Traditionally, one distinguishes between 2 OAM functionalities :

1.Fault Monitoring• OAM runs continuously/periodically at required rate• detection and reporting of anomalies, defects, and failures• used to trigger mechanisms in the

• control plane (e.g. protection switching) and • management plane (alarms)

• required for maintenance of basic connectivity (availability)

2.Performance Monitoring• OAM run :

• before enabling a service• on-demand or • per schedule

• measurement of performance criteria (delay, PDV, etc.)• required for maintenance of all other QoE attributes

OAMQoS-YJS Slide 15

Analog channels and 64 kbps digital channels did not have mechanisms to check signal validity and quality

Thus • major faults could go undetected for long periods of time• hard to characterize and localize faults when reported• minor defects might be unnoticed indefinitely

As PDH networks evolved, more and more OAM was added on :• monitoring for valid signal• loopbacks• defect reporting • alarm indication/inhibitionThe OAM overhead started to explode in size !

When SONET/SDH was designed bounded overhead was reserved for OAM functions

Early OAM

OAMQoS-YJS Slide 16

OAM is more complex for Packet Switched Networks

in addition to the previous defects : • loss of signal• bit errorswe have new defect types• packets may be lost• packets may be delayed• packets may delivered to the wrong destination

The first PSN-like network to acquire OAM was ATM (I.610)

Although technically ATM is cell-based, not packet-based

OAM for Packet Switched Networks

OAMQoS-YJS Slide 17

How do we perform Continuity Check ?• send OAM packets at a constant known rate• if CC packets are not received for >3 intervals then declare a fault

see also LB / echo mode

How do we perform Connectivity Verification ?• send OAM packets to a known destination• if CV packets are received somewhere else then declare a fault

How do we indicate AIS (FDI) ?• when do not receive forward traffic send AIS OAM packets • if AIS packets received then declare a fault

How do we indicate RDI (BDI) ?• when do not receive reverse traffic send RDI OAM packets • if RDI packets received then declare a fault

Note: RDI is often a flag set on CC message

Some FM OAM mechanisms (1)

OAMQoS-YJS Slide 18

How do we use LoopBack ?• non-intrusive (in-service) (echo mode)

• send LB request OAM packet to remote site• remote site replies with LB reply• if LB reply not received then declare a fault

• intrusive (out-of-service)• put remote site into LB mode• remote sites reflects (and does not forward) all traffic

(note that it must monitor OAM traffic)• if packets sent are not received then declare a fault

note: need to inform next hops of LB by locking

How do we use LinkTrace ?• send LB request OAM packet to next hop• send LB request to following hop• etc.

Some FM OAM mechanisms (2)

OAMQoS-YJS Slide 19

How do we measure Packet Loss Ratio ?• Traffic (counter) based

maintain 2 counters:• number of packets transmitted to peer Tx• number of packets received from peer Rx

• send Tx counter to peer at time 1 Tx(1)• peer notes its Rx counter at time of reception Rx(2)

and its Tx counter at time of its reply Tx(3)• originator notes its Rx counter when reply is received Rx(4)calculate PLR in both directions

• Synthetic :do not maintain counters – use OAM packetsNote : synthetic loss is only a rough estimate

How do we measure Throughput?• Primitive way (RFC 2544)

• send packets at maximum rate and observe packet loss• reduce rate until no loss is observed

Note : there are more sophisticated mechanisms !

Some PM OAM mechanisms (1)

OAMQoS-YJS Slide 20

How do we measure 1-way Packet Delay (Latency) ?synchronize clocks at both OAM peers• send timestamp T1 to peer• peer timestamps receipt with T2calculate time difference T2 – T1

How do we measure 2-way Packet Delay (Latency) ?• send timestamp T1 to peer• peer timestamps receipt with T2• peer replies at T3• originator timestamps receipt of reply at T4calculate time difference (T4 – T1) – (T3 - T2)assuming symmetry, 1-way delay is half this amountNote : do not need to synchronize clocks

How do we measure Packet Delay Variation ?• send timestamps at a constant rate• peer calculates timestamp differences and statistics thereofNote : do not need to synchronize clocks

Some PM OAM mechanisms (2)

OAMQoS-YJS Slide 21

ETHERNET OAM

OAMQoS-YJS Slide 22

Carrier Ethernet has replaced ATM as the default layer-2

Ethernet is by far the most widespread network interface

Ethernet has some advantages as compared to ATM• it has network-wide unique addresses• it has a source address in every packet

but some aspects make Ethernet OAM more difficult• ConnectionLess (CL)• multipoint to multipoint• overlapping layering – need OAM for operator, SPs, customer• some specific problematic ETH behaviors (flooding, multicast, …)

What about Ethernet ?

OAMQoS-YJS Slide 23

OAM makes a lot of sense in Connection Oriented environments• connections last a relatively long amount of time• there is some SLA at the connection level

For CL networks, the network path is neither known nor pinned

So it doesn’t really make sense to talk about FMwhat does continuity mean if when a link goes downthe network automatically reroutes around the failure ?

The Ethernet CL problem is solved by overlaying CO functionality :• flows or• EVCs

What’s the problem with CL ?

OAMQoS-YJS Slide 24

For many years there was no OAM for Ethernet(LANs don’t need OAM)now there are two incompatible ones!

• Link layer OAM – 802.3 clause 57 (EFM OAM, 802.3ah)single link onlyslow protocol, limited functionalitysome management functions

• Service OAM – Y.1731, 802.1ag (CFM)any network configurationmultilevel OAM functionality

In some cases one may need to run bothwhile in others only service OAM makes sense

Link layer OAM is only for a single link, which is necessarily COService OAM is most frequently used for infrastructure networks,

which are also CO

Ethernet OAM

OAMQoS-YJS Slide 25

Layer 2 control protocols (L2CPs)

Do not be confused - L2CPs are NOT OAM !Here are a few well-known L2CPs :

protocol DA reference STP/RSTP/MSTP 01-80-C2-00-00-00

802.2 LLC802.1D §8,9802.1D§17 802.1Q §13

PAUSE 01-80-C2-00-00-01 802.3 §31B 802.3x

LACP/LAMP 01-80-C2-00-00-02EtherType 88-09

Subtype 01 and 02

802.3 §43 (ex 802.3ad)

Link OAM 01-80-C2-00-00-02EtherType 88-09

Subtype 03

802.3 §57 (ex 802.3ah)

ESMC 01-80-C2-00-00-02EtherType 88-09

Subtype 10

G.8264

Port Authentication 01-80-C2-00-00-03 802.1X

E-LMI 01-80-C2-00-00-07 MEF-16

Provider MSTP 01-80-C2-00-00-08 802.1D § 802.1ad

Provider MMRP 01-80-C2-00-00-0D 802.1ak

LLDP 01-80-C2-00-00-0EEtherType 88-CC

802.1AB-2009

GARP (GMRP, GVRP) Block 01-80-C2-00-00-20 through 01-80-C2-00-00-2F

802.1D §10, 11, 12

Note: IEEE disallows forwarding of L2CPs, MEF allows it under certain circumstances

OAMQoS-YJS Slide 26

Link Layer OAM (AKA EFM OAM)

Ethernet in the First Mile (Last Mile ?)

EFM networks are mostly p2p DSL links or p2mp PONsthus a link layer OAM is sufficient for EFM applications

Since EFM link is between customer and Service ProviderEFM OAM entities are either active (SP) or passive (customer)active entity can place passive one into LB mode but not the reverse

EFM OAMPDUs are a slow protocol frames – never forwardedEthertype = 88-09 and subtype 03

messages multicast to slow protocol specific group addressOAMPDUs must be sent once per second (heartbeat)messages are TLV-based

DA01-80-C2-00-00-02

SATYPE

8809

SUBTYPE

03

FLAGS(2B)

CODE(1B) DATA CRC

OAMQoS-YJS Slide 27

EFM OAM capabilities

6 codes are defined• Information (autodiscovery, heartbeat, fault notification)• Event notification (statistics reporting)• Variable request (active entity query passive’s configuration) (mngt)

• Variable response (passive entity responds to query) (mngt)

• Loopback control (active entity enable/disable of intrusive LB mode)• Organization specific (proprietary extensions)

and there are flags in every OAMPDU toexpedite notification of critical events • link fault (RDI)• dying gasp• unspecifiedmonitor slow degradations in performance

OAMQoS-YJS Slide 28

Service OAM (AKA CFM, Y.1731)

Many SPs need to monitor full networksnot just single links

Service layer OAM provides end-to-end integrity of the Ethernet service over arbitrary server layers

Because Ethernet is flatnot true client-server layering (except MAC-in-MAC)

service layer OAM is multilevel

Because SPs want to replace transport networks with Ethernetservice OAM must support all OAM featuresand must enable advanced transport capabilities(such as linear/ring protection switching)

a transport network is a network with :1. High availability (Fault Management OAM and Automatic Protection Switching)2. SLA support (Performance Management OAM and QoS mechanisms)3. a Management plane (optionally a control plane) for configuration and provisioning4. Efficiency and Scalability

OAMQoS-YJS Slide 29

Y.1731 messages

Y.1731 supports many OAM message types:• Continuity Check proactive heartbeat with 7 possible rates • Synthetic Loss Measurement on demand loss rate estimation• LoopBack unicast/multicast pings with optional patterns• Link Trace identify path taken to detect failures and loops• AIS periodically sent when CC fails• RDI flag set to indicate reverse defect • Client Signal Fail sent by MEP when client doesn’t support AIS • LoCK signal inform peer entity about diagnostic actions• TeST signal in-service/out-of-service tests for loss rate, etc. • Automatic Protection Switching• Maintenance Communications Channel remote maintenance• EXPerimental• Vendor SPecific

OAMQoS-YJS Slide 30

Y.1731 frame format

after DA, SA and Ethertype (8902)Y.1731/802.1ag PDUs have the following header (may be VLAN tagged)

if there are sequence numbers/timestamp(s)they immediately follow

then come TLVs, the “end TLV”, followed by the CRC

TLVs have 1B type and 2B length fieldsthere may or not be a value fieldthe “end-TLV” has type = zero and no length or value fields

LEVEL(3b)

OPCODE(1B)

VER(5b)

FLAGS(1B)

TLV-OFF(1B)

OAMQoS-YJS Slide 31

Y.1731 PDU typesopcode OAM Type DA

1 CCM M1 or U

3 LBM M1 or U

2 LBR U

5 LTM M2

4 LTR U

6-31 RES IEEE

32-63 unused RES ITU-T

33 AIS M1 or U

35 LCK M1or U

37 TST M1 or U

39 Linear APS M1or U

40 Ring APS M1or U

41 MCC M1 or U

43 LMM M1 or U

42 LMR U DA

45 1DM M1 or U

47 DMM M1 or U

46 DMR UA

49 EXM

48 EXR

51 VSM

50 VSR

52 CSF M1 or U

55 SLM U

54 SLR U

64-255 RES IEEE

OAMQoS-YJS Slide 32

MEPs and MIPs

Maintenance Entity (ME) – entity that requires maintenanceME is a relationship between ME end points

because Ethernet is MP2MP, we need to define a ME Group

MEGs can be nested, but not overlapped

MEG LEVEL takes a value 0 … 7by default - 0,1,2 operator, 3,4 SP, 5,6,7 customer

MEP = MEG end point (MEG = ME group, ME = Maintenance Entity) (in IEEE MEG is called MA = Maintenance Association)unique MEG IDs specify to which MEG we send the OAM message

MEPs responsible for OAM messages not leaking outbut transparently transfer OAM messages of higher level

MIPs = MEG Intermediate Points • never originate OAM messages, • process some OAM messages• transparently transfer others

OAMQoS-YJS Slide 33

MEPs and MIPs (cont.)

OAMQoS-YJS Slide 34

How is OAM used ?

MEF-30 Service OAM FM and MEF-xx Service OAM PMdescribe the use of OAM for Carrier Ethernet networks, such as

• which Y.1731/802.1 features/messages should be used• where to put MEPs, what MA and MEG levels names should be used• minimum number of EVCs that must be supported• what should be reported and how

Y.1564 (ex Y.156sam) Ethernet Service Activation Test Methodologydescribes commissioning procedures (replaces RFC2544-like benchmarking)

Tests that desired performance level can be achieved, including• CIR, EIR (and optionally CBS and EBS for bursting)• traffic policing• rate, loss, delay, delay variation, availability (measured simultaneously)Testing in two steps :• Service Configuration Test – each service separately• Service Performance Test – all services togetherPerformance testing may be for :• 15 minutes (new service on operational network)• 2 hours (single operator network)• 24 hours (multiple operator networks)

OAMQoS-YJS Slide 35

QOS ENFORCEMENT

OAMQoS-YJS Slide 36

There are two approaches to QoS handling

IntServ (guaranteed QoS)• define traffic flows (CO approach)• guarantee QoS attributes for each flow• reserve resources at each router along the flow• signaling protocol (e.g., RSVP) needed

DiffServ (statistical QoS)• retain CL paradigm• no guaranteed QoS attributes• mark packets (differentiated – e.g., gold, silver, bronze)

• marking can be by VLAN, P-bits, IP-ToS/DSCP, or general “flow”• offer special treatment (priority) relative to other packets• no resource reservation

For Ethernet and IP DiffServ is the preferred approach

QoS approaches

OAMQoS-YJS Slide 37

Example:For an IPv4 packet inside Q-in-Q Ethernet

we have various choices for marking priority

Some fields for marking

DA (6B)

SA(6B)

ET=8100 (2B) P(3b) CFI(1b) CVID(12b)

ET=88A8 (2B) P(3b) DEA(1b) SVID(12b)

ET=0800 (2B)

Ver(4b) IHL(4b) ToS(1B) Len(2B)

Source IP Address (4B)

Destination IP Address (4B)

. . .

802.1puser priority field AKA P-bits 0 … 7priority tagging (VLAN=0) if no VLANP=0 means non-expedited traffic802.1Q recommends mappings

IP ToSRFC 2474 redefined ToS to contain• 6 bit DSCP (see also RFC 4594)• 2 bit ECN

OAMQoS-YJS Slide 38

Queuing

Ethernet switches have queues FIFO bufferson each output port

If there were only one queue then traffic handling would be FIF

To enable DiffServ prioritization multiple queues are used

Outgoing frames are inserted into queues according to priority marking

Many methods for emptying queuesThe most popular are :• Strict Priority

always take from nonempty queue of highest priority

• Weighted Fair Queuingtake from nonempty queues accordingto configured “weight”

switchfabric

input p

ort

input p

ort

input p

ort

output port

output port

output port

output portq

ueu

e

qu

eue

qu

eue

qu

eue

OAMQoS-YJS Slide 39

Traffic shaping

One of the most important parts of an SLA is theCommitted Information Rate (bps)

This is the datarate (bandwidth) SP guarantees will be forwarded

There may also be an Extra Information Rate (bps)

This is a datarate that the SP will forward if possible

Packet traffic is often burstyA customer who did not send data for a while

will expect to be able to send a higher rate afterwards

This is accomplished via traffic shaping • time integration is accomplished by leaky/token buckets• the effect of shaping is marking drop eligibility

(marking a packet on the line is only possible with S-tags!)

There is often also traffic policingpolicing simply discards packets to police a maximum rate !

OAMQoS-YJS Slide 40

MEF token bucket algorithm

Metro Ethernet Forum 10.x defines a bandwidth profile

there are two byte buckets, C of size CBS and E of size EBS (in bytes)tokens are added to the buckets at rate CIR/8 and EIR/8

when bucket overflows tokens are lost (use it or lose it)

if ingress frame length < number of tokens in C bucketframe is green and its length in tokens is debited from C bucket

elseif ingress frame length < number of tokens in E bucket

frame is yellow and its length of tokens is debited from E bucket

else frame is red

green frames are deliveredand service objectives apply

yellow frames are deliveredbut service objectives don’t apply

red frames are discarded

CBSEBS

C E

for simplicity we assume• no coupling and • no sharing !