Distributed Systems 3. Network Quality of Service (QoS)pxk/417/notes/content/03-qos-slides.pdf · Quality of Service Problems in IP • Bandwidth mismatches and aggregation –Lead

Distributed Systems

3. Network Quality of Service (QoS)

Paul Krzyzanowski

[email protected]

1 10/1/2012 © 2012 Paul Krzyzanowski

What factors matter for network performance?

• Bandwidth (bit rate)

– Average number of bits per second through the network

• Delay (latency)

– Average time for data to get from one endpoint to its destination

• Jitter

– Variation in end-to-end delay

• Loss (packet errors and dropped packets)

– Percentage of packets that don’t reach their destination


Why do we care?

• We want to provide users with

– Good interactive performance

– Glitch-free, delay-free streaming video

– Good quality IP telephony


Quality of Service Goals

• Provide applications with the data flow performance they need:

– Bandwidth, delay, jitter, loss

• Examples

– VoIP: low bandwidth, low latency (<150 ms), low jitter (<30 ms)

– Video streaming: high bandwidth, longer latency OK

– Remote login: low bandwidth, relatively low latency, high reliability

– File transfer: high bandwidth, high latency OK, high jitter OK , high

reliability


Why do we have QoS problems?

• We don’t have unlimited resources!

– On-host resources

• Multiple processes may want to send packets

– Network

• Available bandwidth

• Bandwidth mismatch

– Network routing & switching

• Processing delays

• Buffer space in routers & switches

– Too many queued packets → queue overflow & dropped packets

→ congestion

• Address quality of service with resource allocation & prioritization


Contributors to congestion

Bandwidth mismatch

Aggregation

Extra traffic resulting from

– Faulty hardware/firmware

– Excess retransmission from data corruption

– High-traffic apps or misbehaving hosts (e.g., malware generating floods)

Congestion leads to packet loss

10 Gbps 50 Mbps

1 Gbps 1 Gbps


Contributors to latency

• Packetization on host

– Create IP & MAC packets

• Serialization

– Send data onto the network

– One packet at a time – others are queued

• Propagation delay

• Processing delay on routers

– Input queue → output queue and packet inspection

• Queuing delay

– Wait for interface to be available


Contributors to jitter

• Packets arrive at a router or switch at the same time

– Used to lead to collision on a shared link

– Buffers store packets

– Transmitted FIFO, not all at once!

– Dequeuing creates jitter

• Dynamic re-routing

• Transport layer

– Caching out of order packets → delayed delivery

• Out-of-order packets need to be buffered

• May need to be retransmitted

– Retransmission

• Transport-layer retransmission

• End-to-end (huge!) jitter

FIFO output queue


Contributors to packet loss

• Buffer overrun on routers/switches (queue full)

• Bad transmission lines (e.g., signal degradation & noise)

• Faulty hardware


QoS Aware Networks

• A QoS-aware network will allow an application to specify its needs and

will reserve sufficient capacity in the network to meet those goals

• Admission Control

– Applications request a particular quality of service from the network

– If the request is granted, the application has the contract

• Traffic Control

– Classifies, queues, and schedules traffic within the network


Service Models for QoS

• No QoS (best effort)

– Default behavior for IP with no QoS

– No preferential treatment

– Host is not involved in specifying service quality needs

• Soft QoS (Differentiated Services)

– No explicit setup

– Identify one type of service (data flow) vs. another

– Certain classes get preferential treatment over others

• Hard QoS

– Network makes commitment to deliver the required quality of service

– Host makes a reservation

• Traffic flows are reserved


Approaches to Network Resource Allocation

• Router-based vs. Host-based

– Router-based

• Each router decides which packet to forward & when

– Host-based

• Each host observes network conditions and adjusts its behavior

• Reservation vs. feedback-based

– Reservation-based

• Host asks for specific capacity from the network

• Each router allocates buffer space for that flow to satisfy the request.

• If a router cannot grant, reservation is denied

– Feedback-based

• Hosts sends data without reserving resources

• Host gets congestion feedback from a router or detects packet loss &

adjusts its transmission rate


Congestion Control

• Congestion:

– Caused by more incoming traffic than we can route

• Three approaches

– Admission control (input)

• Prevent congestion that has been detected from getting worse

• Do not allow packets for a specific flow, new flows, new virtual circuits

– Choke packets (feedback)

– Queue management (output)


Congestion Control: Choke Packets

• If a router determines congestion, send back a “choke packet”

– Packet contains destination address identifying the port that is congested

– This is a request from the router not to send packets to that destination

(voluntary)

• IP handles congestion by dropping packets

• Explicit Congestion Notification (ECN) allows end-to-end notification

of congestion (RFC 3168)


Controlling Jitter

• Multiple queues per output port

• Queue based on priority

• Each packet is assigned to a queue

– Layer 2: 802.1p priority bits, MPLS EXP field

– Layer 3: IP ToS bits, protocol type, source/destination address

– Layer 4: port numbers

– Any combinations

output queues

Cla

ssifie

r

Schedule

r

Incoming packet Transmitted

packet


Controlling Jitter: scheduling packets

• Simplest was a single FIFO queue

• Round robin from multiple queues

– Each class of service gets a fair share – not what we want

• Priority

– If there are always packets in a high priority queue, we get

head of line blocking (simply packets holding up other packets)

• Weighted Fair Queuing (WFQ)

– Each queue gets a priority and a minimum % of link speed

output queues

Cla

ssifie

r

Schedule

r

Incoming packet Transmitted

packet


Flow Detection

• Flow: stream of related packets between hosts

– In IP, set of packets from one address:port to another address:port with

same protocol

• Soft state

– Keeping state for a flow makes resource allocation easier

(e.g., route, average packet size, packet arrival rate)

– This is called a soft state. There is usually no signaling information that

is used to create these states.

• Flow detection in routers:

– Network controls flow rate by dropping or delaying packets

– Define rules for traffic queuing, shaping, and policing

• drop TCP packets over UDP

• Discard specific UDP flows to ensure QoS for other flows


Queue Management With Flows

• Flow-based weighted fair queuing

– Per-flow queues

– Each queue is assigned a weight (volume)

• Class-based weighted fair queuing

– User-defined classes

• Protocols

• Input interfaces

• Specific source and/or destination IP addresses

• Each class is assigned

– Bandwidth

– Queue limit (# packets)

– Weight

Flow 1

Flow 2

Flow 3

Flow 4


Bandwidth Management

• Traffic Shaping

– Goal: regulate average rate of data transmission per flow

– Queue packets during surges and release later

– Example: high-bandwidth link to low-bandwidth link

• Traffic Policing

– Goal: Monitor network traffic and discard offenders

– Discard traffic that exceeds allotted bandwidth


Traffic Shaping: Leaky Bucket

• Visualization

– Bucket with a hole

– Filled up at a varying rate

– Water leaks at a constant rate

• Bucket = packet queue buffer

• If a packet comes in and bucket is full, discard packet

– Buffer overrun

• If there is nothing to transmit (bucket is empty)

– Buffer underrun

• Convert an uneven flow of packets into an even flow


Traffic Shaping: Token Bucket

• Bucket holds tokens that are generated at a certain rate

• To transmit a packet

– The bucket must hold and destroy a token(s)

• The token bucket allows a host to save up permission to send large

bursts later

– Bucket size determines maximum burstiness


Traffic Shaping: Token Bucket

Desired average rate: r bytes/second

Add a token every 1/r seconds

If # tokens > b (bucket is full), discard the token

When packet arrives (size = n bytes):

if # tokens is < n

Traffic shaping: queue (delay) the packet until there are enough

tokens

Traffic policing: drop the packet

else

transmit the packet and remove n tokens

(in implementation, the “tokens” are just one number, not a collection)


Token bucket vs. Leaky bucket

• Token bucket: may be bursty

• Leaky bucket: cannot be bursty


Quality of Service in IP

• IP was not designed with QoS controls in mind

• IP cannot (normally) take advantage of QoS controls offered by an

underlying network (LAN)

– You can’t count on a specific underlying network for end-to-end service

• Some QoS mechanisms created as add-ons


Quality of Service Problems in IP

• Bandwidth mismatches and aggregation

– Lead to congestion

• Inefficient packet transmission (not really a QoS issue)

– 59 bytes to send 1 byte in TCP/IP!

– 20 bytes TCP + 20 bytes IP + 18 bytes ethernet

• Unreliable delivery

– Software to the rescue – TCP/IP

– But leads to jitter

• Unpredictable packet delivery

– No controls on bit rate, delay, jitter

– Packets may take different routes, resulting in different performance


Inefficient Packets

• Some software generates lots of tiny packets

– Can lead to head-of-line blocking

– Nagle’s algorithm:

• Buffer new data if unacknowledged TCP/IP data outstanding

• Header/packet compression

– Link-to-link

– Header compression (RFC 3843)

– Payload compression (RFC 2393)

– Trade-off: $ delivery vs. $ compression

– Routers have to be aware


Differentiated Services (soft QoS)

• Treat some traffic as better than other

– Statistical - no guarantees

• Identify class of service & priority

– Router can use this data to make scheduling/dropping decisions

• Use on Internet (especially across ISPs) limited due to

peering agreement complexities


Differentiated Services (DiffServ)

• ToS field in IP header (bits 9…16)

– Differentiated Services Control Point (DSCP): first six bits

p p p x x x - -

Priority: 0-7

RFC 2474, 2475, December 1998

28

Differentiated Services (DS) field

DSCP value identifies class of service

Priority is used if these three bits are 0

Each router uses this

value to determine how

to prioritize the packet

– it affects “per hop”

behavior of the router

10/1/2012 © 2012 Paul Krzyzanowski

Hard QoS: Integrated Services

• IntServ: Integrated Services (RFC 1633)

– End-to-end reservation of services

• Traffic specification (TPSEC)

– Define token bucket: rate & size

• Request specification (RPSEC)

– Specify levels of assurance

• Best effort (no reservation)

• Controlled Load: soft QoS – data rates may increase or packet loss may

occur

• Guaranteed: hard QoS – tight bounds on delay

• Underlying mechanism

– Resource Reservation Protocol (RSVP)


Reservation & Delivery Protocol

• RSVP: ReSerVation Protocol

– Each unidirectional data stream is a flow

– Hosts request specific quality of service per flow

– Routers reserve resources

– RFC 2205

• Every device through which data flows must support

RSVP


Other mechanisms

• Multi-Protocol Layer Switching (MPLS)

– Make packet routing more efficient … more like layer 2

– Each packet is assigned a routing label

(based on destination & packet priority)

– Routers make queuing and forwarding decisions based on this

label

– MPLS routers can create end-to-end circuits using any protocol

– QoS parameters

• Traffic Class

• ECN (Explicit Congestion Notification)


Other mechanims

• 802.1p: control QoS at the MAC layer (layer 2)

– Eight traffic classes defined in an ethernet frame

– This is not an IP QoS mechanism

Priority Traffic

0 Background

1 Best effort

2 Excellent effort

3 Critical

4 Video <100 ms latency

5 Voice <10 ms latency

6 Internetwork control

7 Network control


Media Delivery Protocols

• Real-Time Control Protocol (RTCP)

– Provides feedback on QoS (jitter, loss, delay)

– RFC 3550

• RTP: Real-Time Transport Protocol

– Not a routing protocol

– No service guarantees

– Provides:

• Payload identification

• sequence #

• time stamp

• RTP/RTCP do not provide QoS controls


Virtual Circuit Networks

• Avoid having to choose a route for every packet

• Route established when a connection is set up

• All traffic from source to destination flows on the same route

• Packets only need to contain a virtual circuit number, not destination

address

• IP networking does not work this way

– TCP/IP’s virtual circuit is modeled entirely in the TCP driver in the OS

– Routers are unaware of the presence of a virtual circuit


ATM: Asynchronous Transfer Mode

• Late 1980’s

• Goal: Merge voice & data networking

low but constant

bandwidth

high but bursty

bandwidth


ATM

• Traditional voice networking

– Circuit switching

• Too costly

• Poor use of resource

• Does not lend to multicasting

• ATM: wide area networking with QoS controls

– Based on

fixed-size packets over virtual circuits

– Virtual circuit must be established before any traffic is sent

• Route is established and all participating switches commit to QoS

– Fixed-size cells provide for

predictive scheduling

– Large cells will not hold up smaller ones

– Rapid cell switching


ATM

• ATM cell

– 53-byte cell: 48-byte data, 5-byte header

• Sender specifies traffic type on circuit setup:

CBR Constant bit rate bandwidth Uncompressed video,

voice

VBR Variable bit rate Avg, peak

bandwidth

Compressed video,

voice

ABR Available bit rate Minimum

bandwidth

web access

UBR Unspecified bit rate – ftp


ATM

• Small cells lots of interrupts

– 622 Mbps link: 12 million interrupts per second

• ATM hardware supports an

ATM Adaptation Layer (AAL)

– Converts cells to variable-sized (larger) packets:

AAL 1: for CBR

AAL 2: for VBR

AAL 3/4: ABR data

AAL 5: ABR data, simplified

AAL 6: MPEG-2 video

• IP traffic: transmitted as data using AAL5


The End


Documents

Distributed Systems 3. Network Quality of Service (QoS)pxk/417/notes/content/03-qos-slides.pdf · Quality of Service Problems in IP • Bandwidth mismatches and aggregation –Lead