Primitives for Achieving Reliability

Primitives for Achieving Reliability

3035/GZ01 Networked SystemsKyle Jamieson

Lecture 7

Department of Computer ScienceUniversity College London

Achieving reliability: the story so far

1. Apply forward error correction (FEC) at physical layer2. Apply error detection at higher layers

Choosing the amount of FEC to apply

• Adding parity bits:1. Increases number of bit

errors receiver can correct2. Increases overhead3. Reduces the bit error rate (BER) after error correction

• Question: What BER after error correction should we target?

Sender ReceiverFECS Network FECR

data bits parity bits

k bits n−k bits

How much FEC do we need to apply?

Bit error rate (BER)

n = 104 (1250 bytes) n = 320 (40 bytes)

FLR

How much FEC? (notes)• What do we really care about? Keep frame loss rate (FLR) low: ≈10-3

• Suppose after applying FEC, probability of a bit error is BER• Pr[Frame of length n bits delivered] = (1 − BER)n

– Assumption: Independence between bit errors (worst case)– Frame loss rate (FLR): 1 − Pr[Frame delivered]

• Add enough FEC to keep FLR below rise, but no more (wastes bits on the channel)

• Where is the rise, for a given packet size?• For small x, binomial expansion of (1+x)n = 1 + nx + O(x2) ≈ 1 + nx• FLR = 1 − (1 − BER)n ≈ n × (BER), therefore goal is to keep n × (BER) < 10-3

• Therefore, for data packet of 1250 bytes (104 bits), need BER after FEC of 10-7

• No matter how many parity bits we add, the network could flip them and data bits, causing errors!– Error detection (CRC) then discards these frames

How to ensure that links are reliable?

Some errors are too severe to be corrected

Sender ReceiverFECS Network FECR

Reliable delivery: Plan

• Goal: ensure every packet is received exactly once

• Exactly-once delivery• Pipelining for performance• Handling retransmissions intelligently• Congestion in the Internet

• Culmination: the Transmission Control Protocol

Today

• Three protocols that achieve reliability:

1. Designing reliable transfer from first principles: the Stop and Wait protocol

2. Exploiting pipelining in the network: Go Back N3. Smarter error recovery: Selective Repeat

Reliable transfer over an unreliable channel

• Task: Design sender and receiver sides of a reliable data transfer (rdt) protocol, using unreliable data transfer (udt) protocol

• Recurs at many different layers: transport/network, link/physical

Reliable data transfer protocol

(sender side)

Reliable data transfer protocol

(receiver side)

rdt_send(data)

udt_send(pkt) rdt_recv(pkt)

deliver_data(data)

Unreliable channel

Approach

• Let’s derive a protocol that achieves reliable transfer from first principles

• Goal: exactly once, in-order delivery of every packet• Unidirectional at first; same principles can generalize to

bidirectional data transfer• Starting assumptions:1. Channel can not introduce bit errors into packets2. Channel can not fail to deliver packets3. Channel can not delay packets4. Channel can not reorder packets• Gradually relax these assumptions, one by one

Reliable data transfer (rdt) v1.0

• Independent state machines at sender, receiver• No need for feedback (underlying channel is reliable)• No flow control in this protocol

Sender state machine:

Receiver state machine:

Assumptions

1. Channel can introduce bit errors into packets– Channel (sender to receiver) can introduce bit errors– Channel (receiver to sender) can not introduce bit errors

2. Channel can not fail to deliver packets3. Channel can not delay packets4. Channel can not reorder packets

Idea: Receiver requests re-sends

• Three fundamental mechanisms:1. Error detection: typically a packet checksum (like the CRC)2. Acknowledgements: small control frame (ACK) transmitted

from the receiver back to the sender• Positive ACKs acknowledge correct receipt• Negative ACKs inform sender of incorrect receipt

3. Timeouts: sender waits a reasonable amount of time, then retransmits the frame

• Protocols using these techniques are called automatic repeat request (ARQ) protocols

• Surprisingly challenging to apply correctly!

A loaf of bread,A loaf of bread,OK.OK.

Sorry?Sorry?A dozen eggs,A dozen eggs,

Reliable data transfer under bit errors

rdt v2.0 receiver state machine:

rdt v2.0 sender state machine:

Reliable data transfer v2.0 analysis

• Stop-and-wait protocol: Sender doesn’t send more data until sure original data received– Performance depends on sender-

receiver delay, and throughput

• Correctness: Two cases1. Data arrives okay

• ACK comes back immediately• Sender sends next packet

2. Data arrives corrupted• NACK comes back

immediately• Sender retransmits

corrupted packet• Exactly once, in-order delivery

Data A

ACK

Data B

NACK

IDLEACK wait

IDLEACK wait

IDLEData B

ACK

ACK wait

Sender Receiver

Assumptions

1. Channel can introduce bit errors into packets– Channel (sender to receiver) can introduce bit errors– Channel (receiver to sender) can introduce bit errors

2. Channel can not fail to deliver packets3. Channel can not delay packets arbitrarily4. Channel can not reorder packets

Human approach to feedback errors

• One possibility: Apply ARQ in reverse– Request retransmits of corrupted feedback

• If corrupt, receiver doesn’t know if forward-link packets are data or feedback retransmit requests

• Clearly heading down a difficult path!

A loaf of bread,A loaf of bread,

??????

Idea: Add checksums to feedback

rdt v2.0 (with checksums) receiver state machine:

rdt v2.0 (with checksums) sender state machine:

Problem for rdt v2.0: duplicates

• Three cases:1. Data okay, ACK okay2. Data corrupted

• NACK comes back immediately

• Sender retransmits corrupted data packet

3. Data okay, ACK corrupted• ACK comes back immediately• Sender retransmits an

identical data packet

• At least once, in-order delivery

Data B

NACK

Data B

ACK

IDLEACK wait

IDLEACK wait

IDLEData B

ACK

ACK wait

Sender Receiver

Assumptions


2. Channel can not fail to deliver packets3. Channel can not delay packets arbitrarily4. Channel can not reorder packets5. Sender or channel can duplicate packets

From at least once to exactly once

• Idea: add sequence numbers to data packets• Sequence number allows receiver to suppress duplicates

rdt v2.1 sender state machine:

From at least once to exactly once

• Two states at receiver: one for expecting seqno=1; the other for expecting seqno=0

rdt v2.1 receiver state machine:

rdt v2.1 error-free operation

• Sender: send data, wait for reply• Receiver: deliver data, send ACK• Sender: send next data, wait• Receiver: deliver data, send ACK• Sender: transition to Idle State 0

Data, seqno=0AW0I0 DW 0

Sender Receiver

DW 1ACKI1

DW 0ACK

Data, seqno=1AW1

I0



rdt v2.1: bit errors on the forward link

• Sender: send data, wait for reply• Receiver: checksum fails; NACK• Sender: resend seqno=0 data• Receiver: deliver data; send ACK• Sender: transition to Idle State 1

I0 DW 0Sender Receiver

I1ACK DW 1

Data, seqno=0AW0

NACK DW 0Data, seqno=0AW0



rdt v2.1: bit errors on the reverse link

• Sender: send data, wait for reply• Receiver: deliver data; send ACK• Sender: resend seqno=0 data• Receiver: dup. data; resend ACK• Sender: transition to Idle State 1


I1ACK DW 1

Data, seqno=0AW0

Data, seqno=0AW0ACK DW 1



rdt v2.2: Getting rid of NACKs

• rdt v2.1 used different packets in feedback direction (ACK, NACK)

• Instead, we can add sequence numbers to ACKs– Sequence number in ACK corresponds to data it acknowledges


I1ACK 0 DW 1

Data, seqno=0AW0

ACK 1 DW 0Data, seqno=0AW0



rdt v2.1: Dropped packets

• Sender: transmit data, wait for a reply from the receiver

• Result: Deadlock


Data, seqno=0AW0



Assumptions


2. Channel can fail to deliver packets3. Channel can delay packets arbitrarily4. Channel can not reorder packets5. Sender or channel can duplicate packets

Retransmission timer breaks deadlockrdt v3.0 sender state machine:

Timer start

Timer stop

rdt v3.0 receiverrdt v3.0 receiver state machine:

rdt v3.0: recovering lost packets

• Sender: send, start timer, wait/reply• Receiver: deliver; send ACK 0 (lost)• Sender: timeout, resend, start timer• Receiver: dup. data; resend ACK 0• Sender: stop timer, go to Idle State 1




I1ACK 0 DW 1

Data, seqno=0AW0ACK 0 DW 1

Data, seqno=0AW0

Timeout

rdt v3.0: delays in the network• Sender: send, start timer, wait/reply• Receiver: deliver; send ACK (delayed)• Sender: timeout, resend, start timer• Sender: stop timer, go to Idle State 1• Receiver: dup. data, resend ACK 0• Sender: recv. dup. ACK 0, do nothing




I1 ACK 0 DW 1

Data, seqno=0AW0

ACK 0DW 1

Data, seqno=0AW0

Timeout

I1

Stop-and-wait performance

sender

receiver

RTT

First bit transmitted, t = 0Last bit transmitted, t = L/R

First bit arrivesLast bit arrives, send ACK

ACK arrives, send next packet, t = RTT + L/R

Data packet size L bits, link bitrate R bits/second

Forward link utilization:

Performance of stop-and-wait protocols

• Packet size L, link bitrate R; utilization

• Internet e.g.: 8000 bit packet; 1 Gbit/s link, 30 ms RTT:

• WiFi e.g.: 8000 bit packet; 54 Mbit/s link, 100 ns RTT(not includingother unrelatedoverheads)

When packet time << RTT, stop-and-wait underperforms.When packet time << RTT, stop-and-wait underperforms.

Idea: Pipelined protocols• Pipelining: sender allows multiple unacknowledged

packets “in-flight” from sender to receiver• Range of sequence numbers must be increased• Buffering at sender and/or receiver

• Two generic forms of pipelined protocols: Go-Back-N, Selective Repeat

Increasing utilization with pipelining

First bit sent, t = 0sender receiver

RTT

Last bit sent, t = L / R

last bit of 1st packet, send ACK

ACK arrives, send next packet, t = RTT + L / R

last bit of 2nd packet, send ACKlast bit of 3rd packet, send ACK

The bandwidth-delay product

• How many packets N should we allow “in flight” (unacknowledged) in order to get maximum link utilization?

sender receiver

RTT

Number of bits“in flight”

Delay × Bandwidth product

Today




Assumptions


2. Channel can fail to deliver packets3. Channel can delay packets arbitrarily4. Channel can reorder packets in forward direction5. Sender or channel can duplicate packets

The Go-Back-N (GBN) protocol

• Sender: send without waiting for an acknowledgement– Up to N unacked packets “in flight” at any time– Why limit to N? Flow control at receiver, congestion

control in the network– Timer for oldest unacknowledged packet– Timer expire: retransmit all unacknowledged packets

(sender can “go back” up to N packets)

• Receiver: sends cumulative acknowledgement: acknowledge receipt of all packets up to and including packet n: ACK(n)

GBN: at the sender

rdt_send(data):if nextseqnum < send_base + N:

sndpkt[nextseqnum] = make_pkt

send data with nextseqnumnextseqnum++

else:(refuse data)

timeout:send(sndpkt[base])send(sndpkt[base+1])…send(sndpkt[nextseqnum−1])

rdt_recv(ackpkt):send_base = ackpkt.seqno +

1

(Timer code not shown)

• GBN: A type of Sliding Window Protocol

GBN: at the receiver• Maintain expectedseqnum variable: sequence number of the next

in-order packet– send_base ≤ expectedseqnum < nextseqnum

• Incoming data packet with seqno=n– n = expectedseqnum: send ACK(n), increment expectedseqnum– n <> expectedseqnum: discard packet, send ACK(expectedseqnum − 1)

• Nothing more, because in the event of loss, the sender will go back to expectedseqnum!

• Could buffer correctly-received out-of-order packets, but the sender will transmit them later anyway

GBN (N = 4) in operation

Sender Receiver

Send packet 2

Send packet 0Receive 0; send ACK 0Send packet 1Receive 1; send ACK 1

Send packet 3(wait) Receive 3; discard;

send ACK 1

Receive ACK 0; send 4 Receive 4; discard;send ACK 1

Resend packet 3 Receive 3; send ACK 3

Timeout; resend 2Receive 2; send ACK 2

Receive ACK 1; send 5(wait) Receive 5; discard;

send ACK 1

Resend packet 4 Receive 4; send ACK 4Resend packet 5

Receive 5; send ACK 5

The role of the retransmission timer

• Keeps track of time at sender since the oldest unacknowledged packet was sent – This is the packet at the left edge of the sender’s window

• Issue: Choosing a reasonable timer value– Compare to RTT of one packet:– Too small causes unnecessary retransmissions– Too big wastes time

• Later: we will see how TCP solves this problem robustly

Today




Selective Repeat: Introduction

• Go Back N– Allows sender to fill pipeline with packets, for high channel utilization– Receiver sends cumulative acknowledgements ACK(n) acknowledging

packet n and all those before– Large pipeline means that a single packet error results in many

duplicate packet transmissions

• Selective Repeat (SR)– Main idea: sender retransmits only packets that don’t reach the

receiver correctly– Receiver individually acknowledges each packet– One retransmission timer per packet

• GBN and SR are both examples of sliding window protocols

Finite sequence number space

• Used in practice for both GBN and SR

• Sequence numbers– k-bit sequence number– Result: a sequence number space in the range [0, 2k−1]– All arithmetic is modulo 2k: wrap-around from end to beginning

0 2k−1

02k−1 1 2

Selective Repeat: Sequence numbers

• Window size N limits number of unacked packets in pipeline• send_base: lowest unacked packet• nextseqnum: last sent packet sequence number, plus 1• rcv_base: last frame delivered, plus 1

AlreadyACKed

Sent, not yet ACKed

Not usable

Usable, not yet sent

Out of order but ACKed

Expected, not yet received

Not usable

Acceptable

Sender’s view:

Receiver’s view:

Window size N

Window size N

send_base nextseqnum

rcv_base

SR sender: Data received from above

• Sender checks nextseqnum:– If in sender window, transmit, increment nextseqnum by one– If not in sender window, refuse data from above

AlreadyACKed

Sent, not yet ACKed

Not usable




Not usable

Acceptable

Sender’s view:

Receiver’s view:

Window size N

Window size N


rcv_base

SR sender: Timeout event

• Recall: each packet has its own retransmission timer• Only packets’ timers in send window will be active• Sender retransmits packet then restarts that packet’s timer

AlreadyACKed

Sent, not yet ACKed

Not usable




Not usable

Acceptable

Sender’s view:

Receiver’s view:

Window size N

Window size N


rcv_base

SR receiver: packet reception

• Correct packet reception with received seqno r. Three cases:1. seqno r in receiver window: deliver data, send ACK, advance window2. seqno r in [rcv_base − N, rcv_base − 1]: resend ACK3. Otherwise: ignore the packet

AlreadyACKed

Sent, not yet ACKed

Not usable




Not usable

Acceptable

Sender’s view:

Receiver’s view:

Window size N


rcv_base

[rcv_base − N, rcv_base − 1][rcv_base, rcv_base + N - 1]

SR sender: ACK received

1. Sender marks packet as already ACKed, stops retransmit timer2. Sender compares send_base and ACK sequence number:

– If ACK is not for send_base, keep window where it is– If ACK is for send_base, advance send_base and send window by one

AlreadyACKed

Sent, not yet ACKed

Not usable




Not usable

Acceptable

Sender’s view:

Receiver’s view:

Window size N

Window size N


rcv_base

SR (N = 4) in operationSender Receiver

Send packet 0[0 1 2 3] 4 5 6 7 8 9

Send packet 1[0 1 2 3] 4 5 6 7 8 9

Send packet 2[0 1 2 3] 4 5 6 7 8 9

Send packet 3[0 1 2 3] 4 5 6 7 8 9Recv ACK 0, send 40 [1 2 3 4] 5 6 7 8 9Recv ACK 1, send 50 1 [2 3 4 5] 6 7 8 9

Timeout 2, resend 20 1 [2 3 4 5] 6 7 8 9Recv ACK 3, send −0 1 [2 3 4 5] 6 7 8 9

Recv 0, deliver, ACK 00 [1 2 3 4] 5 6 7 8 9Recv 1, deliver, ACK 10 1 [2 3 4 5] 6 7 8 9

Recv 3, buffer, ACK 30 1 [2 3 4 5] 6 7 8 9Recv 4, buffer, ACK 40 1 [2 3 4 5] 6 7 8 9Recv 5, buffer, ACK 50 1 [2 3 4 5] 6 7 8 9Recv 2, deliver 2−5,ACK 20 1 2 3 4 5 [6 7 8 9]

Window size vs sequence number spaceN = 3, k=2 (4 sequence numbers)

Sender Receiver

Recv 0, deliver, ACK 00 [1 2 3] 0 1 2Recv 1, deliver, ACK 10 1 [2 3 0] 1 2

Recv 0 ➠ suppress duplicate

Send 0[0 1 2] 3 0 1 2

Send 1[0 1 2] 3 0 1 2

Send 2[0 1 2] 3 0 1 2

Timeout, resend 0

Recv 0, deliver, ACK 00 [1 2 3] 0 1 2Recv 1, deliver, ACK 10 1 [2 3 0] 1 2

Recv 0 ➠ deliver up

Send 0[0 1 2] 3 0 1 2

Send 1[0 1 2] 3 0 1 2

Send 2[0 1 2] 3 0 1 2

Recv ACK 0, send 30 [1 2 3] 0 1 2

Recv ACK 1, send 00 1 [2 3 0] 1 2 3

Window size vs sequence number space

• N > 2k−1: window too large– Right edge of receiver’s window can wrap past left edge of

sender’s window• N ≤ 2k−1: no overlap

AlreadyACKed

Sent, not yet ACKed

Not usable




Not usable

Acceptable

Sender’s view:

Receiver’s view:

Window size N

Window size N

send_base

rcv_base

Next time

Introduction to InternetworkingPre-Reading: P & D, Sections 3.2 and 4.1

Documents

Primitives for Achieving Reliability