Lecture 9: Ordered Multicasting

Lecture 9: Ordered Multicasting

TVS: Section 6.2

(CDK: Section 12.4)

Apr 19, 2023 COMP28112 Lecture 9 2

Contents

• What is an ordered multicast?

• Why is it useful?

• How can it be done?– Using logical timestamps

Apr 19, 2023 COMP28112 Lecture 9 3

Multicast

• 1 process sends a message to a group of other processes

• More efficient than looping in the application sending a message to each

• May be able to guarantee that either all recipients get the message or none does.

Apr 19, 2023 COMP28112 Lecture 9 4

Example: duplicate bank records

• For speed of response to queries, and improved reliability, suppose a bank keeps its database replicated in geographically distant locations (e.g. London & Edinburgh)

• Each update request is sent to both (as a multicast)

• But this leads to an ordering constraint

Apr 19, 2023 COMP28112 Lecture 9 5

Fig 6.11 from TvS

Apr 19, 2023 COMP28112 Lecture 9 6

The Updates

• Suppose Update 1 credits £100 to an account containing £100

• Suppose Update 2 adds 1% interest to the account.

• If not done in the same order, can get balances of £201 and £202 in the two databases.

Apr 19, 2023 COMP28112 Lecture 9 7

Totally-Ordered multicast

• To get the same answers, we need totally-ordered multicast, i.e. all messages are delivered in the same order to each receiver.

• Can be done with Lamport’s logical clocks.• Consider a group of processes multicasting

to each other. • Each message is timestamped with the

logical time of the sender.

Apr 19, 2023 COMP28112 Lecture 9 8

T-O Multicast (2)• Assume multicast also goes to the sender• Assume messages from the same sender are received

by any 1 receiver in the order they were sent• Assume that no messages are lost• When message is received, put it in a local queue

ordered by its timestamp.• The receiver multicasts an ACK to all processes.

(Note ACK is not queued!)• A process can “act on” a queued message when it is

at the head of the queue, and has been ACK’d by every other process.

• Each process has the same contents in the queue …!

Apr 19, 2023 COMP28112 Lecture 9 9

A

B

C

A will have to process messages in a different order(message delivery can be anything as long as it is the

same for all processes)

Apr 19, 2023 COMP28112 Lecture 9 10

Implications

• Note: the need to receive ACKs from all processes reduces advantage of replication! A crashed process will cause everything to stop ….

• Keeping replicas consistent by executing same operations in the same order is a general technique (state machine replication)

Apr 19, 2023 COMP28112 Lecture 9 11

Causally-Ordered Multicasting

• Weaker than totally-ordered because unrelated messages can be processed in different orders on different machines.

• Useful on, e.g., a bulletin board – may want messages delivered in the correct order where they are related, but not to tie down ordering between messages on unrelated topics

Apr 19, 2023 COMP28112 Lecture 9 12

C-O multicasting

• Now we use Vector clocks – but only noticing sends as events ….

• Suppose Pj receives message m from Pi with timestamp ts(m)

• Delay delivery to application until:– ts(m)[i] = VCj[i] + 1– ts(m)[k] <= VCj[k] for all other k

Apr 19, 2023 COMP28112 Lecture 9 13

C-O multicasting (2)

• The 1st condition means that this is the next message Pj was expecting from Pi

• The 2nd means that Pj has seen all the messages seen by Pi when it sent message m.

Apr 19, 2023 COMP28112 Lecture 9 14

Example

P0

P1

P2

(1,0,0)

(1,1,0)

(1,1,0)

(0,0,0) (1,0,0) (1,1,0)

m m m*

m*delay

Apr 19, 2023 COMP28112 Lecture 9 15

Explanation

• m* from P1 arrives at P2 before m from P0

• But m* was sent after m arrived at P1

• Therefore delivery of m* in P2 is delayed until after m has been delivered.

Apr 19, 2023 COMP28112 Lecture 9 16

End-to-end argument

• Some systems have provided T-O and C-O multicasting (e.g. ISIS)

• Some debate about whether this is a Good Thing or not

• Two main problems:– Not all causality is real (e.g. 1 sender may send

unrelated messages, but the system will believe the order matters)

Apr 19, 2023 COMP28112 Lecture 9 17

E-to-E argument (cont.)

– Not all causality may be captured. E.g. users of the bulletin board might discuss issues offline and then post related messages without reading logically preceding ones first.

• A particular application will have its own approach, but we may or may not want to build it on top of an infrastructure which assumes the answers.

Apr 19, 2023 COMP28112 Lecture 9 18

Exercisea b c d f

g i j k l m

n o p q r s

Use (i) Lamport Clocks; (ii) Vector Clocks to order the events