Clock Sync

7/31/2019 Clock Sync

1/59

Clock Synchronization

Topics in Multiprocessing 2004

Presented by: Alon Stern


2/59

Outline Introduction

Event Ordering Partial Ordering Total Ordering


Model Overview The Basic Theorem

External Synchronization


3/59

Importance of clock

synchronization Communication is not enough. Cooperation

is needed synchronization.

Distributed synchronization is needed for:

Transactions (bank account via ATM)

Access to shared resources (networkprinter, databases)

Ordering of events (network gameswhere players have different ping times)


4/59

Clock Synchronization Problem

GoalObtain the tightest achievable estimateof the readings of a remote clock in any givenexecution of the system.


5/59







6/59

Event Ordering The concept of one event happening

before another in a distributed system

is examined.

Defining a partial ordering of events.

Defining total ordering of events.


7/59

Defining Our System The system is composed of a collection of

processes.

Each process consists of a sequence ofevents.

The events of a process form a sequence,where aoccurs before bin this sequence ifa

happened before b. Sending or receiving a message is an event in

a process.


8/59

Causality Event ordering linked with the concept

of causality:

Saying that event ahappened before eventbis the same as saying that it is possiblefor event ato causally effect event b.

If events aand bhappen on processes thatdo not exchange any data, their exactordering is not important.


9/59

Partial Ordering In a distributed system it is sometimes

impossible to say that one of two

events occurred first.

The relationhappened before is

therefore only a partial ordering of theevents in the system.


10/59

The Happened Before

Relation The happened-beforerelation in a distributed

system:

1. Ifaand bare two events in the same process,

and acomes before b, then a b.2. Ifais the sending of a message, and bis the

receipt of that message, then a b.

3. Ifa band b cthen a c.

Two distinct events aand bare said to be concurrentifa band b a.


11/59

q2 q4(rule #1)

q4 r3(rule #2)

r3 r4(rule #1)

Example p1 q2(rule #2)

p1 r4(rule #3)

p3,q3are concurrent(neither can causally effectthe other)


12/59

Logical Clocks Problem: How do we maintain a global

view on the systems behavior?

Solution: Use logical clocks on eachprocess.


13/59

Logical Clocks Define a Logical clockCifor each

process Pito be a function which

assigns a number Ci(a)to any event ain that process.

Monotonically increasing counter.

No relation with physical time: Just away of assigning a number to an event.


14/59

Logical Clocks The clock condition:

Ifa b then c(a) < c(b).

The clock condition is satisfied if:

Ifaand bare events in process Pi, and acomes before b, then Ci(a)


15/59

Logical Clocks Note: We can not expect

the converse condition.

Reason: That would

imply that any twoconcurrent events mustoccur at the same time.

Example: p2and p3are

both concurrent with q3.they must occur at thesame time as q3.C(p2)=C(q3)=C(p3).But, p2 p3.


16/59

Logical Clocks in Space-Time

Diagram There must be a

tick line between

any two events ona process line.

Every message line

must cross a tickline.


17/59

Satisfying the Clock Condition Each process Pi maintains a localcounter

Ci and adjusts this counter according to thefollowing rules:

1. Ci is incremented before each event. Ci=Ci+1.

2. If event ais the sending of a message m, thenthe message mcontains a timestamp Tm=Ci(a).

3. Whenever a message mis received by a processPj, Pj adjusts its local counter Cj:Cjmax{Cj+1,Tm+1}.


18/59

Satisfying the Clock Conditionpublicclass LamportClock {

int c;

public LamportClock() {

c = 1;

}

publicint getValue() {

return c;

}

publicvoid internalAction() {

c = c + 1;

//action execution

}

publicvoid sendAction() {

c = c + 1;

msg.header.timeStamp = c;

//send message

}

publicvoid receiveAction(int src, int sentValue) {

c = max(c, sentValue) + 1;

}


19/59

Total Ordering With Logical

ClocksProblem: it can still occur that two events

happen at the same time.

Solution: Attach a process number to anevent:

Pi timestamps event ewith {Ci(e),i}.

The relation defines a total ordering.abif and only if either:

Ci(a)


20/59

Example of Total Ordering Events (1,0) and

(1,1) were

considered to beconcurrent.

Now we can say

(1,0) (1,1)

(1,0)(1,1)

(2,1)

(3,1)

(4,1)

(2,0)

(5,0)

(6,0)

(7,0)

(5,1)

(6,1)

(7,1)

(1,2)

(2,2)

(4,2)


21/59

Anomalous Behavior Presence of outside interactions.

Carrying a diskette from one machine to

another. Dictating file changes over the phone.

Two ways to avoid such anomalous:

Explicitly introduce into the system the necessaryinformation about the ordering.

Strong clock condition: for any event a,bif abthen c(a)


22/59







23/59

Clock Synchronization Goal: Ensure that physically dispersed

processors will acquire a common notion of

time using: Local physical clocks (whose rates may vary).

Message exchange over a communication network(with uncertain transmission delay).

Question: Can we set all the clocks in adistributed system to have the same time?


24/59

Physical Clocks Every computer contains a physical

clock.

Aclockis an electronic device, whichcounts oscillations in a crystal at aparticular frequency.

Example: Oscillator frequency is 10MHz

increment clock by one second after counting

10M cycles.


25/59

Physical Clocks in Distributed

System Can we synchronize all the clocks to some known high

degree of accuracy?

Well, there are some problems

Its difficult to synchronize the clocks. Crystal based clocks tend to drift over time.

Physical variations in the crystals, temperaturevariations, aging, etc.

Drift is small, but adds up over time.

For quartz crystal time, maximum drift rate is 50 secondsevery 106 seconds=11.6days.

Best atomic clocks have drift rate of one second in 1013seconds = 300,000 years.


26/59

Real Time Exact time was computed by astronomers. Take

noon for two days, divide by 24*60*60 meansolar second.

At present, the real time is taken as the average ofsome 50 cesium-clocks around the world.

Introduces a leap second from time to time tocompensate that days are getting longer. This time

called Universal Coordinated Time (UTC).

UTC is broadcast through short wave radio andsatellite.


27/59

Does This Solve All Our

Problems?

Dont we now have some global timing

mechanism?

Not every machine has UTC receiver.

Suppose we have a distributed system with a UTC

receiver somewhere in it, we still have to distributeits time to each machine.


28/59

Clock Synchronization Variants External Synchronization

Get tight estimate on

some source clock whichshows real time.

Internal Synchronization

Real time is notavailable. Keep all clocksas close as possible.


29/59

Main Difficulties Synchronization tends to deteriorate over time and

space.

When the local clocks are not drift free, the

synchronization loosens timing informationshould be refreshed periodically.

Communication information to distant processors timing uncertainty due to unpredictable

message delays. Usually there exist some a-priory bounds on these

uncertainties.


30/59

Model Overview Local Clock The hardware

clock of the processor.

Network Connectsprocessors to their neighbors.

Send Module When andwhere to send a message.

Clock SynchronizationAlgorithm (CSA) Computesa correctness and an errormargin.


31/59

Local Clock Hardware clock, Physical clock.

Encodes at all states sof processor v a realnumber called local timedenoted LT(s).

Drift free or bounded drift clock. Drift freeRun at the rate of real time, but does

not necessary shows real time.

Bounded driftKnown lower and upper bounds onthe rate of its progress relative to real time,denoted +, -


32/59

Drift Bounds - ExampleTypical workstation - clock accuracy of50

parts per million (ppm).

If it shows that 1000000 time unitshave passed between event pand q,we are guaranteed that RT(p)-RT(q)

[999950,1000050].


33/59

Network Connects processors to their neighbors.

May lose, duplicate and deliver

messages out of order. The real time delay of each message m

is within some known bounds

0L(m)H(m). Bounds may vary from link to link and

from message to message.


34/59

Clock Synchronization Module

(CSA) Uses:

Local time

Received Messages

Specification of the system

To compute Correctness term D

Error margin

The logical time is (T,e)where T=LT+D. sticks extra bytes to an outgoing message and

strips the corresponding bytes of incomingmessage.


35/59

Before we start

Lets take a simpleexample:

Suppose u,vare drift freeclocks.

When the clock of node ushows 12, ucan safety

deduce that the clock at vshows something in[18,20].


36/59

System Execution A sequence of events, where each event p,

has: Real time of occurrence RT(p).

Local time of occurrence LT(p).

We shall think of an execution as a graph,whose nodes are the set of events, with anedge(p,q)if either: qis the receive event whose send event is p.

p,qoccur at the same processor and qis the firstevent following p.


37/59

Execution as a Graph Nodes:

{p,q,r,s,t}

Edges:{(p,q),(q,s),(r,s),(s,t)}

r

RT(r)

LT(r)

s

RT(s)

LT(s)

pRT(p)

LT(p)

q

RT(q)

LT(q)

t

RT(t)

LT(t)


38/59

View Given an execution, its view is obtained by omitting

the real time mapping.

View graphGb

V- The set of points

E- Two anti-symmetrical directed edges joining each pairof adjacent points.

An event of an execution or a view is called apoint.

The local view of a point is the view of all pointswhich happened before the point.


39/59

View Graph - Exampler

LT(r)

s

LT(s)

pLT(p)

q

LT(q)

t

LT(t)

Time


40/59


41/59

Real Time Specifications Bounds on the difference of real times

between pairs of events.

Use the following real time specifications: Message transmit bounds.

Clock drift bounds.

The bounds will be modeled uniformly

nodifference between the bounds oncommunication delay and the bounds ondrifting clocks.


42/59

Bounds MappingA function Bthat maps every pair p,qof

adjacent points to a number B(p,q)>-.

The interpretation of bounds mapping isonly upper bounds.

For every pair of adjacent points p,qwe

have that RT(p)-RT(q)B(p,q).


43/59

Bounds Mapping - Formulation State the bounds mapping using quantities

available to the processor:

L(m), H(m),+,-,virt_del

For a message mwith send point p, receivepoint qand delay in the range [L(m),H(m)]:B(q,p)=H(m)andB(p,q)=(-L(m)).

For two adjacent points p,qat a processorhaving drift bounds+,-

B(q,p)=virt_del(q,p)/-

B(p,q)=virt_del(p,q)/+


44/59

Bounds Mapping - ExampleGiven:

Processors clock may drift up to 100ppm.

Message delivery time is completely arbitrary

(between 0to ).The bounds mapping:

Ifpis the send event and qis the receive event ofthat message B(p,q)=0.

Ifpoccurs before qat the same processor,B(p,q)=(LT(p)-LT(q))/1.0001and

B(q,p)=(LT(q)-LT(p))/0.9999.

For all other cases B(p,q)=.


45/59

Relative Offset Ties together actual and virtual delays. Absolute offset of a point p:d(p)=RT(p)-LT(p).

The offset ofprelative to a point q:d(p,q)=d(p)-d(q).

Lemma: For any two points p,q of a pattern,d(p,q)=-d(q,p).

Lemma: For any three points p,q,r of apattern, d(p,q)=d(p,r)+d(r,q).


46/59

Pause What information do we have till now?

A view graph, containing the local time LTofeach event.

bounds mapping in terms of quantitiesavailable to the processor L(m), H(m),+,-,virt_del

Offset definition which ties together actual

and virtual delay. Using this information we can now define

a synchronization graph.


47/59

Synchronization GraphGiven: A viewb=(V,E). Bounds mapping B.

The synchronization graphGbB=(V,E,w)is: The view graph (V,E)where for each (p,q)E,

w(p,q)=B(p,q)-virt_del(p,q).

Lemma: For every pair (p,q)E, d(p,q)w(p,q). Explanation:act_del(p,q)-virt_del(p,q)B(p,q)-virt_del(p,q).


48/59

Synchronization Graph -

Example Nodes, points.

Two anti-symmetrical

directed edges joiningeach pair of adjacentpoints.

Weighted edges.


49/59

Synchronization Distance Strategy - Reduce synchronization problem

to distance computations in the

synchronization graph. synchronization distance d(p,q): Length

of the shortest directed path from pto qinthe synchronization graph.

Lemma: IfBis satisfied, then for every twopoints p,qGbB, d(p,q)d(p,q)


50/59

Synchronization Distance -

Example

d(p,q) = 10d(q,p) = -8


51/59

The Main Theorem Let p,qbe to points in a view b. In

any execution:

RT(p)-RT(q)[VD(p,q)-d(q,p),VD(p,q)+d(p,q)]

Furthermore, there exist executions

which meets the bounds. Interpretation: The synchronization

distances determine precisely theuncertainty bounds.


52/59

Main Theorem - Example Given

p is an event at node u

q,r are events at node v

The clocks are drift free The delay of the message from p to q is

in the range [0,2].

pLT=7

q

LT=15

rLT=17

Node u Node v

w=0w=0

drift free w(q,r)=w(r,q)=0

w=(-6)

B(q,p)=2 w(q,p)=2-(15-7)=(-6)

w=8

B(p,q)=0 w(p,q)=0-(7-15)=8

RT(r)-RT(p)[(17-7)-8, (17-7)+(-6)]

RT(r)-RT(p) [2,4]

Solution:

d(r,p)=(-6), d(p,r)=8


53/59

Main Theorem - Example Cont RT(p)=5, RT(q)=7, RT(r)=9

Real time specifications are satisfied.

RT(r)-RT(p)=4

pLT=7

q

LT=15

rLT=17

RT(p)=5, RT(q)=5, RT(r)=7

Real time specifications are satisfied.

RT(r)-RT(p)=2

RT(p)=5, RT(q)=10, RT(r)=12

Real time specifications are not satisfied.

X RT(r)-RT(p)=7


54/59

External Synchronization Each processor has a local clock.

Each clock has drift bounds.

Each message has latency bounds.

One of the clocks, Called source, shows real time.Task Processors should provide the best estimate of the

source clock.

An estimate is an interval which is guaranteed to containthe source time.


55/59

Synchronization Algorithms Efficient - If its complexity is

independent of the execution length.

OptimalIf it always outputs thetightest bounds.

GeneralIf it works for all possible

systems.


56/59

Space Lower BoundAny optimal and general algorithm for

external synchronization has high spacecomplexity is not efficient.

There is no algorithm which is general,efficient and optimal.

One of the three should be sacrificed.


57/59

Drift Free Clocks Lemma The distance between any two

points that occur at a drift free clock is 0.

Meaning All points associated with aprocessor with a perfect clock may becollapsed into a single supper point forsynchronization distance purposes.

Number ofnodes inthe graph

Number ofprocessors=


58/59

Drift Free Clocks - Algorithm

Idea Use distribution Bellman-Ford.

Algorithm message from uto vcontains:

timestampu, wu(v,u),wu(u,v),du(s,u),du(u,s)When v receives a message with delay bounds [L,H]:

vd = timestampv - timestampuwv(u,v) = min{H-vd, wu(u,v), wv(u,v)}

wv(v,u) = min{L+vd, wu(v,u), wv(v,u)}dv(v,s) = min{du(u,s)+wu(v,u), dv(v,s)}dv(s,v) = min{du(s,u)+wu(u,v), dv(s,v)}D= (dv(v,s) - dv(s,v))ev = (dv(v,s) + dv(s,v))


59/59

References

A Theory of Clock Synchronization, by Patt-Shamir and Rajsbaum (STOC 94).

Optimal and Efficient Clock SynchronizationUnder Drifting Clocks, by Ostrovsky and Patt-Shamir (PODC 99).

Time, Clocks, and the Ordering of Events in a

Distributed System, by Leslie Lamport.

Documents

Clock Sync