View
213
Download
0
Embed Size (px)
Citation preview
EEC-681/781EEC-681/781Distributed Computing Distributed Computing
SystemsSystems
Lecture 10Lecture 10
Wenbing ZhaoWenbing [email protected]@ieee.org
Cleveland State UniversityCleveland State University
Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao
22
OutlineOutline
• Clock Synchronization issues• Clock Synchronization Algorithms
– Centralized– Distributed
• Event ordering and logical clocks• Due date for project progress report
– 11/20 Monday mid-night– No extension!
Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao
33
Motivation for Clock SynchronizationMotivation for Clock Synchronization
• In everyday life, we are relying on clocks to coordinate our activities– For example, in EEC681, we meet every Monday and
Wednesday between 6-7:50pm
• In computer systems, it is also convenient to use clock as a way to coordinate different activities– For example, the “make” program relies on files’
timestamp to decide if a recompilation is necessary
Apr 19, 2023Apr 19, 2023 Wenbing ZhaoWenbing Zhao
44
Motivation for Clock SynchronizationMotivation for Clock Synchronization
• Stock market buy and sell orders• Secure document timestamps (with cryptographic
certification)• Aviation traffic control and position reporting• Radio and TV programming launch and monitoring• Intruder detection, location and reporting• Multimedia synchronization for real-time
teleconferencing• Network monitoring, measurement and control• Differentiated services traffic engineering
Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao
55
Universal Coordinated TimeUniversal Coordinated Time
• Universal Coordinated Time (UTC):– Based on the number of transitions per second of
the cesium 133 atom (pretty accurate)– At present, the real time is taken as the average of
some 50 cesium-clocks around the world– Introduces a leap second from time to time to
compensate that days are getting longer
• UTC is broadcast through short wave radio and satellite.– Satellites can give an accuracy of about 0.5 ms
Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao
66
Physical Clocks in Computer SystemsPhysical Clocks in Computer Systems
• Every machine has a timer that generates an interrupt H times per second
• There is a clock in machine p that ticks on each timer interrupt– Denote the value of that clock by Cp(t), where t is UTC
time
– Ideally, for each machine p, Cp(t) = t, or, in other words, dC/dt = 1
Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao
77
Clock Time and UTCClock Time and UTC• The relation between clock
time and UTC when clocks tick at different rates
• In practice:
1 - < dC/dt < 1 +
• Maximum drift rate: C
lock time, C
Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao
88
TermsTerms
• Clock drift rate (or clock accuracy)– The amount of deviation from UTC per unit of time (a
day or a week, etc.)
• Clock precision: – Resolution of the clock, e.g., 1s, or 1ns
• Clock skew– The difference in time values of two clocks is called
clock skew– Maximum clock skew of a group of clocks is
determined by the two clocks that have the largest clock difference
Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao
99
Clock SynchronizationClock Synchronization
• Two clocks are said to be synchronized at a particular instance of time if the clock skew of the two clocks is less than some specified constant δ
• A set of clocks are said to be synchronized if the clock skew of any two clocks in this set is less than δ
Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao
1010
Clock Synchronization IssuesClock Synchronization Issues
• A distributed system requires:– External Synchronization
• Synchronize with an external time source
– Internal Synchronization• Clocks within the same network synchronize with each other
• Clock Synchronization requires:– Each node can read the other nodes’ clock values
• Must consider unpredicted communication delay
– Time must never run backward• Smooth adjustments => must maintain the order of the events
Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao
1111
Message Propagation TimeMessage Propagation Time
• Estimate of message propagation time: (T1-T0-I)/2
Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao
1212
Message Delay DistributionMessage Delay Distribution
T1
T3T2
T4
)()( 2314 TTTT )]()[(2
14312 TTTT
Server
Client
x
0
Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao
1313
Clock Synchronization AlgorithmsClock Synchronization Algorithms
• Centralized– Passive Time Server Centralized Algorithm
• Cristian’s algorithm
– Active Time Server Centralized Algorithm• Berkeley Algorithm
• Distributed– Global Averaging Distributed Algorithms– Localized Averaging Distributed Algorithms
Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao
1414
Passive Timer ServerPassive Timer Server• Each node periodically sends a message (time=?) to the
time server at the current local clock time, T0
• The server responds with a message (time = T), T is the current time of the server
• The client receives the message at the local clock time T1, and adjusts its local clock time to T+(T1-T0-I)/2– The time taken by the server to handle the request message is I,
T+(T1-T0-I)/2
– Several measurements of T1-T0, discard the unreliable ones
Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao
1515
Active Time ServerActive Time Server
• The time server periodically broadcasts its clock time (T)• Other nodes receive the message to correct their own
clocks– Each node has the knowledge of the approximate time (Ta)
required for the propagation of the message, T+Ta
– Each nodes replies with the local clock time
• The server – Knows the approximate propagation time from each node– Takes fault-tolerant average of clock values as current time– The server adjusts it own, and sends the amount by which each
node clock requires adjustment to each node
Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao
1616
Centralized Algorithms – Centralized Algorithms – DrawbacksDrawbacks
• Single-point failure
• Scalability
Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao
1717
Global Averaging Distributed AlgorithmsGlobal Averaging Distributed Algorithms
• Each node broadcasts its local clock time periodically• Each node waits for time T
– The node collects the messages broadcast by other nodes– For each message received, the node keeps the local time– At the end of T, the node estimates the skew of its clock with
respect to each of the other nodes on the basis of the times at which it received
– The node computes a fault-tolerant average of the estimated skews and uses it to adjust its local clock
Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao
1818
Localized Averaging Distributed AlgorithmsLocalized Averaging Distributed Algorithms
• Each node exchanges its clock time with its neighbors
• Then sets its clock time to the average of its own clock and the clock times of its neighbors
Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao
1919
ExerciseExercise
• Consider the behavior of two machines in a distributed system. Both have clocks that are supposed to tick 1000 times per millisecond. One of them actually does, but the other ticks only 990 times per millisecond. If UTC updates come in once a minute, what is the maximum clock skew that will occur?
Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao
2020
Event OrderingEvent Ordering
• “Time, Clocks, and the Ordering of Events in a Distributed System”, by Leslie Lamport, Communications of the ACM, July 1978, Volume 21, Number 7, pp.558-565
• He showed that it is possible to synchronize all the clocks to produce a single, unambiguous time standard
• He pointed out the clock synchronization need not to be absolute– What usually matters is not that all processes agree on exactly
what time it is, but rather, that they agree on the order in which events occur
Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao
2121
Happens-Before RelationHappens-Before Relation
• With perfectly accurate physical time– An event a happened before an event b if a happened at an
earlier time than b
• Without using the physical clocks– Assume that the system is composed of a collection of
processes, each process consists of a sequence of events– The events of a process form a sequence, where a occurs
before b in this sequence if a happens before b– Assume sending and receiving a message is an event in a
process
Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao
2222
Happens-before RelationHappens-before Relation• “Happens-before” relation, denoted by “→”, is
defined as follows:– The relation “→” on the set of events of a system is the
relation satisfying the following three conditions:• If a and b are events in the same process, and a comes
before b, then a → b• If a is the sending of a message by one process and b is
the receipt of the same message by another process, then a → b
• If a → b and b → c, then a → c
– Event a causally affects event b
Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao
2323
Partial OrderingPartial Ordering
• Two distinct events a and b are said to be concurrent if a → b and b → a– Neither event can causally affect the other– This introduces a partial ordering of events in
a system with concurrently operating processes
Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao
2424
Logical ClocksLogical Clocks
• Logical clocks: Use the clock just as a way of assigning a number to an event, where the number is the time at which the event occurs– Define a clock Ci for each process Pi
• Assigns a number Ci(a) to any event a in that process
• The entire system of clocks is represented by the function C which assigns to any event b the number C(b), where C(b) =Cj(b) if b is an event in process Pj
• The clocks Ci are logical clocks rather than physical clocks
Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao
2525
Implementation of Logical ClocksImplementation of Logical Clocks
• The logical clocks is correct if the events of the system that are related to each other by the happens-before relation can be properly ordered using these clocks
• Clock condition: – For any event a, b, if ab then C(a) <C(b)
Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao
2626
Implementation of Logical ClocksImplementation of Logical Clocks
• According to our definition of the happens-before relation, the clock condition is satisfied if the following two conditions hold:– C1: if a and b are events in process Pi, and a comes
before b, then Ci(a) < Ci(b)– C2: if a is the sending of a message by process Pi
and b is the receipt of that message by process Pj, then Ci(a) < Cj(b)
Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao
2727
Implementation of Logical ClockImplementation of Logical Clock
• To meet C1: – Each process Pi increments Ci between any two
successive events
• To meet C2: – (a) if event a is the sending of a message m by
process Pi, then the message m contains a timestamp Tm = Ci(a).
– (b) Upon receiving a message m, process Pj sets Cj greater than or equal to its present value and greater than Tm
Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao
2828
Implementation of Implementation of Logical Clocks by CountersLogical Clocks by Counters
• A Lamport logical clock is a monotonically increasing software counter
• Each process Pi keeps its own logical clock Ci which is used to apply Lamport timestamps to events
• To capture the happens-before relation →, processes update their logical clocks and transmit the values of their logical clocks in messages as follows:– Before each event at Pi: Ci := Ci+1
– When Pi sends a message m, it piggybacks t = Ci
– When Pj receives (m,t): Cj := max(Cj,t) + 1
• e → e’ => C(e) < C(e’)
Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao
2929
Implementation of Implementation of Logical Clocks by CountersLogical Clocks by Counters
Question: Which two events are concurrent?
Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao
3030
Total Ordering of EventsTotal Ordering of Events
• We can use the logical clocks satisfying the Clock Condition to place a total ordering on the set of all system events– Simply order the events by the times at which occur– To break the ties, Lamport proposed the use of any
arbitrary total ordering of the processes, i.e. process id
Fall Semester 2006Fall Semester 2006 EEC-681: Distributed Computing SystemsEEC-681: Distributed Computing Systems Wenbing ZhaoWenbing Zhao
3131
Total Ordering of EventsTotal Ordering of Events
• Using this method, we can assign a unique timestamp to each event in a distributed system to provide a total ordering of all events
• Very useful in distributed system– Solving the mutual exclusion problem– Totally ordered reliable multicast => needed to build
fault tolerant systems