Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
SLIDES CREATED BY: SHRIDEEP PALLICKARA L2.1
CS555: Distributed Systems [Fall 2019]Dept. Of Computer Science, Colorado State University
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
CS 555: DISTRIBUTED SYSTEMS
[TIME]
Shrideep PallickaraComputer Science
Colorado State University
August 29, 2019 L2.1 CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.2
Professor: SHRIDEEP PALLICKARA
Frequently asked questions from the previous class survey
August 29, 2019
¨ Quizzes/Exams?¤ Frequency, etc.
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.3Professor: SHRIDEEP PALLICKARA
Topics covered in this lecture
August 29, 2019
¨ Physical Clocks¨ How time is actually measured
¨ Clock synchronization algorithms¤ Network Time Protocol
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
SYNCHRONIZATION
August 29, 2019 L2.4
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.5Professor: SHRIDEEP PALLICKARA
Synchronization
August 29, 2019
¨ Important that processes do not access a shared resource simultaneously¤ Cooperate in granting each other temporary, exclusive access
¨ Multiple processes must sometimes need to agree on ordering of events¤ Was m1 from process P sent out before or after m2 from process Q ?
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.6
Professor: SHRIDEEP PALLICKARA
What we will look at:
August 29, 2019
¨ Synchronization based on actual time
¨ Synchronization in which only relative ordering matters¤ Rather than ordering in absolute time
SLIDES CREATED BY: SHRIDEEP PALLICKARA L2.2
CS555: Distributed Systems [Fall 2019]Dept. Of Computer Science, Colorado State University
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.7Professor: SHRIDEEP PALLICKARA
Let’s look at a make example
August 29, 2019
¨ Large programs split up into multiple source files
¨ make examines times at which all source and object files were modified
¨ input.c (@ 2151) and input.o (@ 2150)?¤ Input.c has changed since input.o was created
¨ output.c (@2144) and output.o (@2145)?¤ No recompilation required
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.8
Professor: SHRIDEEP PALLICKARA
What could happen in a distributed system with make?
August 29, 2019
¨ output.o (@2145)
¨ output.c modified right after that ¤ But assigned (@2144) by the machine
¨ No recompilation required¤ Incorrect!¤ Executable is a mix of object files from new and old sources
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
PHYSICAL CLOCKS
August 29, 2019 L2.9 CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.10
Professor: SHRIDEEP PALLICKARA
Nearly all computers have a circuitry for tracking time
August 29, 2019
¨ Timer is a more precise term than clock
¨ Quartz crystals are kept under tension¤ Oscillates at a well-defined frequency¤ Frequency depends on:
n Type and cut of crystal
n The amount of tension
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.11
Professor: SHRIDEEP PALLICKARA
Each crystal has two registers associated with it
August 29, 2019
¨ {Counter, Holding Register}
¨ Each oscillation decrements the counter
¨ When counter reduces to zero¤ Interrupt is generated: clock tick
¤ Counter reloaded from the holding register
¨ We can program the timer to generate interrupts N times per second
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.12
Professor: SHRIDEEP PALLICKARA
Measuring time
August 29, 2019
¨ Time is stored as number of ticks¤ After some known starting date and time
¨ Stored in special battery-backed CMOS RAM
SLIDES CREATED BY: SHRIDEEP PALLICKARA L2.3
CS555: Distributed Systems [Fall 2019]Dept. Of Computer Science, Colorado State University
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.13
Professor: SHRIDEEP PALLICKARA
How the OS uses the timer to report actual time
August 29, 2019
¨ The OS reads the node’s hardware timer, Hi(t), scales it and adds an offset§ Ci(t) = αHi(t) + β¤ This is the software clock
¨ In general, the clock is not completely accurate§ Ci(t) will always differ from t
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.14
Professor: SHRIDEEP PALLICKARA
On a single machine it is ok if the clock is off by a small amount
August 29, 2019
¨ All processes use the same clock
¨ Processes are internally consistent
¨ Only relative times matter in some cases
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.15
Professor: SHRIDEEP PALLICKARA
Things change when we add multiple clocks
August 29, 2019
¨ Frequency at which the quartz oscillator runs is fairly stable
¨ Impossible to guarantee that all oscillations are at the same frequency
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.16
Professor: SHRIDEEP PALLICKARA
This causes the (software) clocks to slowly get out of synch
August 29, 2019
¨ You get different values during read outs¤ Clock skew
¨ Time is not independent of the machine on which it was reported
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.17Professor: SHRIDEEP PALLICKARA
Physical and software clocks
August 29, 2019
¨ Multiple physical clocks are desirable¤ Efficiency
¤ Redundancy
¨ How do we synchronize software clocks with:¤ Real-world clocks
¤ Each other
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.18
Professor: SHRIDEEP PALLICKARA
Clock skew and drift
August 29, 2019
¨ The instantaneous difference between the readings of any two clocks is called their skew
¨ All clocks (including crystal-based ones) are subject to clock drift¤ This means that they count time at different rates
SLIDES CREATED BY: SHRIDEEP PALLICKARA L2.4
CS555: Distributed Systems [Fall 2019]Dept. Of Computer Science, Colorado State University
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.19
Professor: SHRIDEEP PALLICKARA
Clock drift rates
August 29, 2019
¨ The drift rate is the change in the offset between the clock and a nominal perfect reference clock per unit of time¤ Offset: Difference in reading
¨ Some typical drift rates:¤ Regular Clocks based on quartz crystal
n 10-6 seconds/secondn i.e., 1 second every 1,000,000 seconds or 11.6 days
¤ High precision quartz clocks: 10-7 seconds/second
¤ Atomic clocks: 10-13 seconds/second
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
HOW TIME IS ACTUALLY MEASURED
August 29, 2019 L2.20
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.21
Professor: SHRIDEEP PALLICKARA
Since the invention of mechanical clocks, time has been measured astronomically
August 29, 2019
¨ Sun’s highest apparent position in the sky¤ Transit of the sun
¤ At about noon each day
¨ Interval between consecutive transits of the sun¤ Solar day
¨ Solar second?¤ Solar day/86400
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.22
Professor: SHRIDEEP PALLICKARA
But the earth’s rotation is not constant
August 29, 2019
¨ Proved in the 1940s
¨ Earth is slowing down¤ Tidal friction¤ Atmospheric drag
¨ Based on growth patterns on ancient corals¤ 300 million years ago, we had 400 days per year
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.23Professor: SHRIDEEP PALLICKARA
The length of the year is not believed to have changed
August 29, 2019
¨ Days have simply gotten longer¤ Long-term trend
¨ Short term variations¤ Due to turbulence in the earth’s core
¨ Astronomers measure a large number of days ¤ Take their average
¤ And then divide by 86400
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.24
Professor: SHRIDEEP PALLICKARA
Physicists have taken over the job of timekeeping from the astronomers
August 29, 2019
¨ Atomic clock invented in 1948
¨ Measures time much more accurately¤ Independent of the wiggling and wobbling of earth
¨ Count energy transitions of the cesium 133 atom
SLIDES CREATED BY: SHRIDEEP PALLICKARA L2.5
CS555: Distributed Systems [Fall 2019]Dept. Of Computer Science, Colorado State University
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.25
Professor: SHRIDEEP PALLICKARA
Atomic seconds
August 29, 2019
¨ 1 second is the time for a cesium 133 atom to make 9,192,631,770transitions¤ Transitions are between two hyperfine levels of the ground state of cesium-
133 (Cs133)
¨ Why this number?¤ Atomic second = mean solar second in the year of its introduction
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.26
Professor: SHRIDEEP PALLICKARA
Several labs around the world have cesium 133 atomic clocks
August 29, 2019
¨ Periodically, each lab tells the Bureau International de L’Heure (BIH) in Paris about their ticks
¨ BIH averages this to produce TAI¤ International Atomic Time
¨ TAI is mean number of ticks of cesium-133¤ Since Jan 1, 1958 divided by 9,192,631,770
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.27Professor: SHRIDEEP PALLICKARA
TAI is highly stable, but …
August 29, 2019
¨ 86400 TAI seconds is now slower than the mean solar day¤ Days are getting longer
¨ Using TAI exclusively means that over the years noon would get earlier and earlier …
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.28
Professor: SHRIDEEP PALLICKARA
People notice these kind of things ...
August 29, 2019
¨ In 1582, Pope Gregory XIII decreed 10 days be removed from the calendar
¨ Led to riots!¤ Landlords demanded full month’s rent¤ Bankers wanted full month’s interest
¤ But employers refused to pay for work that was not done
¨ Protestant countries did not accept the Gregorian calendar for 170 years
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.29
Professor: SHRIDEEP PALLICKARA
BIH solves this problem by introducing leap seconds
August 29, 2019
¨ Whenever the discrepancy grows to 800 milliseconds¤ Total number of leap seconds introduced so far? 30
¨ Time based on TAI seconds; but in phase with apparent motion of the sun¤ This is the Universal Coordinated Time (UTC)
¨ UTC has replaced the Greenwich Mean Time
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.30
Professor: SHRIDEEP PALLICKARA
UTC uses and availability
August 29, 2019
¨ Power companies synchronize timing of their 50/60Hz clocks to UTC
¨ NIST operates a shortwave radio station in Ft Collins¤ Call letters WWV (accuracy of WWV is ± 1 msec)
¤ Accuracy is ± 10 msec due to atmospheric fluctuations
¨ Radio receivers for WWV and other UTC sources
SLIDES CREATED BY: SHRIDEEP PALLICKARA L2.6
CS555: Distributed Systems [Fall 2019]Dept. Of Computer Science, Colorado State University
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
GLOBAL POSITING SYSTEM
August 29, 2019 L2.31 CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.32
Professor: SHRIDEEP PALLICKARA
GPS
¨ Uses 30 satellites each orbiting at a height of approximately 20,000 km
¨ Each satellite has up to four atomic clocks¤ Regularly calibrated from special stations on Earth
August 29, 2019
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.33Professor: SHRIDEEP PALLICKARA
Broadcasting
¨ Each satellite continuously broadcasts its position ¤ Timestamps each message with its local time
¨ Broadcasting allows every receiver on Earth to accurately compute its own position
August 29, 2019 CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.34
Professor: SHRIDEEP PALLICKARA
Calculating the receiver’s position [1/4]
¨ Let Δr denote the deviation of the receiver’s clock from actual time
¨ When a message is received from satellite i with timestamp Ti
¤ Measured delay Δi by the receiver consists of two components• Δi = (Tnow - Ti) + Δr
August 29, 2019
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.35Professor: SHRIDEEP PALLICKARA
Calculating the receiver’s position [2/4]
¨ Signals travel at the speed of light, c
¨ Measured distance of the satellite is cΔi¨ If the receiver’s position is (xr, yr, zr)
¤ The real distance is also
€
di = (xi − xr)2 + (yi − yr)
2 + (zi − zr)2
August 29, 2019 CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.36
Professor: SHRIDEEP PALLICKARA
Calculating the receiver’s position [3/4]
¨ When we have 4 satellites¤ We have 4 equations in 4 unknowns
¤ Allows us to solve the coordinates (xr, yr, zr) and Δr
August 29, 2019
SLIDES CREATED BY: SHRIDEEP PALLICKARA L2.7
CS555: Distributed Systems [Fall 2019]Dept. Of Computer Science, Colorado State University
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.37Professor: SHRIDEEP PALLICKARA
Calculating the receiver’s position [4/4]
¨ GPS does not take leap seconds into account¤ Such an error can easily be compensated for in hardware
¨ Other sources of error:¤ Atomic clocks in satellites are not in perfect sync
¤ Receiver’s clock has finite accuracy
¤ Signal propagation speed is not constantn Signals slow down when entering e.g., the ionosphere
August 29, 2019 CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.38
Professor: SHRIDEEP PALLICKARA
Accuracy of GPS receivers
¨ Commodity GPS receivers can be precise within a range of 1-5 meters
¨ Professional receivers have a claimed error of less than 20-35 nanoseconds
August 29, 2019
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
CLOCK SYNCHRONIZATION ALGORITHMS
August 29, 2019 L2.39 CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.40
Professor: SHRIDEEP PALLICKARA
Goals of Synchronization
¨ If 1 machine has a WWV receiver¤ Keep all machines synchronized to it
¨ If no machines have a WWV receiver¤ Keep all machines in synch as well as possible
August 29, 2019
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.41
Professor: SHRIDEEP PALLICKARA
Modeling the setting [1/2]
¨ Timer causes an interrupt H times per second
¨ When timer goes off, interrupt handler adds 1 to software clock¤ Tracks ticks from some agreed-upon time
¤ Value of this clock, C
¨ UTC time, t
August 29, 2019 CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.42
Professor: SHRIDEEP PALLICKARA
Modeling the setting [2/2]
¨ When UTC time is t¨ Value of clock on machine p is Cp(t)
¨ In a perfect world¤ Cp(t) = t for all p
€
Cp' (t) =
dCdt
=1
Frequency of p’s clock at time t
€
Cp' (t)
August 29, 2019
SLIDES CREATED BY: SHRIDEEP PALLICKARA L2.8
CS555: Distributed Systems [Fall 2019]Dept. Of Computer Science, Colorado State University
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.43Professor: SHRIDEEP PALLICKARA
Skews and Offsets
¨ Offset relative to a specific time t is Cp(t) – t
¨ Skew indicates extent to which frequency differs from that of a perfect clock
€
Cp' (t) −1
August 29, 2019 CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.44
Professor: SHRIDEEP PALLICKARA
Timer specifications
August 29, 2019
¨ Real timers do not interrupt exactly H times per second¨ With H = 60; a timer should generate 216000 ticks per hour
¨ Relative error in modern timer chips is about 10-5
¤ In our example of H=60; about ±2
¤ Between 215,998 to 216,002 ticks per hour
¤ Manufactures usually specify a drift rate
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.45
Professor: SHRIDEEP PALLICKARA
Clocks ticking at different rates
Cloc
k Ti
me,
C
UTC, t
€
dCdt
=1
Perfe
ct Clock
Slow Clock
€
dCdt
<1Fast
Cloc
k
€
dCdt
>1
August 29, 2019CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
NETWORK TIME PROTOCOL
August 29, 2019 L2.46
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.47Professor: SHRIDEEP PALLICKARA
Network Time Protocol
¨ Clients connect to a time server
¨ Time server provides accurate time¤ Has WWV receiver¤ Accurate clock
¨ CHALLENGE
¤ Network latencies¤ Time is outdated by the time you get it
August 29, 2019 CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.48
Professor: SHRIDEEP PALLICKARA
Getting time from a time server
B
A
TIME SERVER
CLIENT
T2 T3
T1 T4
Offset at A = T3 – T4 + time the message was in transit
Offset at A = T3 – T4 + (T2 – T1) + (T4 – T3) 2
Delay = (T4 – T1) + (T3 – T2)
2
August 29, 2019
SLIDES CREATED BY: SHRIDEEP PALLICKARA L2.9
CS555: Distributed Systems [Fall 2019]Dept. Of Computer Science, Colorado State University
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.49
Professor: SHRIDEEP PALLICKARA
Let’s look at an example
B
A
TIME SERVER
CLIENTT1=10 T4=250
100 T2=115 T3=120
Offset at A = T3 – T4 + (T2 – T1) + (T4 – T3) 2
= 120 – 25 + (115 – 10) + (25 – 120) 2
= 100
August 29, 2019 CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.50
Professor: SHRIDEEP PALLICKARA
In the NTP protocol, both the offsets and delays are used
August 29, 2019
¨ Eight pairs of {offset, delay} tuples are buffered
¨ Minimum value of delay is chosen¤ The corresponding offset is taken as the most reliable one
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.51Professor: SHRIDEEP PALLICKARA
Time adjustment rules
¨ Time is not allowed to run backward¤ Even though the client’s clock is fast
¨ Changes are introduced gradually¤ If timer is set to generate 100 interrupts/second
¤ Each interrupt adds 10 milliseconds to the clockn To slow down we would add only 9 millisecondsn To speed up we would add 11 milliseconds
August 29, 2019 CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.52
Professor: SHRIDEEP PALLICKARA
It is possible to apply NTP symmetrically
August 29, 2019
¨ If time server is known to be more accurate¤ No point adjusting time with the client
¨ NTP divides servers into strata
¨ Server with a WWV receiver or atomic clock¤ Stratum-1 server (clock is stratum-0)
CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.53Professor: SHRIDEEP PALLICKARA
Communication and changing stratums
¨ A contacts B for offset adjustments¤ Only if A’s stratum is higher than B¤ E.g. A may be stratum 5, and B is stratum-1
¨ After offset adjustments¤ If B was a stratum-k, A becomes stratum-(k+1)¤ E.g. A is stratum-5, B is stratum-1
n Then A becomes stratum-2
August 29, 2019 CS555: Distributed Systems [Fall 2019]
Dept. Of Computer Science , Colorado State University
L2.54
Professor: SHRIDEEP PALLICKARA
The contents of this slide-set are based on the following references
August 29, 2019
¨ Distributed Systems: Principles and Paradigms. Andrew S. Tanenbaum and Maarten Van der Steen. 2nd Edition. Prentice Hall. ISBN: 0132392275/978-0132392273.
¨ Wikipedia: Atomic Clocks http://en.wikipedia.org/wiki/Atomic_clock