Global Time in Distributed Real-Time Systems

Dr. Konstantinos Tatas

OUTLINE

• Revision of real-time system

• Distributed real-time system requirements

• Global time

• Clock synchronization

What is real-time? Is there any other kind?

• A real-time computer system is a computer system where the correctness of the system behavior depends not only on the logical results of the computations, but also on the physical time when these results are produced.

• By system behavior we mean the sequence of outputs in time of a system.

Real-time means reactive

• A real-time computer system must react to stimuli from its environment

• The instant when a result must be produced is called a deadline.

• If a result has utility even after the deadline has passed, the deadline is classified as soft, otherwise it is firm.

• If severe consequences could result if a firm deadline is missed, the deadline is called hard.

• Example: Consider a traffic signal at a road before a railway crossing. If the traffic signal does not change to red before the train arrives, an accident could result.

Distributed RT system model

• From the POV of an outside observer, a real-time (RT) system can be decomposed into three communicating subsystems: – a controlled object (the physical subsystem, the behavior of

which is governed by the laws of physics),– a “distributed” computer subsystem (the cyber system, the

behavior of which is governed by the programs that are executed on digital computers)

– a human user or operator • The distributed computer system consists of

computational nodes that interact by the exchange of messages.

• A computational node can host one or more computational components.

Design Challenges in Distributed Systems

• Theoretically, a distributed system features the same design challenges as a centralized embedded system in terms of performance, power consumption, battery life, etc.

• However, an additional challenge exists: synchronization between nodes

Definition of time

• Newtonian physics time model is adequate for most temporal phenomena and is much simpler than relativistic time

• Time is modeled as an infinite set [T] with the following properties:– {T} is an ordered set, that is, if p and q are any two

instants, then either p is simultaneous with q, or p precedes q, or q precedes p, where these relations are mutually exclusive. We call the order of instants on the timeline the temporal order.

– {T} is a dense set. This means that there is at least one q between p and r iff p is not the same instant as r, where p, q, and r are instants.

Events in the time model• A section of the time line between two different instants is called a

duration. • An event takes place at an instant of time and does not have a

duration. • If two events occur at the same instant, then the two events are

said to occur simultaneously.• Instants are totally ordered; however, events are only partially

ordered,• Events can be totally ordered if another criterion is introduced to

order events that occur simultaneously

t0 1 2 3 4 5

D1=3-1=2

D2=5-2=3

E1

E2

T={0,1,2,3,..,N}

Causal order

• In many real time systems determining cause and effect relations between events is of interest, especially determining the primary event

• Temporal order is necessary but not sufficient to establish causal order

Global time

• An important yet challenging task is maintaining a consistent global time in a distributed real-time system

• There is no global clock, only local clocks• Local clocks drift arbitrarily• No local clock is always correct• A global time is an abstract notion that is

approximated by properly selected microticks from the synchronized local physical clocks of an ensemble.

Global time• Assume a set of nodes, each one with its own local physical clock that

ticks with granularity . Assume that all of the clocks are internally synchronized with a precision Π, i.e., for any two clocks j, k, and all microticks i

• It is then possible to select a subset of the microticks of each local clock k for the generation of the local implementation of a global notion of time.

• We call such a selected local microtick i a macrotick (or a tick) of the global time.

• For example, every tenth microtick of a local clock k may be interpreted as the global tick, the macrotick , of this clock (see Fig. 3.2).

• If it does not matter at which clock k the (macro) tick occurs, we denote the tick ti without a superscript.

• A global time is thus an abstract notion that is approximated by properly selected microticks from the synchronized local physical clocks of an ensemble.

kckg

Reasonable global time

• The global time t is called reasonable, if all local implementations of the global time satisfy the condition

• g>Π• Then for a single event e, that is

observed by any two different clocks of the ensemble, their global time-stamps can differ by at most one tick.

• This is the best we can achieve.

Temporal order

• When to events differ by less than two ticks temporal order cannot be maintained

Errors in duration measurement

Internal clock synchronization

• The purpose of internal clock synchronization is to ensure that the real-time clocks of each correct node are within precision Π, independently of their drift rates.

• The global time ticks of each node must be periodically resynchronized within the ensemble of nodes to establish a global time base with specified precision.

• The period of resynchronization is called resynchronization interval. After that the clocks are left to drift again until they are resynchronized.

Internal clock synchronization

• The synchronization algorithm must bring the clocks so close together that the amount of divergence during the next free-running resynchronization interval will not cause a clock to leave the precision interval.

• Φ+Γ≤Π• Where Φ is the convergence function and Γ is

the drift offset• Φ=2ρRint• Where Rint is the length of the resynchronization

interval and ρ is the maximum specified drift rate

Malicious clock

• clock synchronization can only be guaranteed in the presence of Byzantine errors if the total number of clocks N (3k + 1), where k is the number of Byzantine faulty clocks.

Central Master Synchronization• the central master, periodically sends the

value of its time counter in synchronization messages to all other nodes

• the slave records the time-stamp of message arrival.

• The difference between the master’s time, contained in the synchronization message, and the recorded slave’s time-stamp of message arrival, corrected by the known latency of the message transport, is a measure of the deviation of the clock of the master from the clock of the slave.

• The slave then corrects its clock by this deviation to bring it into agreement with the master’s clock.

• Used at system startup• Not fault-tolerant

Master clock: 03:05

slave clock slave clock

2s comm latency

3s comm latency

Message time: 03:05Slave time: 03:03

Corrected time: 03:07

Message time: 03:05Slave time: 03:09

Corrected time: 03:08

External clock synchronization• External synchronization links the global time of a cluster to an

external standard of time. • For this purpose it is necessary to access a timeserver, i.e., an

external time source that periodically broadcasts the current reference time in the form of a time message.

• GPS (Global Positioning System).– The accuracy of a GPS receiver is better than 100 ns and it has an

authoritative long-term stability – in some sense, GPS is the worldwide measurement standard for measuring the progression of time. Alternatively, the external time source can

• temperature compensated crystal oscillators (TCXO) – Typical drift rate of better than 1 ppm, causing a drift offset of better 1

μs/s • atomic clocks

– Rubidum clock typical drift rate in the order of 10^-12 causing a drift offset of about 1 μs in 10 days.

Example 1

• Given a clock synchronization system that achieves a precision of 90 μs, what is a reasonable granularity for the global time?

• What are the limits for the observed values for a time interval of 1.1 ms?

Example 2

• Given a – latency jitter of 20 μs, – a clock drift rate of 10^-5 s/s, and – a resynchronization period of 1 s

• what precision can be achieved by the central master algorithm?

Example 3

• A distributed system uses GPS for clock synchronization:– What is the reasonable granularity for the

global time?– What are the limits for the observed values for

a time interval of 200 ms?

References

• H. Kopetz, “Real time systems: Design principles for distributed systems” Springer

• Kopetz, H. & W. Ochsenreiter. (1987). Clock Synchronization in Distributed Real-Time Systems. IEEE Trans. Computers. Vol. 36(8). (pp. 933-940).

• Kopetz, H. (1992). Sparse Time versus Dense Time in Distributed Real-Time Systems. Proc. 14th Int. Conf. on Distributed Computing Systems. IEEE Press. (pp. 460-467).