1

Clock Synchronization in WSN:

from Traditional Estimation Theory

to Distributed Signal Processing

Yik-Chung WU

The University of Hong KongEmail: ycwu@eee.hku.hk, Webpage: www.eee.hku.hk/~ycwu

2

• Applications require clock synchronization

– Event detection

– Data Fusion

– Sleep and wake-up cycle for power management

– TDD transmission schedule

3

Challenges?• Ideal case

• In reality delays exist

– The delay between radio chip

interpret and the CPU responding

– The time for radio chip to

transform the message to EM

wave

– The time for converting the

received EM wave into the

original message

– Finally signal to CPU reception is

completed

Node i Node j

My time is

4:28:12pm

4

System model• Clock model:

• Two-way message exchange is used

to establish the clock relationship

between two nodes:Real time

Ideal clockNode A clock

slope= α

q

slope=1

Node B clock( )i i ic t t q=

2 1

4 3

, ,

, ,

real_time real_time

real_time real_time

n n

n n

i j j nt t

j i i nt t

d w

d w

=

=

2 1

,

3 4

,

1 1[ ( ) ] [ ( ) ]

1 1

[ ( ) ] [ ( ) ]

j n j i n i j n

j i

j n j i n i i n

j i

c t c t d w

c t c t d w

q q

q q

=

=

5

Pairwise Synchronization: Gaussian case [1]

• Synchronize node j to node i (assume node i is the

reference)

• Approach 1: MLE

• Assume N rounds of time-stamp exchange, we have

2 1

,

3 4

,

1( ) ( )

1( ) ( )

j

j j

j

j j

j n i n j n

j n i n i n

c t c t w

c t c wdt

dq

q

=

=

211 ,11

21,

4 3,11 1

4 3,

( ) -1( )

( ) -1 1/( )

/( ) - ( ) 1

( ) - ( ) 1

j ji

j N j j Ni N

j j ii j

i Ni N j N

i

c t wc t

c t wc td

wc t c t

wc t c t

q

=

1 t jd = 1 T θ z

6

• Log-likelihood function:

• q is linear and can be easily estimated:

• Put it back to the log-likelihood function, d can be

obtained by maximizing

• Differentiating this function w.r.t. d and set it to zero:

• Finally, the clock parameters are recovered from

2

2 2, |

2 2ln ( , ) ln

i j

i j

Nf

dd

=

t T θ1θt T

1ˆ( ) ( ) ( )H H

j jj id d= θ T T T t 1

1 2

2( ) || ( ( ) )( ) ||H H

N j j j j id d

=

=

P

I T T T T t 1

1ˆ2

H H

i i

Hd

=

1 Pt t P1

1 P1

1 2 1ˆ ˆ ˆˆ ˆ ˆ ˆˆ 1/ [ ( )] , [ ( )] / [ ( )]j jd d d q= =θ θ θ

7

• Approach 2: Low complexity estimator

• d appears in both system model equations, so we can

eliminate it by adding two equations

• Putting the N rounds of message into a vector form:

• The MLE for this equation is

• This estimator is of lower complexity since there is no

need to compute d

• But since we are not estimating the unknowns from the

original equations, there may be some loss in

performance

2 3 1 4

, ,

1[ ( ) ( )] [ ( ) ( )] 2j n j n i n i

j

j j

n j n i nc t c t c t c t w wq

=

2 31 41 1 ,1 ,11 1

' '

1 4 2 3, ,

( ) ( ) -2( ) ( )1/

/

( ) ( ) ( ) ( ) -2

j

j j i ji i

i j

i N j

j

Ni N i N N

j

j N j

c t c t w wc t c t

w wc t c t c t c t

q

= =

t T θ z

' ' 1 ' 'ˆ ( )H H

j j j i

=θ T T T t

8

Comparison

• We have shown theoretically that the relative loss of the

performance bound of the low complexity estimator w.r.t.

to CRB is less than 1%

9

Pairwise synchronization under

exponential delays [2][3]

• Approach 1: MLE

• Revisit the message exchange equations:

• If wj,n and wi,n are i.i.d. exponential R.V.s, the likelihood function is (with j=1/j, j=qj/j)

2 1 3 4

, ,

1 1( ) ( ) , ( ) ( )

j j

j j j j

j n i n j n j n i n i nc t c t w c td c wdtq q

= =

1 2 3 4

1

2 2 3 4 1

1

2 1 3 4

1 1

({ ( ), ( ), ( ), ( )} | , , , )

exp [( ( ) ( )) 2 ( ) ( )]

[ ( ) ( ) 0] [ ( ) ( ) 0] [ 0]

N

i n j n j n i n n j j

NN

j n j n j i n i n

n

N N

j j n i n j j j n i n j

n n

f c t c t c t c t d

c t c t d c t c t

c t c t d c t c t d d

=

=

= =

=

10

• Closed-form estimate of can be obtained by

differentiating the above equation w.r.t. and set it to

zero

• Putting back into the likelihood function, we can show

that the MLE that maximizes the profile likelihood

function is

• This is a linear programming problem, and can be solved

using existing solver, but the worst case complexity is at least (N3)

• We have also proposed a low complexity algorithm for solving this problem, and the worst case only takes (N)

* * * 2 3

, ,1

2 1

1

3 4

1

[ , , ] arg max [( ( ) ( )) 2 ]

{ ( ) ( ) 0} ; 0subject to

{ ( ) ( ) 0} ; 0

j j

N

j j j n j n jd

n

N

j j n i n j n

N

j j n i n j n

d c t c t d

c t c t d d

c t c t d d

=

=

=

=

11

• Approach 2: Iterative weighted median

• Add the two message exchange equations:

• wj,n - wi,n becomes Laplacian R.V. with location parameter 0 and scale parameter 1/

• The log-likelihood function is

• An estimate of j and j can be obtained by minimizing

the second term

,,

2 3 1 4

, ,[ ( ) ( )] [ ( ) ( )] 2

s nr n

j j n j n i n i n j j n i n

TT

c t c t c t c t w w

==

=

, , 1 , ,

1

ln ({ , } | , ) ln | 2 |2

NN

s n r n n j j s n j r n j

n

f T T N T T

=

=

=

, ,,

1

min | 2 |j j

N

s n j r n j

n

T T

=

12

• Now, consider two sub-problems– When j is fixed, the problem is

The solution is the median value of the sequence

– When j is fixed, the problem is

This is a weighted median problem for the data set

Simple procedure exists to compute this

• Two steps are iteratively updated

• Since the objective function is convex, it will converge to

the global optimal solution

, ,

1

min 2 | 0.5( ) |j

N

j r n s n j

n

T T

=

, , 10.5( )

N

j r n s n nT T

=

, , ,

1

min | ( / 2 ) |j

N

r n s n r n j j

n

T T T

=

, , , 1, ( / 2 )

N

r n s n r n j nT T T

=

13

Comparison

Iterative weighted median

has a significant loss w.r.t.

optimal solution

• The main reason is that iterative weighted median method adds the two message exchange equations together before estimation

• This is in contrast to Gaussian setting where this operation does not lead to significant loss

14

Proposed low-complexity algorithm has

the same performance as LP solver

Proposed low-complexity

algorithm has

the lowest complexity

15

Network-wide synchronization• How to extend pairwise algorithm to work for network-

wide synchronization?

Tree based Clustered based

• Need overhead to build and maintain tree or cluster structure

• Error accumulation is quick as number of layers increases

• Vulnerable if gateway node dies

16

Fully Distributed Algorithms• Approach 1: Coordinate Descent [4]

• Add the two-way message exchange equations to first

eliminate d:

• Assume each node perform N times two-way message

exchanges with each of its direct neighbors

2 3 1 4

, ,

1 1[ ( ) ( ) 2 ] [ ( ) ( ) 2 ]j n j n j i n i n i j n i n

j i

c t c t c t c t w wq q

=

2

1 1 ( )

2

2 3 1 4

NLL({ , } , )

1 1[ ( ) ( ) 2 ] [ ( ) ( ) 2 ]

N MM

i i i

n i j i

j n j n j i n i n i

j i

c t c t c t c t

q

q q

=

= =

But this is non-convex w.r.t. unknowns

17

• With transformation

• This is convex w.r.t. i and i , and we can alternatively

minimize this NLL

• Differentiating NLL w.r.t. i and set it to zero (also w.r.t i ),

we get two coupled iterative equations

1/ and /j j j j j q = =

{ , }{ , },,

2

2 3 1 4

2

1 1 ( )

NLL({ , } , ) [ ( ) ( )] 2 [ ( ) ( )] 2

i ji js nr n

N MM

i i i j j n j n j i i n i n i

n i j iTT

c t c t c t c t =

= =

( ) ( ) { , } { , } ( ) { , } { , } { , } { , }

, , , , , ,

1 ( )( 1)

{ , } 2 { , } 2

, ,

1 ( )

( 1) ( ) ( ) { , } { , }

, ,

ˆˆ ˆ[2 2 ][ ] [ ]

ˆ

[( ) ( ) ]

1 ˆˆ ˆ4 [ ]4 | ( ) |

Nm m i j j i m i j i j j i j i

i j s n r n j r n s n s n r n

n j im

i Ni j j i

s n r n

n j i

m m m i j j i

i j i s n r n

T T T T T T

T T

T TN i

=

=

=

=

( ) { , } { , }

, ,

1 ( )

ˆ [ ]N

m i j j i

j r n s n

n j i

T T=

18

• Approach 2: Belief Propagation [5]

• The two-way time-stamp message exchange equation

(between node i and j) can be put into matrix form

• Marginalized posterior distribution at node i:

• Computational demanding, needs centralized processing

2 3 1 4

, ,

2 3 1 41 1 ,11 1

2 3 1 4

1 1[ ( ) ( ) 2 ] [ ( ) ( ) 2 ]

( ) ( ) - 2 ( ) ( ) -21/ 1/

/ /( ) ( ) - 2 ( ) ( ) -2

j n j n j i n i n i j n i n

j i

j j ji ij i

j j i i

j N j N i N i N

c t c t c t c t w w

c t c t wc t c t

c t c t c t c t

q q

q q

=

=

,1

, ,

, , ,

i

j N i N

j i j i j i j i

w

w w

=A β A β z

, , 1 1 1

1 { , }

( ) ( ) ( , | , ) ... ...M M

i i i j j i i j i i M

i i j E

g p p d d d d

=

β β A A β β β β β β

19

• Express the joint posterior distribution using factor graph

– Factor node: local likelihood function

– or prior distribution

– Variable node: i

• Marginal distribution at each node can be obtained by

message passing on factor graph:

– Message from variable node to factor node

, , ,

2

, , ,

( , | , )

( | , )

i j i j j i i j

j i j i j i i j N

f p

=

=

A A β β

A β A β I

( )i if p= β

,

,

( )

( )\

( )

( ( ))j

j

i

i

j

l

f j

f B f

l

j f j jm m

= β β

20

– Message from factor node to variable node:

– In each iteration, messages are updated

in parallel with the information from direct

neighbouring nodes

– Each node computes the belief locally:

• As the likelihood function is Gaussian,

the messages involved in this algorithm

keep the Gaussian form

, ,

( ) ( 1

,

)

(( ) )i j i j

l

j f

l

f i i jj i jf dm m

= β ββ

(

)

))

(

( () )(i

ll

i

f B

f i imb

= β

ββ

, , ,

( ) ( ) ( )( ) ( )| ,i j i j i j

l l l

i f i i i f i fm β β v C

, , ,

( ) ( ) ( )( ) ( | , )j i j i j i

l l l

f i i i f i f im β β v C

21

• In practice, each real node computes

both kinds of messages

• Information passes between real nodes

is set as message from factor-to-

variable, and inherits the properties

– Still Gaussian. Only mean and covariance

needed to be exchanged

– Updated in parallel by local computation

with received mean and covariance

messages from neighboring nodes

• The belief computed at node i, with the

received messages from all direct

neighbouring nodes, is still Gaussian:

where

( ) ( ) ( )|( ) ~ ,( )l l l

i i i ib β β μ P

,

( )

( )

( )

i j

l

i

j

l

f i

i

=

11

P C, ,

( ) ( )( )

( )

( )

i j i j

l l

f i

l

fi i

l

j i

i

=

1

vμ CP

22

• The estimate at node i is

• The qi and i can be recovered from after

convergence

• It is generally known that if the FG contains cycles,

messages can flow many times around the graph,

leading to the possibility of divergence of BP algorithm

• Two properties:

• BP in this application converges regardless of network

topology, even under asynchronous message update

• The converged solution can also be proved to be equal

to the centralized ML solution

( ( )( )) (ˆ )l

i

l l

i i ii db= = β β μββ

ˆiβ

23

Comparison• Simulation setting: 25 nodes, dij [8,12], qi[-5.5,5.5],

i[-0.955,1.055], , 1000 topologies,

initialization=[1, 0], N=10

2 0.1i =

reference node

24

BP algorithm under

asynchronous message

exchange

BP converges much faster

than CD as second order

information is included in the

messages

25

Distributed tracking with DKF [4]

• Clock parameters may stay constant within a short

period of time

• But it will change over time

• We can either redo synchronization (throwing away

previous estimates), or we can do tracking

• If the change is slow, tracking is preferred

• Re-representation of clock model:

• After sampling:

0

0

0

( ) ( )

( )

i i i i

t

i i

c t t p B t

d

q

q

=

=

This term is due

to phase noise

( ) '( )i i it p B t =

10

0 0 0

0

( 1)

( )

( ) [ ( ) 1] [ ( ) 1]

i

i

l

i i i i

m

l

l

c l l m l

q

=

=

26

• Writing the accumulated skew and offset in recursive

forms:

• Clock parameter evolution model:

• Measurement equations based on two-way message

exchange

• Gather all measurement equations for

' '

( )

0

( ) ( 1) [ ( ) ( 1)]

( ) ( 1) [ ( ) 1]

i

i i i

u l

i i i

l l p B l B l

l l l

=

=

Gaussian with zero

mean and variance 2pi

0 00

( )

( ) ( 1) ( )1 0 0

1( ) ( 1) ( )

iii

i i i

i i i

l

l l u l

l l u l

=

bAx

{ , } { , } { , }

, , 2 ( ) 2 ( )i j i j i j

r n s n j i lT T l l V =

( )j i

, , ( ) ( ) ( )i l i l i il l= z C x v

27

• If all information is gathered in a single place, the optimal

solution is centralized KF

• KF cannot be implemented in distributed way since the

Kalman gain matrix contains correlation among nodes

• Solution: Impose a block diagonal structure on the

Kalman gain matrix:

• We can solve for Ki(lk) in closed form

• Each round of tracking includes time-stamp exchange

and messages exchange for KF update

1 1 1

1 , , ( ) 1

ˆ ˆ( | ) ( | )

ˆ ˆ( | ) ( | ) ( )( ( | ))k

i k k i i k k i

i k k i k k i k i l i l i k k

l l l l

l l l l l l l

=

=

x A x b

x x K z C x

( )

2

( ) arg min Tr ( | )

s.t. ( ) ( )

k

k k kl

M T

k i i i k i

l l l

l l=

=

=

KK P

K U K Ω

28

• Initialization:

– but it takes a long time to converge

– CD + bootstrap for covariance estimation (5 rounds of initial

time-stamp exchange)

-1(0 | 0) [1 0] , (0|0)=T

i =x P I

29

• Start with 25 nodes (B=15)

• If a node fails during tracking, we simply remove it from the

equations

• If the node later resume working, we will use its previously stored

clock parameter estimates and covariance matrix

• If a new node suddenly join in, we will use-1(0 | 0) [1 0] , (0|0)=T

i i =x P I

30

Distributed algorithm under

exponential delays [6]

• The pairwise LP problem can be easily extended to

network-wide setting:

• This is a LP problem, but very large in size

• Centralized solution is computational expensive and has

large communication overhead

* * 2 3 4 1

,1 ( ) 1 1

2 1

3 4

[ , ] arg max ( ( ) ( )) ( ( ) ( )) 2

( ) ( ) 0

subject to ( ) ( ) 0 ( , ) , 1,...,

0

M N N

j j n j n i i n i n ij

i j i n n

j n j i n i j i ij

j n j i n i j i ij

ij

c t c t c t c t Nd

c t c t d

c t c t d i j E n

d

= = =

=

=

x d

x d

N

• Challenge of solving this problem in a distributed way:

constraints are coupled for parameters at different nodes

• By introducing slack variables w and auxiliary replica

variables z, we can transform the problem to

• This problem can be solved by ADMM (iteratively

minimizing the augmented Lagrangian function w.r.t. the

unknowns, x, d, w, z, and the Lagrange multipliers)

31

, , ,2 1 ( )

{ , } { , } { , } { , }

1 2

{ , } { , }

2 3 4

4 { , }

1

arg min ( 2 ) ( ) ( )

s.t. , ,

, ,

( , )

M MT

i i ij

i i j i

i j i j i j i j

i j

i j i j

ij N ij

i j

q q

N d

d

i j E

= =

=

= =

= =

=

x d w za x d 0 w 0

B x z E x z

1 z w z

z 0

• Properties:

– The resultant algorithm only involves local computation at each

node and communications with its direct neighbors

– Closed-form expressions are available for each update step

– It will converge to the centralized ML solution

• Contrast to existing applications of ADMM:

– Direct application of ADMM to the original LP would not result in

distributed algorithm

– Most existing applications that result in distributed algorithms are

for Gaussian likelihood

– Most existing works consider a single (or a small set of) common

parameter

32

33

• Simulation results on a 25

nodes network

• K=5• =1

• CD as initialization help to

speed up convergence

34

Conclusions• We discussed clock synchronization in wireless sensor

networks

• We started with pariwise synchronization: Gaussian case

and exponential case

• Then we discussed network-wide synchronization: CD,

BP, ADMM

• We also discussed tracking using distributed KF

• Future works:

– BP based distributed algorithm for exponential delays?

– How about arbitrary distributed delays, asymmetric delays?

References[1] Mei Leng and Yik-Chung Wu, ``On Clock Synchronization Algorithms for Wireless Sensor Networks

under Unknown Delay," IEEE Trans. on Vehicular Technology, vol. 59, no.1, pp. 182-190, Jan 2010.

[2] Mei Leng and Yik-Chung Wu, ``Low Complexity Maximum Likelihood Estimators for Clock

Synchronization of Wireless Sensor Nodes under Exponential Delays," IEEE Trans. on Signal

Processing, Vol. 59, no. 10, pp. 4860-4870, Oct 2011.

[3] Mei Leng and Yik-Chung Wu, ``On joint synchronization of clock offset and skew for Wireless Sensor

Networks under exponential delay," Proceedings of the IEEE ISCAS 2010, Paris, France, pp. 461-464,

May 2010.

[4] Bin Luo and Yik-Chung Wu, ``Distributed Clock Parameters Tracking in Wireless Sensor Network,"

IEEE Trans. on Wireless Communications, Vol. 12, no. 12, pp.6464-6475, Dec 2013.

[5] Jian Du and Yik-Chung Wu, ``Distributed Clock Skew and Offset Estimation in Wireless Sensor

Networks: Asynchronous Algorithm and Convergence Analysis," IEEE Trans. on Wireless

Communications, Vol. 12, no. 11, pp. 5908-5917, Nov. 2013.

[6] Bin Luo, Lei Cheng, and Yik-Chung Wu, ``Fully-distributed Clock Synchronization in Wireless Sensor

Networks Under Exponential Delays," Signal Processing, Vol. 125, pp. 261-273, Aug 2016.

http://www.sciencedirect.com/science/article/pii/S0165168416000578

Further related readings:

Yik-Chung Wu, Qasim M. Chaudhari and Erchin Serpedin, ``Clock Synchronization of Wireless

Sensor Networks," IEEE Signal Processing Magazine, Vol. 28, no. 1, pp.124-138, Jan. 2011. 35

Jun Zheng and Yik-Chung Wu, ``Joint Time Synchronization and Localization of an unknown node

in Wireless Sensor Networks," IEEE Trans. on Signal Processing, Vol. 58, no. 3, pp. 1309-1320,

Mar 2010.

Mei Leng and Yik-Chung Wu, ``Distributed Clock Synchronization for Wireless Sensor Networks

using Belief Propagation," IEEE Trans. on Signal Processing, Vol. 59, no. 11, pp. 5404-5414, Nov

2011.

36