Fine Synchronization for Wireless Sensor ... - Semantic Scholar

This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the ICC 2008 proceedings.

Fine synchronization for wireless sensor networks using gossip averaging algorithms Nicolas Maréchal

Jean-Benoît Pierrot

Jean-Marie Gorce

CEA-LETI MINATEC Grenoble, France [email protected]

CEA-LETI MINATEC Grenoble, France [email protected]

CITI - INSA Lyon Lyon, France [email protected]

Abstract—As synchronization preambles represent a great overhead on wireless data packets, it could be interesting to powerful them. Fine synchronization solves this problem by homogenizing clock drift values on network regions. When coupled with any clock bias consensus method (coarse synchronization), fine synchronization allows the setup of time-based medium multiplexing schemes (TDMA, Slotted-ALOHA, ...) . This paper deals with a lightweight method able to perform the computation of local clock drift corrections using simple interaction rules between neighbor nodes. The solution proposed in this work has the main advantage to be robust to measurement noise while having the technologically interesting ability to drive clock drifts towards an average consensus. Index Terms—Distributed algorithm, time synchronization, networked systems, consensus algorithms, information fusion.

I. I NTRODUCTION Sensor networks are gaining much interest during the last years because of the great amount of applications they are candidates to. However, many problems arise when considering heterogeneity and dynamic behaviors of nodes. One of these problems is the deployment of sophisticated channel resource sharing methods like time slotted scheme (Time Division Multiple Access). This challenge relies heavily on the ability of the network to synchronize both coarsely and finely. By fine synchronization we mean the adjustment of clock drifts while coarse synchronization refers to the adjustment of time biases and frame positioning (called "fine grained synchronization" in [1] and [2]). The impact of fine synchronization is of great interest: it permits to reduce the overhead and thus to increase the payload size. The study of couplings between oscillators in control theory and physics aims the description of system reaching global consensus or equilibrium in both discrete and continuous cases ([3], [4]) but generally provides no garantee on the quality of the limit solution and its robustness to external noise. In this paper, a simple consensus algorithm for fine synchronization is proposed, which makes reuse of some standard structures currently deployed in DVHop coarse localization systems (tree-based propagation of beacon information). In many networking applications, properties of timing devices are very important. These properties include stability and accuracy of clocks, homogeneity of parameters through a set of entities (biases, drifts, ...), and good measurement abilities of these parameters. However, all these points are linked to one:

the clock drift, i.e. the deviation with respect to the univeral time. For example, the local time of a clock (clock bias) read at universal time t is usually modelled using the following integral formula ([5]): t ε(t)dt (1) T (t) = T0 + t0

where ε(t) is the instantaneous clock drift, and T0 is the clock bias at universal time t0 (see [6]). It has been shown that the performances of radiolocalization in UWB systems are affected by clock accuracy in terms of drift (see [6]). More important, this concerns a wide range of crucial radiocommunication subsystems including: • coarse synchronization protocols: ECMA 368, ... • medium access schemes: TDMA, slotted ALOHA, ... • modulation techniques: OFDM (CFO issues), ... In the next section, we begin with a short introduction to gossip algorithms and recall some graph theoretic definitions. After giving details on CDR measurement process and its modelization, we will explain how to smartly combine measures for averaging purposes by taking opportunity of flooding schemes and structures used in particular in the deployment of DV-Hop protocols. The proposed algorithm will run two mechanisms simultaneously: • the computation of a time average of drift estimates w.r.t. to a chosen reference (root) on every node • a peer-to-peer average consensus algorithm with correction abilities (space-time diffusion). As a matter of consequence, JEGA will then naturally appear as building block for providing a solution to the problematic of this paper. II. G OSSIP - BASED SYNCHRONIZATION ALGORITHM A. Gossip-based consensus algorithms A distributed algorithm is told be gossip-based whenever it relies on simple and local interactions between neighbor nodes. A widely studied subclass of gossip-based algorithms is dedicated to address the problem of reaching a common agreement (consensus) upon a value or a decision. More specifically, a particular application of consensus protocol is the computation of averages ([3], [7]). Despite its discouraging simplicity, this case is of great interest: it easily extends to the extraction

978-1-4244-2075-9/08/$25.00 ©2008 IEEE


of a large variety of aggregates and statistical quantities like sums/products, max/min values, but also means/variances, quantiles and rank statistics ([8] and [9]). As a corollary, linear regression and polynomial map fitting can also be easily performed. Moreover, gossip-based consensus are achievable in a fully decentralized, asynchronous and lightweight fashion, even in extremely large and highly dynamic networks. This makes of gossip-algorithms good candidates for sensor networks applications, where bandwidth, energy consumption and CPU/memory usage are enduring severe limitations for the sake of nodes lifetime and cost. Despite their suboptimality, consensus algorithms can be used as a prelude to more sophisticated protocols by homogenizing parameters upon groups of nodes. The analysis of performances of gossiping algorithms relies essentially on diffusion speed statistics and is then closely related to performances of flooding/multicasting processes and mixing time of Markov chains: some asymptotic bounds on convergence time are given in [9] and [8]. Gossipbased average consensus algorithms (gossip averaging) have been widely studied in literature ([7], [8] , [3], ...) and extended in [10] and [11] in order to jointly compute an average consensus while gaining accuracy in the estimation of input parameters. The latter version is a fully asynchronous solution which is robust to measurement noise and able to asymptotically stabilize the consensus. B. Spanning trees and graph theoretic notations Given an undirected graph G = (V, E), a spanning tree T on G is an acyclic connected subgraph of G having the same vertices set. For our purposes, the distance dG (i, j) between two vertices i and j will be the length of the shortest (undirected) path linking node i and j. We extend the definition of T by marking a vertex of V as root for T. This procedure induces a hierarchy (partial ordering) over the neighbor vertices of T according to their relative distance to the root: p is the unique parent node of q, denoted p = π(q) and q ∈ π −1 (p), whenever (p, q) ∈ ET and dT (p, R) < dT (q, R). A vertex which is not the parent of any other is called a leaf, and such vertices are stored in the set L. On the other side, the root node has clearly no parent. On a spanning tree, the shortest path between two vertices is uniquely defined. Thus, without ambiguity, the collection of vertices encountered on the shortest path linking i and j is written Γ(i, j), including both extrimities. C. Principles of the fine synchronization algorithm The solution presented in this paper gives a way of reaching an average consensus on clock drifts. For this purpose, two parallel process will run: a propagation scheme and an average consensus algorithm. The convergence towards this consensus relies on estimates of the drift ratio between each node and an elected root node (reference). As shown in this paper, these values allow the computation of the necessary clock correction for each node. Depending on the kind of network, the reference clock can be a classic sensor node, a base station, or a dedicated clock reference (GPS for instance). The type

of reference and its network coverage will impact estimation quality and convergence rate. D. Clock drift measurement and estimation We suppose it is possible to measure the clock drift ratio (CDR) γij = εi /εj of two nodes at the receiver side. There is a lot of way of obtaining this value. For example, in [6] the author proposes to measure the duration of a packet of known length. Another classical solution is to measure the frequency offset at the output of the preamble synchronization system (PLL for instance) used for packet reception. Independently of the manner in which this measure is obtained, CDRs are assumed to be known up to a centered (τ ) (non necessarily gaussian) observation noise bij with variance 2 σij : (τ )

(τ )

γ ij = γij + bij

∆

2 with σij = V [bij ]

(2)

The power of external noises (thermal and interferences) affects σij but for the sake of simplicity bij will be taken stationary through time, but not necessary through space (differs between pairs of nodes). In [6], σij is a linear function of the ratio εi /∆, where ∆ represents the supposed packet length. The set of measurement dates of γij is denoted Tij . (t) (t) We also define Tij and µij to be: (t) ∆ (t) ∆ (t) Tij = Tij ∩ [0, t] µij = Card Tij In our protocol, clock drifts are supposed to remain constant (except negligible and external action: small clock speed instability, thermal drift, cosmic rays, ...) or vary extremely slowly in comparison to the convergence time of our algorithm, and (τ ) ij , are i.i.d. random variables. As measurements of γij , i.e. γ a matter consequence, the classical sample mean estimator, as in (3), will be optimal w.r.t. the Cramér-Rao lower bound whenever observation noises are gaussian: (t)

γ ij =

1

(t) µij (t) τ ∈Tij

(τ )

γ ij

(3)

E. DV-Hop and tree-based linkage DV-Hop is a protocol for coarse localization of nodes on an ad-hoc wireless network invented by Niculescu and Nath [12]. In this system, location-aware nodes (GPS, ...) are called beacons. Each beacon floods the network with packets containing its location. Nodes update their estimated position using a trilateration process and the knowlegde of their distance to the beacons (in hops). The idea developped in this paper, is to use the propagation mode of DV-Hop-like flooding procedures in order to create tree structures necessary for our algorithm. This way, a clear background is set for computing the CDR between any node and a reference (DV-Hop beacon for instance). The computation of these ratios is based on a chain rule for CDRs which states that for any sequence i1 , i2 , . . . , im of nodes, one can write:


F. Computation of the reference drift correction

εi εi ε i εi εi1 = 1 2 . . . m−2 m−1 εim εi2 εi3 εim−1 εim

(4) (t)

pπ(p) are shortFor clarity reasons, indices in γpπ(p) and γ ened in a way such that only one index p stands for the couple (p, π(p)), i.e. a node and its parent in the spanning tree. Using this notation, it is shorter to describe the propagation phase of the algorithm. The idea is to measure locally the CDR between a node and the beacon. This can be done on each node p by taking the last numerical value send by the parent node π(p) and multiplying it with the current estimation of γp (t)

zR = 1,

(t)

zi

(τ

(t)

) (t)

π(i)i = zπ(i) γ i

=

(t)

(τqi )

γ q

(5)

q∈Γ(R,i) q=R

(t)

where τqi stands for the last update time of γq corresponding to the information contained in the last packet received by i up to time t. In the propagation scheme described in (5), every εq . As a term of the product is an independent estimate of επ(q) (·)

consequence, the chain rule (4) implies that zi is an unbiased estimate of εεRi . (t) E zi =

(t)

(τqi )

E γ q

=

q∈Γ(R,p) q=R

γq = γiR =

q∈Γ(R,p) q=R

εi εR

(6) Whenever the paths joining respectively nodes i and j are bifurcating at node p, i.e. p is the node of Γ(i, R) ∩ Γ(j, R) at maximal distance of R, it is possible to compute the crossmoments of these estimates through time and entities :

(s) (t)

E zi zj

= γip γjp

q∈Γ(R,p) q=R

 γq2 +



σq2 (s)

(t)

(max(τqi ,τqj )) µq

 (7)

Equation (7) is sufficent for computing the covariance terms st for any pair of nodes (i, j), i.e.: Cij (s) (t) (t) zj − E zi − E zi (s) (t) = E zi − γiR zj − γjR (s) (t) = E zi zj − γiR γjR

st =E Cij

(s)

zi

(8) (9) (10)

It is realistic to assume that propagation flow is strong (t) enough for making µp grow to +∞ with time at each node p = R of the network . Using Landau’s notations w.r.t. time variable s + t, one easily deduce from (7) that (s) (t) 2 + o(1) (11) E zi zj = γip γjp γpR When combined with (10), this equation leads to a crucial result allowing the use of our joint averaging/estimation algorithm: st =0 (12) lim Cij (s+t)→∞

This point will be made clearer in the following.

JEGA (Joint estimation and gossip-averaging algorithm) is a distributed average consensus algorithm (gossip averaging algorithm) that the authors developped in a previous study (see [11]) and results of a generalization of the work presented in [10]. It is designed for situations where data to be averaged are estimated. This algorithm allows the parallelization of the two tasks, and then reduce latency for the availability of an estimate of the average value. Parameter estimates are stored in the n-dimensional vector Zt : T (t) (t) (t) Zt = zR , z1 , . . . , zn−1

(13)

In our case, the p-th component of Zt denotes the estimate of γpR . By (12), requirements of JEGA in terms of estimation covariances are respected, and thus, convergence is statistically ensured. In the following, Xt will be the n-dimensional vector of local average estimates: ∆ (t) (t) (t) (14) Xt = xR , x1 , . . . , xn−1 Under these notations, iterations of JEGA can be formulated as a simple difference equation: Xt+1 = Wt Xt + Zt+1 − Zt ,

X0 = Z0 = 0n

(15)

Here, Zt stands for the drift estimates vector obtained at the last iteration: one must eventually rescales t in order (t) to give its real sense to µp . In fact, this represents no difficulty except for formal representation and notations: time occurencies disappear in the protocol. Moreover, each node (t) only stores its own components: node p keeps track of xp , (t) (t−1) zp and zp in its memory. In (15), Wt represent the classic peer averaging matrix as used in [7]. Let’s denote by ep the n-dimensional vector of zeros except p-th entry equal to 1, and suppose that nodes i and j are interacting at time t. Then Wt takes the form: Wt = I −

T

(ei − ej ) (ei − ej ) 2

(16)

In opposition to the propagation phase, interactions are not limited to neighbor nodes of the spanning tree but can occur between nodes having a direct radio link in the real network, i.e. neighbors in the whole connectivity graph of the network. The assumptions made upon the networked system and the stochastic quantities are satisfying the requirements of JEGA. Full asynchronicity between propagation phase and JEGA iterations can be assumed without any problem. (t) As zi is an estimate of γiR at time instant t, then X(t) converges to the vector 1 εi ε¯ = 1 (17) X∞ = n εR εR i∈V

where ε¯ is the average value of CDRs. Equation (17) directly implies that each node i is able to compute the correction it


needs on his clock by: (t)

∀i ∈ V,

lim

t→+∞

xi

(t) zi

=

ε¯ εi

(18)

No assumption was made upon the frequency of interactions and the choice policy for neighbors. These two points are strongly related to the performances of the algorithm in terms of convergence speed. By now, no results is available concerning JEGA’s convergence rate. Nevertheless, the qualitative behaviour of error will be discussed when analyzing simulation data. G. Complete algorithm Our solution relies essentially on the construction of a spanning tree. This task can be executed in a simple manner by taking opportunity of any flooding process (broadcast, routing, ...). The other steps only require bidirectional (but possibly asynchronous) exchanges of two or three values per iteration depending on the existence of eventual relationship between the two protagonists. In every case, each node will (t) transmit its value of xi . In addition, if they are parent-child related, a propagation of γ iR occurs. Thus, the simplest version of our algorithm takes the following form:

(σi only depends on εi and ∆) and equal to 10−4 . For the sake of simplicity, a node is chosen uniformly at each iteration and will initiate an iteration with one of its neighbors chosen uniformly as well: this correspond to the gossip averaging iteration model from [7] when clocks are ticking according to sufficiently wide Poisson distribution. B. Simulation results and analysis As predicted, each term zi (t) converges towards ε¯/εR , i.e. the average-to-root CDR (fig. 1). However local estimates of γiR are not altered. Thus, when a node performs the computation of its own ε¯/εi , it benefits from averaging on the (t) value xi but not on γiR . This phenomenon is shown on figure 2: the convergence rate of clock drifts is rapidly decreasing due to the slow convergence of estimated link ratios γiR , whenever estimates of ε¯/εR stabilize rapidly and closely around its true value. One should keep in mind that no assumption was made on MAC: simultaneous iterations should occur, and in order to compare our solution with synchronous gossip algorithms, time must be strongly rescaled.

1.15

It is possible to apply the corrections but for stability reasons it is necessary to identify corrected packets. These packets must not be taken in account for estimates computation. However, digital data used for averaging or flooding purposes can be transmitted using corrected clocks. III. S IMULATION AND RESULTS A. Simulation hypotheses Without loss of generality, 33 nodes are distributed uniformily on a unit square. Two nodes are connected whenever the distance between them is less than 0.10. The connectivity graph is nevertheless asked to be strongly connected. The spanning tree is build at random upon the connectivity graph, and a root is chosen with an uniform probability among all nodes. Clock drifts are distributed according to a normal law of mean µ0 = 1 and variance σ0 = 33ppm. For this case study, measurement variance terms σij are assumed to be identical

1.1

zi(t) = εavg / εR

Build a spanning tree over the network; while Stopping criterion is not reached do if Propagation packet received then measure γ p and update γ p ; (t) p zπ(p) ; compute zp = γ store and propagate zp to nodes of π −1 (p); end if JEGA packet received or transmitted then Process JEGA iteration; if Neighbor = π(p) then measure γ p and update γ p ; end end end Algorithm 1: Fine synchronization algorithm

1.05 1 0.95 0.9 0.85 0.8

0

500

1000

1500 2000 Iteration number

2500

(t)

Fig. 1: Average-to-Root CDRs zi

3000

through time

Until now, no formal result was available about the rate of convergence of JEGA. However, some intuitive considerations can be formulated to explain the general behaviour of the mean squared error norm (MSE) through time (fig. 2). More specifically, when the variance of the measurement noise is small in comparison with the spread width of terms γiR , one can observes a transition in the slope of MSE. In the case previously described, it can be conjectured that JEGA is experiencing two differents states. The first state corresponds to the coarse homogeneization of the components of Xt just as if all the estimates to be averaged were constant (classical gossip averaging, see [13] and [7]), while the second state corresponds to the slow cancellation of estimation noise. Despite the stabilization of estimated values, the residual noise made them deviate from their true value. This deviation vanishes through time at a rate much slower than error of pure averaging. Let us give some heurisitics in order to argue this theory: the variance of estimation noise is decreasing at a rate


inversely proportional to the number of measures (i.e. αt−1 ) while the averaging error is decreasing exponentially (at least αλt2 where λ2 < 1 is the second largest eigenvalue of E [Wt ] when random matrices Wt are i.i.d.). Thus, curves on fig. 2 slowly go to 0 as t increases : one should not be duped by the logarithmic scale and think of an asymptote to some positive constant value. After the transition occurs, one can consider that estimate corrections are "instantly" averaged by JEGA. 1e+10

2

σ0 MSE

drift measured on the vertices of its relative spanning tree. Then if all spanning trees covers the same set of vertices, then each realization will converge to the same local drift correction. If the coverage are not equal, these corrections will differ. Nevertheless, one can argue that realizations are not far one from each other if covered regions are dense enough to provide a good approximation of the average drift. As a future work, we will study the quality of this approximation, try to find a way of combining several realizations of our algorithm when launched on disjoint clusters, and compare both solutions.

1e+09

IV. C ONCLUSION

Variance of εi/εavg in ppm

2

1e+08

1e+07

1e+06

100000

10000

1000

100

10 0

2000

4000

6000

8000 10000 iteration number

12000

14000

16000

2 Fig. 2: E ε(t)/¯ ε − 1 vs time

Nevertheless, the sample mean estimator was used for CDR estimation. As told before, this algorithm is widely known to be optimal w.r.t. the Cramér-Rao bound in case of gaussian noise. This optimality has important drawbacks: if the convergence rate of our algorithm is bounded by the variance of estimators as postulated, then no other estimator could improve performances. However, once again, the results presented here are based on quite restrictive hypotheses, and timescale may be seriously shrinked by tuning the propagation tree and presupposed models for MAC and interferences. C. Optimisation and robustness The most limiting element of the synchronization algorithm is the slow speed of CDR estimations. This can be accelerated by updating the estimates everytime a node receives a packet from its parent. Moreover, topology changes can be a problem for maintaining spanning trees. There are two major ways of answering to this problem. On one hand, nodes can estimate CDRs for several (or every) neighbors. On another, our tree-based propagation scheme can be extended to some more general kind of flooding or broadcast protocol. The latter solution can take great opportunity of routing protocols (distance vectors, diffusion procedures, ...). However, multiple paths make the computation of estimates more complicated due to the uncontrolled mixture of random variables: can γpR be estimated in a lightweight and proper fashion ? Another issue is the presence of multiple references. In such a situation, two cases can occur depending on the sets of covered vertices. Each realization of our algorithm converges towards the true local drift correction at each node with respect to the average

We introduced a distributed algorithm for homogenizing clocks drifts in an ad-hoc/sensor network. Due to the use of JEGA, the solution proposed in this paper is robust to measurement noise, generates a small overhead and is extremely light and easy to implement. The idea of this protocol is to achieve synchronization by reusing as much as possible structures and procedures of coexistant protocols according to the philosophy of sensor networks design. On the other hand, they will benefit of an improved quality in terms of time accuracy. This is of great importance for applications such as localization algorithms or MAC protocols. Actually, the authors are trying to jointly increase robustness and optimize performances while focusing on th vulnerability to topology changes. R EFERENCES [1] J. E. Elson, “Time synchronization in wireless sensor networks,” Ph.D. dissertation, UCLA, 2003. [2] J. E. Elson and K. Römer, “Wireless sensor networks: A new regime for time synchronization,” UCLA, Technical Report, july 2002. [3] R. Olfati-Saber, J. A. Fax, and R. M. Murray, “Consensus and cooperation in networked multi-agent systems,” in Proc. of the IEEE, 2007. [4] O. Simeone and U. Spagnolini, “Distributed time synchronization in wireless sensor networks with coupled discrete-time oscillators,” EURASIP Journal on Wireless Communications and Networking, 2007. [5] K. Römer, “Time synchronization in ad hoc networks,” in MobiHoc ’01: Proc. of the 2nd ACM international symposium on Mobile ad hoc networking & computing. ACM Press, 2001, pp. 173–182. [6] J. B. Pierrot, “Time synchronization in uwb ad hoc networks using toa estimation,” in 2005 IEEE International Conference on Ultra-Wideband (ICU 2005), Sept. 2005. [7] S. Boyd, A. Ghosh, B. Prabhakar, and D. Shah, “Randomized gossip algorithms,” IEEE/ACM Trans. Netw., vol. 14, 2006. [8] M. Jelasity, A. Montresor, and O. Babaoglu, “Gossip-based aggregation in large dynamic networks,” ACM Trans. Comput. Syst., vol. 23, no. 3, pp. 219–252, 2005. [9] D. Kempe, A. Dobra, and J. Gehrke, “Gossip-based computation of aggregate information,” focs, vol. 00, p. 482, 2003. [10] L. Xiao, S. Boyd, and S. Lall, “A space-time diffusion scheme for peerto-peer least-squares estimation,” in ISPN ’06: Proceedings of the fifth international conference on Information processing in sensor network. ACM Press, 2006, pp. 168–176. [11] N. Maréchal, J. B. Pierrot, and J. M. Gorce, “Jega: a joint estimation and gossip averaging algorithm for sensor network applications,” INRIA, ˜, Research Report, september 2007. [12] D. Niculescu and B. Nath, “Ad hoc positioning system (APS),” in Proc. GLOBECOM 2001, november 2001. [13] L. Xiao, S. Boyd, and S. Lall, “A scheme for robust distributed sensor fusion based on average consensus,” in Proceedings of IPSN ’05, 2005. [14] D. S. Scherber and H. C. Papadopoulos, “Locally constructed algorithms for distributed computations in ad-hoc networks,” in Proc. of IPSN ’04, 2004.