First, the previous theory for state feedback is reviewed in sections II-III, but ..... [5] Cédric Langbort, Ramu Sharat Chandra, and Raffaello. D´Andrea. Distributed ...
Proceedings of the 45th IEEE Conference on Decision & Control Manchester Grand Hyatt Hotel San Diego, CA, USA, December 13-15, 2006
ThB18.1
A Separation Principle for Distributed Control Anders Rantzer
z2 , u2
Abstract— A linear quadratic control problem is considered, where several different controllers act as a team, but with access to different measurements. The state feedback solution obtained in a previous paper is here extended to output feedback and a separation principle is proved. The condition that certain measurements are unavailable to some state estimators is enforced by imposing a requirement that the generated estimates must be insensitive to the changes in corresponding covariances between measurement noise and process noise.
I. INTRODUCTION Decision making when the decision makers have access to different information concerning the underlying uncertainties has been studied since the late 1950s [8], [6]. The subject is sometimes called team theory, sometimes decentralized or distributed control. We are using the latter term to indicate that some restricted communication between the decision makers is allowed. The theory was originally static, but work on dynamic aspects was initiated by Witsenhausen [14], who also pointed out a fundamental difficulty in such problems. Some special types of team problems were solved in the 1970’s [13], [3], but the problem area has recently gain renewed interest. Spatial invariance was exploited in [1], [2], conditions for closed loop convexity were derived in [12], [10] and approximate methods using linear matrix inequalities were given in [5]. In our previous paper [9] a linear quadratic stochastic optimal control problem was solved for a state feedback control law with covariance constraints. Such problems have previously been solved using socalled S-procedure [7], [15]. The method gives a nonconservative extension of linear quadratic control theory to distributed control with bounds on the rate of information propagation. In this paper the idea is extended to observer based output feedback in analogy with standard linear quadratic Gaussian control. First, the previous theory for state feedback is reviewed in sections II-III, but with the main theorem somewhat rephrased. Then an optimal Kalman filter is introduced in section IV and a separation theorem is proved in section V. II. DISTRIBUTED
CONTROL BY COVARIANCE CONSTRAINTS
The idea that distributed control problems can be treated by linear quadratic optimization with covariA. Rantzer is with Automatic Control LTH, Lund University, Box 118, SE-221 00 Lund, Sweden, rantzer at control.lth.se.
1-4244-0171-2/06/$20.00 ©2006 IEEE.
z4
z1 , u1
z3 Fig. 1. The graph illustrates the interconnection structure of the states. For example, the update of state z3 depends directly on z2 , z3 and z4 , but only indirectly on z1 .
ance constraints will next be explained by considering an example. Consider the following linear discrete time system: ⎡ ⎤⎡ ⎤ ⎡ ⎤ z1 (t + 1) 0 Φ 13 0 z1 (t) Φ 11 ⎢ z2 (t + 1)⎥ ⎢Φ 21 Φ 22 Φ 23 ⎢ ⎥ 0 ⎥ ⎢ ⎥ ⎢ z2 (t)⎥ ⎥ ⎢ ⎣ z3 (t + 1)⎦ = ⎣ 0 ⎦ ⎣ Φ 32 Φ 33 Φ 34 z3 (t)⎦ z4 (t + 1) 0 0 Φ 43 Φ 44 z4 (t) ⎡ ⎤ (1) ⎤ ⎡ Γ1 0 w1 (t) ⎢w2 (t)⎥ ⎢ 0 Γ 2 ⎥ u1 (t) ⎥ ⎥ +⎢ +⎢ ⎣w3 (t)⎦ ⎣0 0 ⎦ u2 (t) w4 (t) 0 0 where w is white noise with unit variance, and w(t) is independent of z( j ) for j ≤ k. The zero positions in the Φ -matrix reflect the graph structure shown in Figure 1. We will consider the control problem of minimizing the stationary variance E ( z2 + u2 ) using feedback of the form
u1 (t) = µ 1 z¯1 (t), z¯2 (t − 2), z¯3 (t − 1), z¯4 (t − 2)
(2) u2 (t) = µ 2 z¯1 (t − 1), z¯2 (t), z¯3 (t − 1), z¯4 (t − 2) where z¯i denotes the time history of the state zi : ⎤ ⎡ zi (t) ⎢ zi (t − 1)⎥ ⎥ ⎢ z¯i (t) = ⎢ zi (t − 2)⎥ ⎦ ⎣ .. . Notice that also the communication delay pattern in (2) reflects the graph structure Figure 1. It takes one time step for the value of a state to be passed from one node to another along the graph.
3609
45th IEEE CDC, San Diego, USA, Dec. 13-15, 2006
ThB18.1
The communication delay pattern can be viewed as the consequence of delays in the communication channels between the controllers. The delay pattern satisfies one particular condition which is essential for tractability of the synthesis problem: There is no signaling incentive. In other words, information propagates through the communication channels at least as fast as it propagates through the plant itself. This condition, closely related to the “partially nested information structure” discussed in [4], [3], was explicitly stated in [2] and for a more general setting in [12]. In order to illustrate our idea in the state feedback case, we introduce the matrix notation Φ , Γ to write (1) as z(t + 1) = Φ z(t) + Γ u(t) + w(t) For specification of the communication delays, it is convenient to also have an extended state realization x(t + 1) = Ax(t) + Bu(t) + Fw(t) where x(t) ∈ R12 , u(t) ∈ R2 T x(t) = z(t)T w(t − 1)T w(t − 2)T T = x1 (t) . . . x12 (t)
⎡
Φ A = ⎣0 0
0 0 I
0 0 0
⎤ ⎦
⎡ ⎤ Γ B = ⎣0⎦ 0
⎡ ⎤ I F = ⎣ I⎦ 0
III. STATE
= E ( x2 + u2 ) − 8 = E x6 (t)u1 (t) = E x10 (t)u1 (t) = E x7 (t)u1 (t)
FEEDBACK WITH COVARIANCE CONSTRAINTS
Theorem 1: Given A ∈ Rnn , B ∈ Rnm and Q1 , . . . , Q J ∈ R(n+m)(n+m) , consider a Markov process x satisfying the stochastic difference equation x(t + 1) = Ax(t) + Bu(t) + w(t)
(4)
where w(t) is a stationary Gaussian zero mean process with unit variance. Then, for every γ¯ , the following statements are equivalent: ( i) There exists a feedback law u(t) = µ ( x(t)) that together with (4) has a stationary zero mean solution satisfying T T x x x x Q1 QJ E = ... = E ≤ γ¯ (5) u u u u
(ii)
Then the control problem can be restated as follows: Find a map µ : R12 → R2 such that the stochastic difference equation (3) together with the feedback law u(t) = µ ( x(t)) has a stationary zero mean solution satisfying
γ ≥ E ( z2 + u2 ) 0 = E w2 (t − 1)u1 (t) 0 = E w2 (t − 2)u1 (t) 0 = E w3 (t − 1)u1 (t) 0 = E w4 (t − 1)u1 (t) 0 = E w4 (t − 2)u1 (t) 0 = E w1 (t − 1)u2 (t) 0 = E w3 (t − 1)u2 (t) 0 = E w4 (t − 1)u2 (t) 0 = E w4 (t − 2)u2 (t)
because z(t − 2) determines the state completely at time k − 2 and the only additional information in z¯1 (t) and z¯3 (t − 1) is the influence of w1 (t − 2), w1 (t − 1) and w3 (t − 2). It follows in particular that that the specified correlation constraints must hold. We will see in the next section that the performance achievable subject to correlation constraints can always be achieved by a linear controller. As was further explained in [9], it is also true that any linear controller u1 (t) = µ 1 ( z(t)) subject to the correlation constraints must be of the form (3). Hence, equivalence holds.
There a positive semidefinite X = X xx X xuexists ∈ R(n+m)(n+m) with X ux X uu T (6) X xx ≥ A B X A B + I
γ¯ ≥ tr ( X Q1 ) = . . . = tr ( X Q J )
(7)
−1 X ux X xx
Moreover, if L = and X satisfies the conditions of (ii) , then the conditions of (i) hold for the linear control law u = Lx + v, where v is a zero mean stochastic variable independent of w and x and with −1 E vvT = X uu − X ux X xx X xu . Proof. First assume that the conditions of (i) hold for the stationary zero mean processes x, u and the map µ . Define T X xx X xu x x X = =E X ux X uu u u
= E x8 (t)u1 (t) = E x12 (t)u1 (t) = E x5 (t)u2 (t) = E x7 (t)u2 (t) = E x8 (t)u2 (t) = E x12 (t)u2 (t)
Notice that the communication delay structure (2) here has been replaced by correlation constraints. In fact, the two conditions are equivalent. To see this, first note that the control law for u1 has the form specified by (2) if and only if it can be written as
u1 (t) =µ¯ 1 z(t − 2), w1 (t − 2), w1 (t − 1), w3 (t − 2)
Then all the conditions of (ii) follow from the corresponding conditions of (i) . Next assume that the inequalities of (ii) hold. Then −1 X uu ≥ X ux X xx X xu because X is positive semidefinite. −1 and v is a Let u = Lx + v, where L = X ux X xx stationary stochastic process independent of w and x, −1 while E v = 0 and E vvT = X uu − X ux X xx X xu . Then the difference equation (4) can be written x(t + 1) = Ax(t) + B Lx(t) + Bv(t) + w(t) Let x0 be a Gaussian stochastic variable with
(3)
3610
E x0 = 0
E x0 x0T = X xx
45th IEEE CDC, San Diego, USA, Dec. 13-15, 2006
ThB18.1
Then, because of linearity, also x1 is a Gaussian stochastic variable with E x1 = 0 and E x1 x1T = ( A + B L)[E x0 x0T ]( A + B L)T + BE vvT B T + I
−1 Moreover, if K = Sxx Sxy and S satisfies the conditions in (ii) , then the conditions of (i) hold for the estimator defined by
x(t + 1) = A x(t) + K [ C x(t) − y(t)]
−1 −1 T = ( A + B X ux X xx ) X xx ( A + B X ux X xx ) −1 + B ( X uu − X ux X xx X xu ) B T + I T = A B X A B + I ≤ X xx = E x0 x0T
Repeating the argument gives the decreasing sequence X xx = E x0 x0 T ≥ E x1 x1T ≥ E x2 x2T ≥ . . . ≥ I
and
2
λ j ( z − x j )( z − x j )T
j
for all z ∈ R . Proof. The first equality follows as xx T + λ j ( x − x j )( x − x j )T − x j xTj
= xx + T
FILTERING WITH UNCERTAIN COVARIANCE
The control problem in the previous section is analogous to an observer problem aimed to construct a state estimate that is insensitive to variations in the noise covariances. The relationship to distributed control with bounded rate of information propagation is immediate: The condition that certain measurements are unavailable to some state estimators can be enforced by imposing a requirement that the generated estimates must be insensitive to the changes in the corresponding covariances between measurement noise and process noise. The following theorem shows how a Kalman filter can be constructed subject to such conditions. Theorem 2: Given detectable A ∈ Rnn , C ∈ R pn and positive semidefinite R1 , . . . , R J ∈ R(n+ p)(n+ p) , consider a process y with J different stationary realizations: x j (t + 1) Ax j (t) + v j (t) = (8) y(t) Cx j (t) + e j (t) T v (t) v j (t) E j = Rj (9) e j (t) e j (t) where j = 1, . . . , J. Here v j (t), e j (t) are independent of v j (τ ), e j (τ ) for t = τ . Then, for every γ the following statements are equivalent. ( i) There exists a map ν such that the state x(t) = ν ( y(t − 1), y(t − 2), y(t − 3) . . .) estimate for all t satisfies E x1 (t) − x(t)2 = . . . = E x J (t) − x (t)2 ≤ γ
(ii)
λ j ( x − x j )( x − x j )T ≤ n
T x x Qj E ≤ tr ( X Q j ) ≤ γ¯ u u
IV. KALMAN
j
j
In particular
for j = 1, . . . , J.
Theorem 2 will be proved using the following lemma. n Lemma J ∈ R and λ 1 , . . . , λ J ∈ R 1: Given x1 , . . . , x with j λ j = 1, define x = j λ j x j . Then λ j x j xTj = xxT + λ j ( x − x j )( x − x j )T j
The limit is stationary with T T x x x0 x0 ≤E =X E u u Lx0 + v Lx0 + v
(12)
There positive semidefinite S = Sxx Sxy exists(n+ap)( n+ p) ∈R , with Syx Syy T A A +I Sxx ≥ S (10) C C γ ≥ tr( SR1 ) = . . . = tr( SR J ) (11)
j
λ j xxT − x j xT − xxTj
j
λ j x j xT + xxTj = 0
= 2xx − T
j
and the second as λ j ( x − x j )( x − x j )T − ( z − x j )( z − x j )T j
=
λ j xxT − x j xT − xxTj − zzT − x j zT − zxTj
j
= xx T − xx T − xx T − zzT + xzT + zx T = − ( x − z)( x − z)T ≤ 0 2 Proof of Theorem 2 Assume first that (ii) holds. Let −1 Sxy, define x by (12) and let x(t) = x(t) − x (t). K = Sxx Then
x(t) + v(t) + K e(t) x (t + 1) = ( A + K C ) J Define X = E x (t) x(t)T and let R = j =1 λ j R j for some λ j with j λ j = 1. Then X =E x(t + 1) x (t + 1)T T T A A = I K X +R I K C C T I A A tr( Sxx X ) = tr Sxx I K X +R C C KT T T A A = tr I K Sxx I K X +R C C T A A ≤ tr S X +R C C T A A = tr S X + SR ≤ tr ( Sxx X − X + SR) C C
3611
45th IEEE CDC, San Diego, USA, Dec. 13-15, 2006
ThB18.1
E x(t) − x (t)2 = tr( X ) ≤ tr ( SR) ≤ γ so condition (i) follows from condition (ii) . Next assume that (ii) fails, but (i) holds. Strong duality holds for the minimization of tr( SR1 ) subject to the inequality constraints S ≥ 0 and (10) as well as the equality constraints tr( SR1 ) = . . . = tr( SR J ). This means that failure of (ii) implies existence of λ 2 , . . . , λ J ∈ R and 0 ≤ Y ∈ Rnn such that
γ < tr( SR1 ) + − tr Y
J
J
Combining (14) and (15) gives Y ≤ X , so
γ < tr( Y ) ≤ tr( X )
λ j tr[ S( R j − R1 )]
j =2
By Lemma 1, this implies failure of (i) and the proof 2 is complete.
T A A Sxx − S −I C C
Sxy for all S = SSxx ≥ 0. With λ 1 = 1 − Jj=2 λ j and Syy yx R = Jj=1 λ j R j , this implies that
γ
0, the following two statements are equivalent: ( i) There exists a stabilizing feedback law u(t) = ν ( y(t − 1), y(t − 2), y(t − 3), . . .) with stationary solutions for j = 1, . . . , J such that T x x E j Qi j (18) u u
j
=
j λ j X j , Lemma 1 gives T I A A X +R X ≥ I K (14) C C KT Multiplying (13) from left and right by I K and its transpose gives that T I A A Y≤ I K Y +R (15) C C KT
Hence, with X =
Hence
(ii)
λ j [( A + K j C) x¯ j (t) − K j y(t)]
j
Define X j = E [ x j − x∗ ][ x j − x∗ ]T , K = j λ j K j and λ j ( A + K j C) x¯ j (t) z = ( A + K C )−1
has the same value for all i, j ∈ {1, . . . , J } and the value is not greater than γ . The conditions of (i) hold for the feedback law defined by −1
x (t + 1) = A x(t) + Sxx Sxy [ C x(t) − y(t)] −1
u(t) = X ux X xx x (t) xx X xu where the matrix X = XXux satisfies (6) X uu Sxy (7) with minimal possible γ¯ and S = SSxx Syy yx
satisfies (10)-(11) with minimal possible γ .
j
Then
Proof. The implication from (ii) to (i) is obvious. To prove the opposite implication, assume that (i) holds. x j (t + 1) − x∗ (t + 1) = ( A + K C )[ x j (t) − z] + v j + K e j As pointed out in the proof of Theorem 2, strong (» – ) » » –T – I A ( due to the detectability assumption) for duality holds T A X j = [ I K ] C E [ x j (t) − z][ x j (t) − z] C + R j KT the minimization of tr( SR1 ) subject to the inequality x∗ (t + 1) = Az + K [ Cz − y(t)]
3612
45th IEEE CDC, San Diego, USA, Dec. 13-15, 2006
ThB18.1
constraints S ≥ 0 and (10) as well as the equality constraints tr( SR1 ) = . . . = tr( SR J ). Hence the minimal value γ ∗ can equivalently be obtained by finding λ 1 , . . . , λ J ∈ R and 0 ≤ Y ∈ Rnn to maximize tr( Y ) subject to the constraints T J J Y 0 A A Y + λ j Rj λj = 1 ≤ 0 0 C C j =1
j =1
λ ∗1 , . . . , λ ∗J
∈ R be the optimal values. Completely Let analogously, strong duality holds (due to the stabilizability assumption) for the minimization of tr( X Q1 ) subject to X ≥ 0 and (6) as well as the equality constraints tr( X Q1 ) = . . . = tr( X Q J ). Here the dual is to find η 1 , . . . , η J ∈ R and P ∈ Rnn to maximize tr( P) subject to the constraints J J T P 0 ηjQj ηj = 1 ≤ A B P A B + 0 0 j =1
j =1
η 1∗ , . . . , η ∗J
Let ∈ R be the optimal values for this problem. Then, in particular there is a stabilizing feedback law u(t) = ν ( y(t − 1), y(t − 2), y(t − 3), . . .) such that ⎛ ⎞ T J x xj j ∗ ∗ ⎝ ⎠ γ ≥ λ jE ηj Qj u u j
j =1
for all solutions to (16)-(17) with j = 1, . . . , J. Rewriting the right hand side gives ⎛ ⎞ T J x∗ ⎝ x∗ ∗ ⎠ γ ≥E ηj Qj u u j =1 ⎛ ⎞ T J ∗ x −x x − x∗ ⎝ + λ ∗j E j η ∗j Q j ⎠ j 0 0 j
The state realization used in section I is pedagogically useful, but certainly not optimal for implementation purposes. Another interesting issue is the rich hidden structure of the optimization problems that stem from the graph topology. By taking this structure into account, we believe that drastic computational and conceptual advantages can be gained.
j =1
where x∗ is defined as in the proof of Theorem 2 by applying Lemma 1 to the conditional expectations x¯ 1 , . . . , x¯ J . The first term is minimized by the optimal state feedback, while the second is minimized by the optimal estimator. Hence (i) implies (ii) and the proof 2 is complete. VI. CONCLUSIONS Recent research has indicated that there is a rich class of distributed control problems (team problems) that leads to multi-variable control with convex specifications on the input-output map [2], [11]. In this paper, we have brought the analysis a significant step further, by solving the observer based output feedback synthesis problem that arises in distributed control with bounds on the rate of information propagation. We expect that these results mark the starting point for a new research direction in linear quadratic control theory. One issue that remains to be discussed is complexity and minimality of the distributed controllers.
VII. ACKNOWLEDGMENT This work was supported by a Senior Individual Grant from the Swedish Foundation for Strategic Research. REFERENCES [1] Bassam Bamieh, Fernando Paganini, and Munther A. Dahleh. Distributed control of spatially invariant systems. IEEE Transactions on Automatic Control, 47(7), July 2002. [2] Bassam Bamieh and Petros G. Voulgaris. A convex characterization of distributed control problems in spatially invariant systems with communication constraints. Systems & Control Letters, 54(6):575–583, June 2005. [3] Yu-Chi Ho. Team decision theory and information structures. Proceedings of the IEEE, 68(6):644–654, June 1980. [4] Yu-Chi Ho and Kai-Ching Chu. Team decision theory and information structures in optimal control problems–part i. IEEE Transactions on Automatic Control, 17(1):15–22, Feb 1972. [5] Cédric Langbort, Ramu Sharat Chandra, and Raffaello D´Andrea. Distributed control design for systems interconnected over an arbitrary graph. IEEE Transactions on Automatic Control, 49(9):1502–1519, September 2004. [6] J. Marshak. Elements for a theory of teams. Management Sci., 1:127–137, 1955. [7] A. Megretski and S. Treil. Power distribution inequalities in optimization and robustness of uncertain systems. Journal of Mathematical Systems, Estimation and Control, 3(3):301–319, 1993. [8] R. Radner. Team decision problems. Ann. Math. Statist., 33(3):857–881, 1962. [9] A. Rantzer. Linear quadratic team theory revisited. In Proc. of American Control Conference, 2006. [10] M. Rotkowitz and S. Lall. A characterization of convex problems in decentralized control. IEEE Trans. on Automatic Control, 51(2):274–286, Feb 2006. [11] M. Rotkowitz and S. Lall. A characterization of convex problems in decentralized control. IEEE Transactions on Automatic Control, 51(2):274 – 286, Feb 2006. [12] Michael Rotkowitz and Sanjay Lall. Decentralized control information structures preserved under feedback. In Proceedings of the IEEE Conference on Decision and Control, December 2002. [13] Nils R. Sandell and Michael Athans. Solution of some nonclassical LQG stochastic decision problems. IEEE Transactions on Automatic Control, 19(2):108–116, April 1974. [14] H.S. Witsenhausen. A counterexample in stochastic optimum control. SIAM Journal on Control, 6(1):138–147, 1968. [15] V.A. Yakubovich. Nonconvex optimization problem: The infinite-horizon linear-quadratic control problem with quadratic constraints. Systems & Control Letters, 19(1):13–22, July 1992.
3613
