Conservation Laws in Optimal Control - Semantic Scholar

Conservation Laws in Optimal Control⋆ Delfim F. M. Torres Control Theory Group R&D Unit “Mathematics and Applications” Department of Mathematics, University of Aveiro 3810-193 Aveiro, Portugal [email protected] Abstract. Conservation laws, i.e. conserved quantities along Euler–Lagrange extremals, which are obtained on the basis of Noether’s theorem, play an prominent role in mathematical analysis and physical applications. In this paper we present a general and constructive method to obtain conserved quantities along the Pontryagin extremals of optimal control problems, which are invariant under a family of transformations that explicitly change all (time, state, control) variables.

1

Introduction

A number of conservation laws – first integrals of Euler–Lagrange differential equations – are well known in physics (see e.g. [2], [29]). The most famous conservation law is the energy integral , discovered by Leonhard Euler in 1744 [10]: when the Lagrangian L corresponds to a conservative system of point masses, then −L +

∂L · x˙ ≡ constant ∂ x˙

(1)

holds along the solutions of the Euler–Lagrange equations. In 1877, Erdmann published a generalization of the above [9]: in the autonomous case, i.e. when the Lagrangian L does not depend explicitly on time t, relation (1) is a firstorder necessary optimality condition of the corresponding basic problem of the calculus of variations. Conservation law (1) is now known as the second Erdmann condition. Emmy Amalie Noether, a distinguished german mathematician, was the one to prove, in 1918 (cf. [19]), that conservation laws in the calculus of variations are a manifestation of a very general principle: “The invariance of a system with respect to a parameter-transformation, implies the existence ⋆

The work is part of the author’s Ph.D. project which is carried out at the University of Aveiro, Portugal, under supervision of A. V. Sarychev. The research has been partially supported by the program PRODEP III 5.3/C/200.009/2000 and the European TMR Research Network ERB-4061-PL-97. This is a draft. Final version in Dynamics, Bifurcations and Control, F. Colonius, L. Gr¨ une, eds., Lecture Notes in Control and Information Sciences 273, Springer-Verlag, Berlin, Heidelberg, 2002, pp. 287–296.

2

D. F. M. Torres

of a conservation law for that system”. Her result comprises all theorems on first integrals known to classical mechanics. Thus, for example, the invariance relative to translation with respect to time yields the energy integral, while conservation of linear and angular momenta reflect, respectively, translational and rotational invariance. Noether’s theorem is applicable also in quantum mechanics, field theory, electromagnetic theory, and has deep implications in the general theory of relativity. Her result is so general and powerful and the paper [19] so deep and rich that a overwhelming number of “generalizations” of Noether’s theorem are indeed particular cases (see [29]). Typical application of conservation laws is to lower the order of the differential equations (see e.g. [3], [5], [20]). In this direction, conservation laws may also simplify the solution of the optimal control problems (see [28]). They are, however, a useful tool for many other reasons. Several important applications of conservation laws, both physical and mathematical, can be found in the literature. For example, in the calculus of variations they have been used to prove Lipschitzian regularity of the minimizers (see [8]), to construct examples with the Lavrentiev phenomenon (see [14]), and to prove existence of minimizers (see [7]). In control theory, for purposes of analyzing stability, controllability, etc. of nonlinear control systems, conservation laws have been used for the system decomposition in terms of lower dimensional subsystems (see [13] and [22]), while in optimal control they were used also to prove Lipschitzian regularity of the minimizers (see [24]). Extensions of Noether’s theorem can also be found in the literature. For example, an analog of Noether’s theorem for discrete systems, such as cellular automata on finite lattices, can be found in [4] or [12]. Here we are interested in formulations of Noether’s theorem for optimal control problems. Resolution of problems of optimal control is reduced to integration of Hamiltonian differential equations. Generalizations of E. Noether’s theorem for control systems whose dynamics can be described by Hamiltonian equations of motion (Hamiltonian control systems) are found in the works of van der Schaft [27] (see also [18, Ch. 12] and references therein). Generalizations of the theorem of E. Noether from the calculus of variations to optimal control have been carried out by Sussmann in [25], by Jurdjevic in [16, Ch. 13] (see also [17]) and by Torres in [26]. In [25] and [16] the parametertransformation depend on the state variables while in [26] the transformations may also depend on the independent and control variables. In [25] and [16] the parameter-transformation acts on the state variables. In [26] a second step was done, via time-reparameterization, and, besides the state variables, time-transformation is also permitted. In this work we extend previous optimal control formulations of Noether’s theorem (Theorem 3 in §3.2). We deal with a parameter-transformation depending and acting simultaneously on time, state, and control variables. The proof of the result is based on a necessary and sufficient condition that is trivially obtained from the definition of conservation law and from the definition of the Pontryagin extremal

Conservation Laws in Optimal Control

3

(Theorem 2 in §3.1). Our approach is straightforward and, in contrast to [26], no time-reparameterization is required in the proof. The result is constructive and easily leads to extra conservation laws and new information (§4).

2

Preliminaries

Problems of optimal control can be formulated in several different ways. Well known examples are the Mayer, Lagrange, Bolza and time-optimal forms. All these are theoretically equivalent (see e.g. [6]). 2.1

The Lagrange Problem of Optimal Control

We will deal with problems of optimal control in Lagrange form. Problem (P ). J [x(·), u(·)] =

Z

b

L (t, x(t), u(t)) dt −→ extr ,

(2a)

a

x(t) ˙ = ϕ (t, x(t), u(t))

a.e. on [a, b] .

(2b)

Here x(t) ˙ = dx(t) dt . The integral cost functional (2a), to be minimized or maximized, and the dynamic control system (2b), involve functions L : R × Rn × Rr → R and ϕ : R × Rn × Rr → Rn , assumed to be continuously differentiable with respect to all variables. Problem (P ) will be considered in the space n A = W1,1 ([a, b]; Rn ) × Lr∞ ([a, b]; Ω) ,

that is, we shall consider the pairs (x(·), u(·)) where the state trajectory x(·) is an absolutely continuous n-dimensional vector function, and the control u(·) is a measurable and essentially bounded r-dimensional vector function taking values on a given arbitrary set Ω ⊆ Rr , called the control constraint set. Definition 1. A pair (x(·), u(·)) ∈ A is said to be a process of (P ) if it satisfies (2b). All classical problems of the calculus of variations, and in particular all problems from mechanics, can be formulated in the form of (P ). For example, for the basic problem of the calculus of variations we put ϕ = u and Ω = Rn . 2.2

The Maximum Principle

The best known first-order necessary optimality condition for the Lagrange problem (P ) is the maximum principle. The maximum principle is one of the central results of optimal control theory. It was conjectured by L. S. Pontryagin and proved in the middle of 1950’s by him and his collaborators (see [21]).

4

D. F. M. Torres

Theorem 1 (Maximum principle for the optimal control problem (P)). If (x(·), u(·)) is an optimal process for problem (P ), then there exists a nonzero pair of Hamiltonian multipliers (ψ0 , ψ(·)), where ψ0 ≤ 0 is a constant and ψ(·) is an absolutely continuous n-dimensional vector function with domain [a, b], such that for almost all t ∈ [a, b] the quadruple (x(·), u(·), ψ0 , ψ(·)) satisfies: (i) the Hamiltonian system  ∂H   ˙ = (t, x(t), u(t), ψ0 , ψ(t))  x(t) ∂ψ  ∂H  ˙  ψ(t) (t, x(t), u(t), ψ0 , ψ(t)) =− ∂x

with the Hamiltonian

H(t, x, u, ψ0 , ψ) = ψ0 L(t, x, u) + ψ · ϕ(t, x, u) ;

(3)

(ii) the maximality condition H (t, x(t), u(t), ψ0 , ψ(t)) = sup H (t, x(t), u, ψ0 , ψ(t)) . u∈Ω

Furthermore, the function H (t, x(t), u(t), ψ0 , ψ(t)) is an absolutely continuous function of t and satisfies the equality dH (t, x(t), u(t), ψ0 , ψ(t)) ∂H = (t, x(t), u(t), ψ0 , ψ(t)) . dt ∂t

(4)

Definition 2. A quadruple (x(·), u(·), ψ0 , ψ(·)) satisfying the conditions of Theorem 1 is called a Pontryagin extremal (or just extremal for brevity) for the problem (P ). An extremal is called normal if ψ0 6= 0 and abnormal if ψ0 = 0. Classical Noether’s theorem is formulated for Euler–Lagrange extremals of the basic problem of the calculus of variations. We will formulate our Noether theorem for Pontryagin extremals of optimal control problems. Definition 3. A quantity C(t, x, u, ψ0 , ψ) which is constant along every extremal (x(·), u(·), ψ0 , ψ(·)) of (P ) will be called a conservation law . Definition 4. A quantity C(t, x, u, ψ) which is constant along every abnormal extremal (x(·), u(·), 0, ψ(·)) of (P ) will be called an abnormal conservation law .


3

5

Main Results

Both classical Noether’s theorem for the calculus of variations (cf., e.g., [1], [2], [11], [15], [23]) and respective versions for optimal control (cf. [16], [25], [26]) have been formulated as implications: “invariance implies a conservation law”. Here we shall begin with a weak notion of invariance (5) and formulate a necessary and sufficient condition (§3.1). Afterwards, in order to build conservation laws constructively, we impose a stronger notion of invariance (Def. 5) and derive as corollary (§3.2) a new version of Noether’s theorem where the control variable is explicitly changed by the parameter-transformation. 3.1

A Necessary and Sufficient Condition

The following proposition is a consequence of Definition 3, the Hamiltonian system (i) in the maximum principle and equality (4). Theorem 2. The absolutely continuous function C(t, x, u, ψ0 , ψ) = ψ0 F (t, x, u) + ψ · X(t, x, u) − H(t, x, u, ψ0 , ψ)T (t, x, u) is a conservation law, with H the Hamiltonian (3) associated to the problem (P ), if and only if the equality ¶ µ ∂L d d ∂L T+ ·X +L T − F ψ0 ∂t ∂x dt dt ¶ µ ∂ϕ d d ∂ϕ T+ · X + ϕ T − X = 0 (5) +ψ· ∂t ∂x dt dt holds along every extremal (x(·), u(·), ψ0 , ψ(·)) of (P ). d Proof. By definition, C is a conservation law if and only if dt C = 0 along every extremal. Having in mind Theorem 1, the proof follows by direct computations: d (ψ0 F + ψ · X − HT ) 0= dt d d dF + (ψ · X) − (HT ) = ψ0 dt dt dt dF dX dH dT = ψ0 + ψ˙ · X + ψ · − T −H dt dt dt dt ∂H dX ∂H dT dT dF − ·X +ψ· − T − ψ0 L −ψ·ϕ = ψ0 dtµ ∂x dt ∂t ¶ dt dt ∂L ∂L dT dF = −ψ0 T+ ·X +L − ∂t ∂x dt dt µ ¶ ∂ϕ ∂ϕ dT dX −ψ · . T+ ·X +ϕ − ∂t ∂x dt dt Remark 1. The maximality condition is the only condition of Theorem 1 which is not explicitly used in the proof of Theorem 2. It appears implicitly, however, as long as equality (4) is required.

6

3.2

D. F. M. Torres

Noether Theorem for Optimal Control

The following notion of invariance, admitting an explicit one-parametertransformation of the control variable u, generalize previous notions of invariance used in connection with the Noether theorem. Definition 5. Let hs be a one-parameter family of C 1 mappings, hs : [a, b] × Rn × Ω → R × Rn × Rr , hs (t, x, u) = (T (t, x, u, s), X (t, x, u, s), U(t, x, u, s)) , 0

h (t, x, u) = (t, x, u) ,

(6)

n

∀(t, x, u) ∈ [a, b] × R × Ω .

If for all processes (x(·), u(·)) the following two conditions hold: (i) There exists a function F (t, x, u, s) ∈ C 1 ([a, b], Rn , Ω; R) such that L (t, x(t), u(t)) +

d F (t, x(t), u(t), s) dt = L ◦ hs (t, x(t), u(t))

d T (t, x(t), u(t), s) ; dt

d d X (t, x(t), u(t), s) = ϕ ◦ hs (t, x(t), u(t)) T (t, x(t), u(t), s); dt dt then the problem (P ) is said to be invariant under the transformations hs .

(ii)

Remark 2. The functions T (t, x(t), u(t), s) and X (t, x(t), u(t), s) are assumed to be differentiable in t. Theorem 3. If the Lagrange optimal control problem (P ) is invariant under the family of transformations (6), then ψ0 F (t, x, u) + ψ · X(t, x, u) − H(t, x, u, ψ0 , ψ)T (t, x, u) is a conservation law, where ¯ ∂T (t, x, u, s) ¯¯ , T (t, x, u) = ¯ ∂s ¯s=0 ∂X (t, x, u, s) ¯¯ , X(t, x, u) = ¯ ∂s ¯ s=0 ∂F(t, x, u, s) ¯¯ F (t, x, u) = . ¯ ∂s s=0

Remark 3. For the basic problem of the calculus of variations, and for F ≡ 0, Theorem 3 coincides with the Noether’s theorem in [11]. The triple (T (t, x, u), X(t, x, u), U (t, x, u)) is called the tangent vector field of {hs } and the constructed conservation law is the value of the Cartan differential 1-form w = ψ dx − H dt on the tangent vector field. For a survey of these questions and the role of E. Noether’s results in mechanics see [29].


7

Proof (Theorem 3). Let (x(·), u(·), ψ0 , ψ(·)) be an arbitrary extremal of (P ). By virtue of Theorem 2, we only need to prove that conditions (i) and (ii) of Definition 5 imply the equality (5) along the extremal. Differentiating the relations (i) and (ii) with respect to s, at s = 0 we get: ∂L ∂L ∂L d d F = T+ ·X + ·U +L T ; dt ∂t ∂x ∂u dt ∂ϕ ∂ϕ ∂ϕ d d X= T+ ·X + ·U +ϕ T ; dt ∂t ∂x ∂u dt where all functions are evaluated at (t, x(t), u(t)) and ¯ ∂U(t, x(t), u(t), s) ¯¯ . U (t, x(t), u(t)) = ¯ ∂s s=0

(7a) (7b)

Multiplying (7a) by ψ0 , (7b) by ψ(t), and summing the two equalities, one gets: µ ¶ ∂L ∂L ∂L d d ψ0 T+ ·X + ·U +L T − F ∂t ∂x ∂u dt dt ¶ µ ∂ϕ ∂ϕ d d ∂ϕ T+ ·X + · U + ϕ T − X = 0 . (8) + ψ(t) · ∂t ∂x ∂u dt dt According to the maximality condition (ii) of Theorem 1, the function Mt (s) = ψ0 L (t, x(t), U (t, x(t), u(t), s)) + ψ(t) · ϕ (t, x(t), U (t, x(t), u(t), s)) attains its maximum for s = 0. Therefore ¯ dMt (s) ¯¯ = 0, ds ¯s=0

that is, ψ0

∂L (t, x(t), u(t)) · U (t, x(t), u(t)) ∂u ∂ϕ (t, x(t), u(t)) + ψ(t) · · U (t, x(t), u(t)) = 0 . ∂u

(9)

Using (9) for simplification of (8) one obtains ¶ µ ∂L d d ∂L T+ ·X +L T − F ψ0 ∂t ∂x dt dt µ ¶ ∂ϕ d d ∂ϕ + ψ(t) · T+ · X + ϕ T − X = 0, ∂t ∂x dt dt and the proof is complete. Remark 4. For ψ · X(t, x, u) − H(t, x, u, ψ0 , ψ)T (t, x, u) to be an abnormal conservation law under the family of transformations (6), it suffices that condition (ii) of Definition 5 holds.

8

4

D. F. M. Torres

Examples

It is not difficult to find examples for which Theorem 3 provides conservation laws which are not easily derived from the previously known results. Two such examples follow. Example 1 (n = r = 1). Z b etx(t) u(t) dt −→ min , J [x(·), u(·)] = a

x(t) ˙ = tx(t)u(t)2 ,

u∈Ω.

We claim that ¡ ¢ ψ0 tetx(t) u(t) + ψ(t)x(t) (tu(t))2 + 1

is constant along any Pontryagin extremal (x(·), u(·), ψ0 , ψ(·)) of the problem. Indeed, setting T = e−s t , X = es x , U = es u , F ≡ 0, it follows from Theorem 3 that ψx + Ht is a conservation law. The conservation law holds independently of the control set Ω, and can be viewed as a necessary optimality condition. Example 2 (n = 3, r = 2). Z b 2 2 (u1 (t)) + (u2 (t)) dt −→ min , a   x˙1 (t) = u1 (t) x˙2 (t) = u2 (t)   2 x˙3 (t) = u1 (t) (x2 (t)) . Choosing F ≡ 0 and

T = e−2s t , X1 = e−s x1 , X2 = e−s x2 , X3 = e−3s x3 , U1 = es u1 , U2 = es u2 , one concludes from Theorem 3 that ψ1 x1 + ψ2 x2 + 3ψ3 x3 − 2Ht is a conservation law for the Pontryagin extremals (x1 (·), x2 (·), x3 (·), u1 (·), u2 (·), ψ0 , ψ1 (·), ψ2 (·), ψ3 (·)) . In previous results (cf. [16,25]) no time transformation appears in the autonomous case. Example 2 is autonomous but transformation of the timevariable is required in order to obtain the conservation law.


9

Acknowledgments I would like to thank Emmanuel Tr´elat for the stimulating conversations during the workshop; Andrey Sarychev for the suggestions regarding improvement of the text.

References 1. Alekseev V. M., Tikhomirov V. M., Fomin S. V. (1987) Optimal Control. Plenum, New York 2. Arnold V. I. (1987) M´etodos Matem´ aticos da Mecˆ anica Cl´ assica. Mir, Moscovo 3. Arnold V. I. (1988) Geometrical Methods in the Theory of Ordinary Differential Equations. Springer-Verlag, New York 4. Baez J. C., Gilliam J. W. (1994) An Algebraic Approach to Discrete Mechanics. Lett. Math. Phys. 31, 205–212 5. Blankenstein G., van der Schaft A. J. (1999) Symmetry and Reduction in Implicit Generalized Hamiltonian Systems. Memorandum No. 1489, Faculty of Mathematical Sciences, University of Twente, The Netherlands 6. Cesari L. (1983) Optimization – Theory and Applications. Springer-Verlag, New York 7. Clarke F. H. (1993) An Indirect Method in the Calculus of Variations. Trans. Amer. Math. Soc. 336, 655–673 8. Clarke F. H., Vinter R. B. (1985) Regularity Properties of Solutions to the Basic Problem in the Calculus of Variations. Trans. Amer. Math. Soc. 289, 73–98 ¨ 9. Erdmann G. (1877) Uber Unstetige L¨ osungen in der Variationsrechnung. J. f¨ ur reine und angew. Math. 82, 21–30 10. Euler L. (1744) Methodus Inveniendi Lineas Curvas Maximi Minimive Proprietate Gaudentes, Sive Solutio Problematis Isoperimetrici Lattisimo Sensu Accepti. Bousquet, Lausannae et Genevae, E65A.O.O. Ser. I 24 11. Gelfand I. M., Fomin S. V. (2000) Calculus of Variations. Dover, New York 12. Gilliam J. W. (1996) Lagrangian and Symplectic Techniques in Discrete Mechanics. Ph.D. dissertation, Univ. California Riverside 13. Grizzle J. W., Marcus S. I. (1985) The Structure of Nonlinear Control Systems Possessing Symmetries. IEEE Trans. Automat. Control 30, 248–258 14. Heinricher A. C., Mizel V. J. (1988) The Lavrentiev Phenomenon for Invariant Variational Problems. Arch. Rational Mech. Anal. 102, 57–93 15. Jost J., Li-Jost X. (1998) Calculus of Variations. Cambridge University Press, Cambridge 16. Jurdjevic V. (1997) Geometric Control Theory. Cambridge University Press, Cambridge 17. Jurdjevic V. (1999) Optimal Control, Geometry, and Mechanics. In: Baillieul J., Willems J. C. (Eds.) Mathematical Control Theory, Springer, New York, 227–267 18. Nijmeijer H., van der Schaft A. J. (1990) Nonlinear Dynamical Control Systems. Springer-Verlag, New York 19. Noether E. (1918) Invariante Variationsprobleme. Nachr. Akad Wissen. G¨ ottingen, Math.-Phys. Kl. II, 235–257 (see Transport Theory and Stat. Phys. 1 (1971), 186–207 for an English translation)

10

D. F. M. Torres

20. Olver P. J. (1993) Applications of Lie Groups to Differential Equations. Springer-Verlag, New York 21. Pontryagin L. S., Boltyanskii V. G., Gamkrelidze R. V., Mischenko E. F. (1962) The Mathematical Theory of Optimal Processes. John Wiley, New York 22. Respondek W. (1982) On Decomposition of Nonlinear Control Systems. Syst. Contr. Lett. 1, 301–308 23. Rund H. (1966) The Hamilton–Jacobi Theory in the Calculus of Variations, Its Role in Mathematics and Physics. D. Van Nostrand Co., Ltd., London–Toronto, Ont.–New York 24. Sarychev A. V., Torres D. F. M. (2000) Lipschitzian Regularity of Minimizers for Optimal Control Problems with Control-Affine Dynamics. Appl. Math. Optim. 41, 237–254 25. Sussmann H. J. (1995) Symmetries and Integrals of Motion in Optimal Control. In: Fryszkowski A., Jakubczyk B., Respondek W., Rzezuchowski T. (Eds.) Geometry in Nonlinear Control and Differential Inclusions, Banach Center Publications 32, Mathematics Institute of the Polish Academy of Sciences, Warsaw, Poland, 379–393 26. Torres D. F. M. (2000) On the Noether Theorem for Optimal Control. Cadernos de Matem´ atica CM00/I-13, Dep. Mat., Univ. Aveiro, Portugal. Accepted to European Control Conference ECC 2001. To appear in the proceedings. 27. van der Schaft A. (1981/82) Symmetries and Conservation Laws for Hamiltonian Systems With Inputs and Outputs: A Generalization of Noether’s Theorem. Systems Control Lett. 1, 108–115 28. van der Schaft A. (1987) Symmetries in Optimal Control. SIAM J. Control and Optimization 25, 245–259. 29. Yemelyanova I. S. (1999) Some Properties of Hamiltonian Symmetries. European women in mathematics (Trieste, 1997), Hindawi Publ. Corp., Stony Brook, New York, 193–203