Asymptotic Convergence Algorithms - Science Direct

Available online at www.sciencedirect.com

ScienceDirect Procedia Engineering 181 (2017) 920 – 923

10th International Conference Interdisciplinarity in Engineering, INTER-ENG 2016

Asymptotic Convergence Algorithms Diana M˘arginean Petrovai∗ “Petru Maior” University of Târgu Mures N. Iorga street nr. 1, 540088, Romania

Abstract The subject of general theory of convergence is to study, if possible, the behavior of a class of algorithms, particularly that the algorithms differ in most cases by details of implementation. This leads naturally to a general pattern in which algorithms or classes of algorithms are represented by multifunctions, i. e. the maps A : Rn → P(Rn ). An algorithm is defined by a map A : Rn → P(Rn ), which associates to each point xk already know a new point xk+1 = A(xk ), k ∈ N. The asymptotic convergence describes how fast the sequence {xk } could arrive to an optimal solution if exists. c 2017 by Elsevier 2017Published The Authors. Published © by Elsevier Ltd. This is an openLtd. access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of the organizing committee of INTER-ENG 2016. Peer-review under responsibility of the organizing committee of INTER-ENG 2016

Keywords: algorithm; sequence; compact interval; asymptotic convergence algorithm.

1. Ray of convergence and average order of convergence of a sequence of real numbers ∗

−r | ∗ Let {rk }k≥0 ⊆ R, such that lim rk = r∗ . We will find {p} p≥0 , for which lim |r|rk+1 ∗ p < ∞. Then p = sup{p} is called k −r | k→∞

k→∞

the order of convergence of the sequence {rk }k≥0 . −r∗ | We denote lim |r|rk+1 ∗ p = β < 1, where β is called ray of linear convergence of sequence. k −r | k→∞

If β = 0 it follows thatsuperior liniar convergence of the sequence. 1 We denote by p˜ = inf p ≥ 1 lim |rk − r∗ | pk < +∞ and we call p˜ average order convergence of sequence. k→∞

We have lim |rk − r∗ | k = β < 1 [2], [9]. 1

k→∞

We consider in the following the nonlinear programming problem (1)

inf{ f (x) | x ∈ X}, f : Rn → R, f ∈ C1 (Rn ).

Let be a map E : X → R, X ⊆ Rn . Every x ∈ X we associate E(x) ∈ R and every xk we associate E(xk ) ∈ R and rk := E(xk ), for any k ∈ N. ∗

Corresponding author. Tel.: +40-265-262275; fax: +40-265-262275. E-mail address: [email protected]

1877-7058 © 2017 Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license

(http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of the organizing committee of INTER-ENG 2016

doi:10.1016/j.proeng.2017.02.487

921

Diana Mărginean Petrovai / Procedia Engineering 181 (2017) 920 – 923

The map (E(x) = f (x)) (often) or E(x) = x − x∗ , hence E(x∗ ) = 0. Example 1. (gradient method) [9], [15] Let be inf{ f (x) | x ∈ Rn }, f : Rn → R, f ∈ C1 (Rn ). Let x0 ∈ Rn and we consider f (x0 + αy) the map depending on y and α > 0, y ∈ Rn . Indeed f (x0 + αy) = f (x0 ) + αyT ∇ f (x0 ) + o(α) ⇒ f (x0 , y) = lim

α→0

(1)

f (x0 + αy) − f (x0 ) = yT ∇ f (x0 ) α

inf{yT ∇ f (x0 ) | y = 1}

(2)

sup{yT ∇ f (x0 ) | y = 1}.

Obviously yT ∇ f (x0 ) ≤ y · ∇ f (x0 ) (Cauchy-Bunyakovsky-Schwarz inequality). For y with y = 1 we have − ∇ f (x0 ) ≤ yT ∇ f (x0 ) ≤ ∇ f (x0 ) . Therefore y = − ∇ f 1(x0 ) · ∇ f (x0 ) is solution for problem (1) and y = ∇ f 1(x0 ) · ∇ f (x0 ) is solution for problem (2). It follows that the direction of the fastest decreasing in the neighborhood of x0 is −∇ f (x0 ) and the direction of the fastest increasing is ∇ f (x0 ). If f ∈ C2 (Rn ) it follows that 1 f (x) = f (x0 ) + (x − x0 )T ∇ f (x0 ) + (x − x0 )T · H f (x0 ) · (x − x0 ) + R2 f (x) (Taylor’s formula). 2 From this reason can be take function E(x) a quadratic function f (x), where f : Rn → R, f (x) = pT x +

1 T x Cx, 2

with C symmetric and positive defined. Proposition 1. Let f, g : X → R∗ , continue functions, which verify the following conditions: there exists a, b > 0 such that 0 ≤ a f (x) ≤ g(x), for any x ∈ X. If the sequence {xk } linear converge to x∗ with average rate β with respect to f , then {xk } also converge to x∗ with average ray β with respect to g. Proof. We have from definition 1k 1 1 1 (g(x)) k = lim (g(x)) k k→∞ k→∞ a k→∞ 1k 1 1 1 1 k β = lim ( f (x)) ≥ lim (g(x)) k = lim (g(x)) k k→∞ k→∞ b k→∞ 1

β = lim ( f (x)) k ≤ lim

(1) (2)

Therefore from (1) and (2) it follows that 1

1

β = lim ( f (x)) k = lim (g(x)) k k→∞

k→∞

2. Nonlinear programming problem Let (1) inf{ f (x) | x ∈ X}, R(x) := {y | y ∈ Rn , (∃) α > 0, such that x + αy ∈ X, for any α ∈ [0, α]}. From x0 ∈ X we construct the sequence {xk } as xk+1 = xk + αk yk , where yk ∈ R(xk ), yk 0, and αk > 0 usually choose, so that xk + αyk ∈ X, α ∈ [0, α]. The value of αk is called step size. We consider for x ∈ X, y ∈ R(x), y 0 the map f (x + αy) : [0, α] → R and map

922


S : R2n → P(Rn ), S (x, y) = {x ∈ Rn | (∃) α ∈ [0, α] such that z = x + αy and f (z) = inf ( f (x + αy))} (search α≥0

algorithm exactly) For δ, ε > 0 let S δ (x, y) = {z ∈ Rn | (∃) α∗ ∈ [0, δ] such that z = x + α∗ y, and f (z) = inf ( f (x + αy))}, (search algorithm exactly α∈[0,δ]

on the compact interval) S ε (x, y) := z ∈ Rn | (∃) α∗ ≥ 0, such that z = x + α∗ y and f (z) < inf f (x + αy) + ε α≥0

(search algorithm approximate on the compact interval). ε δ S (x, y) = z ∈ Rn | (∃) α∗ ∈ [0, α], such that z = x + α∗ y and f (z) < inf f (x + αy) + ε . α∈[0,α]

3. Asymptotic convergence Let G : Rn → P(R2n ), G(x) = {(x, y) | y ∈ R(x)}. From above xk+1 = A(xk ), for any k ≥ 0, where A = S ◦ G. Theorem 1. Let be f continuous and (x, y) such that x ∈ X, y ∈ R(x), y 0. Suppose that f (x + αy) has points of global minimum on interval [0, +∞). Then S is closed in (x, y) [3]. Proof. Let be the sequences {xk } and {yk } such that xk → x, yk → y, {xk } ⊂ X, yk ∈ R(xk ), for any k ≥ 0 and zk ∈ S (xk , yk ), for any k ≥ 0, zk → z. We want to show z ∈ S (x, y). For any k ≥ 0, there exists αk ≥ 0, such that zk = xk + αk yk and f (zk ) = inf{ f (xk + αyk ), α ≥ 0}. Since y 0 it follows that y ≥ 0, hence yk → y it follows that there exists k0 such that yk > 0, for any k ≥ k0 . k k ∗ For any k ≥ k0 we put write zk − xk = αk yk and we obtain zk − xk = αk yk , αk = z y−x → z−x k y = α . Hence z = x + α∗ y, α∗ ≥ 0. Since f (zk ) ≤ f (xk + αyk ) for any α ≥ 0 and f is continuous it follows that f (z) ≤ f (x + αy) for any α ≥ 0, hence f (z) = inf{ f (x + αy), ∀ α ≥ 0}, therefore z ∈ S (x, y). 4. Armijo’s rule Let f : Rn → R, f ∈ C1 (Rn ), y ∈ R(x), y 0 and y(α) = f (x + αy), α ≥ 0, x, y fixed, ε ∈ (0, 1). Then there exists α ≥ 0, such that ε ≤ y(0) + (1 − ε)αy (0) ≤ 1 − ε. f (x + αy) − f (x) 1 ≤ 1 − ε≤ ε, ε ∈ 0, (Goldstein’s formula). αyT ∇ f (x) 2 f (x) ≤ 1 − ε, ε ∈ (0, 1) . Let G(x, y) = z | (∃) α ≥ 0, such that z = x + αy, ε ≤ f (x+αy)− αyT ∇ f (x) Theorem 2. If f ∈ C1 (R), (x, y) as y 0 then G is closed in (x, y). Proof. (main result) Let be the sequences {xk } and {yk } such that xk → x, yk → y, zk ∈ G(xk , yk ), zk → z. We want to show z ∈ G(x, y). For any k ∈ N, there exists αk ≥ 0, such that zk = xk + αk yk , and ε≥

f (xk + αk yk ) − f (xk ) ≤ 1 − ε. αk yTk ∇g f (xk ) ϕ(xk ,yk ,αk )


Since y 0 it follows that there exists k0 such that for any k ≥ k0 , yk > 0. k k → α∗ ≥ 0 it follows that z = x + α∗ y. From αk = z y−x k k k Since ε ≤ ϕ(x , y , αk ) ≤ 1 − ε it follows that ε ≤ ϕ(x, y, α∗ ) < 1 − ε. Therefore z ∈ G(x, y) [2]. In conclusion the asymptotic convergence algorithms generally applies to algorithms for nonlinear problems without restrictions, such as the fastest decreases method and conjugate directions methods.

References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15]

W. I. Zangwill, Nonlinear Programming, Prentice Hall, Englewood Cliffs, N. J., 1969. L. Armijo, Minimization of functions having Lipschitz continuous first partial derivatives, Pacific Journal of Mathematics, 16 (1966), 1–3. De Wengu Sun, Ya-Xiong Yuan, Optimization theory and methods: Nonlinear programming, Springer, 2006. A. S¸tef˘anescu, C. Zid˘aroiu, Cercet˘ari operat¸ionale, Editura Didactic˘a s¸i pedagogic˘a, Bucures¸ti, 1981. C. Zid˘aroiu, Programare dinamic˘a discret˘a, Editura Didactic˘a s¸i pedagogic˘a, Bucures¸ti, 1975. S¸. Cruceanu, Teoria optimiz˘arii sistemelor mari, Editura Didactic˘a s¸i pedagogic˘a, Bucures¸ti, 1975. M. Molit¸a, C Zid˘aroiu, Matematica organiz˘arii, Editura Didactic˘a s¸i pedagogic˘a, Bucures¸ti, 1975. C. Udris¸te, E. T˘an˘asescu, Minime s¸i maxime ale funt¸iilor reale de variabile reale, Editura Tehnic˘a, Bucures¸ti, 1980. W. Rudin, Principles of mathematical Analysis, Mc Graw-Hill, 1976. Y. Nesterov, Introductory lectures on convex optimization, A Basic Course, Kluwer, 2004 D. G. Luenberger, Linear and nonlinear programming, Kluwer, 1994. T. Gal, Postoptimal analysis, prametric programming and related topics, de Gruyer, 1995. S. Boyd, L. Vondenberghe, Convex optimization, Cambridge Universiry Press, 2004. D. P. Bertsekas, Nonlinear Programming, Athena Scientific, 1999. I. Marusciac, Metode de rezolvare a problemelor de programare neliniar˘a, Editura Dacia, Cluj-Napoca, 1973.

923