Accelerated Convergence in Newton's Method

0 downloads 0 Views 373KB Size Report
f"(a) $ 0, and consider the function F(x) := f (x)g(x), where the function g(x) is to be ... and after dropping constant factors we obtain x3-5. F2(X)=. vX4+5x ...
Accelerated Convergence in Newton's Method Author(s): Jurgen Gerlach Source: SIAM Review, Vol. 36, No. 2 (Jun., 1994), pp. 272-276 Published by: Society for Industrial and Applied Mathematics Stable URL: http://www.jstor.org/stable/2132465 Accessed: 29/11/2010 06:49 Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you may use content in the JSTOR archive only for your personal, non-commercial use. Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at http://www.jstor.org/action/showPublisher?publisherCode=siam. Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed page of such transmission. JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected].

Society for Industrial and Applied Mathematics is collaborating with JSTOR to digitize, preserve and extend access to SIAM Review.

http://www.jstor.org

?

SIAM REVIEW Vol.36, No. 2, pp.272-276,June1994

1994SocietyforIndustrial andAppliedMathematics 007

ACCELERATED CONVERGENCE IN NEWTON'S METHOD* JURGENGERLACHt ofthefunction whoserootsaretobe determined Abstract.Newton'sMethodis basedona linearapproximation to increase is knowntoconverge quadratically. In a procedure algorithm takenat thecurrent point,andtheresulting thetargetfunction in sucha waythatNewton'sMethodappliedto the therateof convergence theauthormodifies rateofconvergence. modified function willyielda faster rateofconvergence Key words.Newton'smethod, AMS subjectclassifications. 65-01,65B99,65H05

convergence forNewton'smethod. withachievinghigher-order Thispaperis concerned and it is based on a linearapproximation Newton'smethoditselfconvergesquadratically, at thecurrent iterate Xk: (tangent line)to thefunction f(x) The equationl(Xk+l) (1l)

=

1(x)

= f(Xk) + f'(Xk)(X - Xk).

0 leadstothefamiliar Newtonscheme Xk+1

=

Xk

f (Xk )

f'(Xk)

numerical analysistextsfrequently suggestusing InordertoaccelerateNewton'smethod, to f atXk;see,e.g., [2] or [3]. In thisnotewe takea different a higher-order approximation does Newton'smethod perform particularly approach,and ask: For whatclass offunctions havebeenidentified we ask: How can we modify oncethesefunctions well? And,secondly, a givenfunction insucha waythattheorderofconvergence is increased? Throughout we assumethat 1. f is sufficiently manytimesdifferentiable; 2. f has a simplerootatx = a, i.e., f(a) = 0 and f'(a) $ 0; andthat to a will close to a so thatconvergence 3. theinitialapproximation xo is sufficiently occur. questionis givenbythefollowing. The answerto ourfirst above letus assumethatf"(a) = f"'(a) = THEOREM 1. In additionto thehypotheses = f(m-l) (a) = 0, andthat theNewton by(1) sequence{Xk} defined f(m) (a) 5$0. Then yields (2)

IXk+1

-al, < CIxk

-aim

forsomeconstantC. can be foundas an exercisein thebookby Dennisand Schnabel[2]; its This theorem saysthatthemoref lookslikea linearfunction, proofis includedintheAppendix.Itroughly will converge.Our nextgoal is to molda givenfunction thefastertheNewtoniterations butitlooksnearlylinearin a intoa newone in sucha waythattherootsremainunchanged, ofNewton'smethodwillbe accelerated. oftherootso thattheconvergence neighborhood Beforewe presenta generalresult,letus beginwiththecase f (a) = 0, f'(a) > 0, and g(x) is to be F(x) := f (x)g(x), wherethefunction f"(a) $ 0, and considerthefunction August10, 1993. 14, 1993;acceptedforpublication *ReceivedbytheeditorsJanuary Box 6942,Radford, VA 24142. andStatistics, RadfordUniversity, ofMathematics tDepartment

272

273

CLASSROOM NOTES

determined We haveF(a) = 0 and,ifg(a) :$ 0, we getF'(a) :$ 0. Moreover, subsequently. sincef (a) = 0, F"(a) = 2f'(a)g'(a) + f"(a)g(a). we onlywishto satisfy theequationF"(a) = 0 at thesinglepointx = a, we turn Although itintoa differential equationforthefunction g: I g'(x) = - -f,(x) g(x). 2 f'(x) we obtaing(x) = C/ jf`(x as a generalsolution,and withC = 1 the Upon integration F becomesF(x) = f (x)/ f'(x). Newton'smethodappliedto F(x) yields function f (Xk) f (Xk)

F(Xk)

f'(xk)2 - ' f(xk)f'"(xk)

F'(Xk)

Thisis an exampleofan iteration schemewithcubicconvergence, andcan be tracedbackto Halleyin 1694(see [4]). Let us nowturntoa generalmethod, andpresent theresultin thefollowing form. THEOREM2. Let f(a) = 0, f'(a) > 0, f"(a) = *. = f(m l)(a) = 0, and let f(m)(a) :$ 0. Thenthefunction (4)

f x)

f(x)

satisfiesF(a) = 0, F'(a) > 0, F"(a) = F(m)(a)= 0. Moreoverform > 3 we have F(m+l)(a) =-1 f(m+1)(a)/ m f'(a). of m Againwe referto theAppendixfortheproofofthisassertion.The determination toknowthattheperformance ofNewton's maynotalwaysbe easy.However,itis reassuring methodforF(x) = f(x)/ m f`(x) will neverbe worsethantheone foundforf(x) itself, ofthechoiceofm. The Newtoniterations regardless basedon thefunction F aregivenby -

f (Xk) f (Xk)

F(Xk)

F'(Xk)

9 f (xk) f"(xk)

f'(xk)2-

inanalogyto (3). Theorem2 providesa procedure toincreasetheorderofconvergence ofNewton'smethod :$ = > If 2 at least one: set else the smallest 0 m m withf(m-l)(a) = 0 by f"(a) 2, identify = and f(m)(a) :$ 0. Now formthefunction and Newton'smethod F(x) f(x)/ ;fj7ix-, will convergeto therootx = a at a rateof m + 1 or better.Repeatedapplicationwill furnish an algorithm togenerate an iterative schemethatwillconverge toa rootoff (x) inany desiredorder,at theexpenseoftheuse ofhigher-order of f, (and someawkward derivatives computations). Example. Compute 5 as a rootoff (x) = x3-5, andletus assumex > 0 throughout. A first ofTheorem2 withm = 2 leadsto application fix

f(x)

vtx-7 f'(x

_

x-

5

1

2

\X

We maynowapplyTheorem2 withm = 3 to thefunction constant Fl, and afterdropping factors we obtain F2(X)=

x3-5

vX4+5x

274

CLASSROOM NOTES

ofrootsinF1andF2defeatsthepurposeoffinding glanceitappearsthattheoccurrence Atfirst Newtoniterations to computeX. But,similarto (5), a look at theresulting an algorithm in theprocess,as showninTable 1. ofrootsarenotrequired showsthatcomputation TABLE 1

Newtonscheme

Function of(x)

Xk+1

= Xk-

Fl(x)

Xk+1

=Xk

Xk(X'r-5)

-2+5 3xk(2x3+5)(x3 -5) kOx+8x k+25

= Xk -

Xk+1

F2(X)

k

Thefirst threeiterations we usedxO = 2 as aninitialapproximation. experiments In numerical areshownbelowinTable2. absoluteerrors withtheirrespective fortherespective functions TABLE 2

Fl(x)

F2(X)

X1 1.75 error 4.0 x 10-2

1.714... 4.3 x 10-3

1.7103... 3.7 x 10-4

1.7108... X2 error 9.1 X 10-4

1.709975964... 1.8 x 10-8

1.7099759466766982... 1.2 x 10-15

1.7099764 X3 error 4.8 x 10-7

1.709975946676696989 ... 1.5 x 10-60 1.4 x 10-24

f(x)

ofTheorem2 withm = 1 can be used In closingitshouldbe pointedoutthata variation rootat in thecase ofmultiple roots,i.e.,iff has a multiple to restore quadraticconvergence F(x) = f (x)/f'(x) has a simplerootatx = a. See [1] fordetails. x = a, thenthefunction twoproofs. theremaining Appendix.We present Proofof Theorem1. SupposeXkhas alreadybeen computed.By Taylor'stheorem, vanishatx = a, thereexistconstants withthefactthatf andmanyofitsderivatives together that a and such and between Xk 4j tO f(xk) = f'(a)(xk-a)+ f'(xk) = f'(a) +

fM) (tO)(Xk mr!

f(M) (6) J

(xk -a

-

and

a)m

1

1) we obtain intoNewton'sformula oftheseexpressions hold.Uponsubstitution Xk+I -

a

=

f(

Xk -a-

=

) =

f'(Xk)

f(m) (6)

1 f'(xk)

(mn-

l

rnf(m)(~i)

(m-i)! -

f'(Xk) _

f(m)(tO)

m!

f(m)(~o) (Xk

f(Xk)Mn!

{f '(xk)(xk

- a)mm

-

(xk-a)m

a)

-

f(Xk)}

275

CLASSROOM NOTES

Now we fixa neighborhood N = [a - 1,a + 1] of a forsomesuitablysmall1, so thatthe inequalitiesIf'(x) I > co > 0 and If(m)(x) I < cl holdtrueon N forsomeconstants co and then If Xk E N, cl. ) - f(m))

mf(m)(

| < MCI + Cl m!c0

f'(xk)m!

=

C

andthus

-al < Clxk-aLm. IXk+1

(2)

Ifnecessary, we decrease1suchthat1 < 1,and m-lC < 1 aresatisfied. ThenXk E N implies < < a a < and im from that the -a -a 1, (2) it follows C 1, i.e., sequenceremainsin lXk IXk+I N andtheestimate(2) holdsforall subsequent termsofthesequence. O ProofofTheorem2. The case m = 2 was outlined justbeforethetheorem.We assume m > 3 andwe proceedintwosteps. 1 = g(x)mf'(x), Step1. Wedefinethefunctiong(x):= 1/mf'(x). Bydefinitionwehave differentiation andimplicit yields 0 = mg(X)mlg (x)f (x) + g(X)mf" (x), andthus 0 = mg'(x)f'(x) + g(x) f"(x).

(6)

#0, impliesg(a)

withf '(a) f"(a) = 0, together (6) leadsto O

+ (m(k- 1)

mg(k) (x) f '(x)

=

+

(7)

(k-1)1

=

0. Further of equation differentiation (X) *X+

+ 1)g(kI)(X)f

(j + m(k- i))g(k-i)(x)f(i+l)(x)

+ (m+ k -

1)gl(X)f(k)(X)

*

+

+ g(X)f(k+l)(x)

Sincef"(a) = * f(m-l)(a) = 0, equation (7) implies g(k)(a) 0 for1 < k < m - 2. Fork = m - 1 andx = a equation(7) reducesto -

mg(ml1)(a)

f'(a)

f(m) (a)

= 0,

f(m + ) (a)

= 0.

+ g(a)

whilefork = m andx = a we obtain mg(m) (a) f (a)

+ g(a)

oftheproduct rule Step2. We nowinvestigate F(x) := f (x)g(x). Repeatedapplication yields F'(x) = f(x)g'(x) + f'(x)g(x) F"(x) = f (x)g"(x) + 2f'(x)g'(x) + f"(x)g(x)

F(k) (X)

-

f(X)g(k)

+

(k

(X) + kf (X)g(k

l)(X)

f(j) (X)g(k-j) (x) +

+kf(kl)(x)gf(x)

+ *

*

+ f(k)(x)g(x),

CLASSROOM NOTES

276

foranyfunction g(x). By definition F(a) = 0, andifwe chooseg(x) as beforeanduse the from we for the of F atx = a results derivatives Step 1, obtain F'(a) = f'(a)g(a) = f'(a)l-/1m > 0 F"(a) = 2f'(a)g'(a) = 0 F(k)(a) = kf (a)g(k-1)(a)

=

0

) ff(a)g(m-2)(a) = 0

-

(m -

F(m)(a)

-

mfj(a)g(m-l)(a) + f(m)(a)g(a) = 0

F(m+l)(a)

-

(m + 1)f(a)g(m)(a) + f(m+l)(a)g(a)

=

(

F(ml)(a)

whichprovesthetheorem.

m-

+ 1) f(m+l)(a)g(a)

=

f_

m

f'(a)

O REFERENCES

Boston,1989. Weber,Schmidt, 4thed.,Prindle, Analysis, [1] R. L. BURDEN AND J.D. FAIRES, Numerical and NonlinearEquaOptimization forUnconstrained [2] J.E. DENNIS AND R. B. SCHNABEL, NumericalMethods NJ,1983. Hall,EnglewoodCliffs, tions,Prentice New York,1989. Mathematik, Springer-Verlag, [3] G. HAMMERLIN AND K.-H. HOFFMANN, Numerische New York,1979. Vol.1,Numerical Analysis, Birkhauser-Verlag, [4] J.TODD,Basic NumericalMathematics,