Numerical Methods for Ordinary Differential Equations

CHAPTER 1

Numerical Methods for Ordinary Differential Equations In this chapter we discuss numerical method for ODE . We will discuss the two basic methods, Euler’s Method and Runge-Kutta Method. 1. Numerical Algorithm and Programming in Mathcad 1.1. Numerical Algorithm. If you look at dictionary, you will the following definition for algorithm, 1. a set of rules for solving a problem in a finite number of steps; 2. a sequence of steps designed for programming a computer to solve a specific problem. A numerical algorithm is a set of rules for solving a problem in finite number of steps that can be easily implemented in computer using any programming language. The following is an algorithm for compute the root of f (x) = 0, Input f , a, N and tol . Output: the approximate solution to f (x) = 0 with initial guess a or failure message. • Step One: Set x = a • Step Two: For i=0 to N do Step Three - Four Step Three: Compute x = x − ff0(x) (x) Step Four: If f (x) ≤ tol return x • Step Five return ”failure”. In analogy, a numerical algorithm is like a cook recipe that specify the input — cooking material, the output—the cooking product, and steps of carrying computation — cooking steps. In an algorithm, you will see loops (for, while), decision making statements(if, then, else(otherwise)) and return statements. • for loop: Typically used when specific number of steps need to be carried out. You can break a for loop with return or break statement. 1

2 1. NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS

• while loop: Typically used when unknown number of steps need to be carried out. You can break a while loop with a break statement or a return statement. • if – else(otherwise): Used when condition(s) must meet before carrying out some steps. The else(otherwise) provide alternative steps if any.

1.2. Programming in Mathcad. To translate a numerical algorithm into a program in Mathcad , you need to define a function name with function arguments and function body. The function name can be any sequence of alphabetic letter begin with an English letter, such any ”myFunction”, ”Myfunction1” etc. The function arguments specify the input to the function, typically the items specified in the input statement of an algorithm and is an comma(,) separated list enclose by () after function name, such as myFunction(f, a, N, tol), myOne(a,b). Notice the function argument is optional, you can define a function with out argument as myFunction(). The function body implements the step section of an algorithm and the output statement. In Mathcad the function body is on the right side of := with vertical bars as grouping symbol. The entire function body is considered as a group, inside, it might has many subgroups. For the algorithm of finding f (x) = 0, there is an subgroup that contains step three and four, as shown in the following screen shot, the screen shot also shows how to get the programming toolbar from the math toolbar.

Figure 1. mySolver and Programming toolbar

1. NUMERICAL ALGORITHM AND PROGRAMMING IN Mathcad

3

To enter code in Mathcad , (1) First enter the function name with arguments list and assignment operator :=, with sequence of typing, mySolver(f,a,N,tol): (2) Click the Add Line button or press ] to get a vertical bar and two (you should add more by press ] several times). (3) enter x and hold [Shift] type ] to get the local assignment ← or click on the programming toolbar, at the type a. Notice x ← a reads as ”assign a to x.” (4) Click for on the programming menu or [Shift][Ctrl][’] to get for ∈ in after for enter i, in after ∈, enter ”1;N”, and in under for, press left bracket ] to get a vertical line and several . (5) To get if operator, you need either click on the toolbar or press [Shift][Ctrl][ ] ]. you will get if enter condition in after if and other statements in before if. (6) To get return operator, you need either click on the toolbar or press [Shift][Ctrl][ ], you get return in type any information you want to return.

After defining a function, we call it with concrete input data, as shown in the screen shot, that call the mySolver function to find zero for both f (x) = x2 − 1, with initial guess x = 2 and g(t) = t3 − 3t2 − 2t, with initial guess t = −2. Notice there is two ways to call a function, one way is to define all arguments before calling the function, another is to define some arguments and call the function with concrete values for the undefined arguments. As in all programming language, Mathcad allows you to call one function from within another. For example, we could create a function called myDerivative which will compute derivative of a given function f (x) at a given number x and step size h > 0 using the forward (x) difference formula: f 0 (s) ≈ f (x+h)−f . The following screen shot shows h the call of myDerivative inside mySolver,

Figure 2. Call another function


The algorithm for this function is very simple (here, we break down the computation in three steps), Input f , x, and h . Output: the approximation to f 0 (x) . • Step One: Set w = f (x + h) • Step Two: Set d = f (x) • Step Three: return w−d h 2. Euler’s Method 2.1. Euler’s Method. Euler’s method is the simplest method in find approximate solutions to first order equations. From the forward difference formula f (x + h) − f (x) f 0 (x) ≈ , h we have (1)

f (x + h) ≈ f (x) + f 0 (x)h

Now if x0 (t) = f (t, x) is a first order differential equations, apply (1), we have x(t + h) ≈ x(t) + f (t, x(t))h. Suppose we want to find approximate solution over interval [a, b] with initial value x(a) = x0 , we divided the interval into n subintervals each with length h = b−a , the ends of the subintervals is t0 = a, t1 = n a + h, t2 = a + 2h, · · · , tn = a + nh = b. Start with x0 we can compute x1 = x0 + f (t0 , x0 )h x2 = x1 + f (t1 , x1 )h .. .xn = xn−1 + f (tn−1 , xn−1 )h Example 2.1. Find approximate to x(1) if x0 (t) = t2 −ex sin(t), x(0) = 1 with h = 0.25. Solution Here the interval is [0, 1], so a = 0, b = 1. Since h = = 0.25 we have n = 4 and we need to compute x1 , x2 , x3 , x4 starting with x0 = x(0) = 1. b−a n

x1 x2 x3 x4

= x0 + f (t0 , x0 )h = 1 + f (0, 1) ∗ 0.25 = 0.15853 = x1 + f (t1 , x1 )h = 0.15853 + f (0.25, 0.15853) ∗ 0.25 = 0.01833 = x2 + f (t2 , x2 )h = 0.01833 + f (0.5, 0.01833) ∗ 0.25 = 0.23811 = x3 + f (t3 , x3 )h = 0.23811 + f (0.75, 0.15853) ∗ 0.25 = 0.30128

So the approximation to x(1) is x(1) ≈ x4 = 0.30128. Also x(0.25) ≈ x1 = 0.15853, x(0.5) ≈ x2 = 0.01833, x(0.75) ≈ x3 = 0.23811. a

2.

EULER’S METHOD

5

2.2. Mathcad implementation of Euler’s Method. First, we have the following simple algorithm for the Euler’s method, Input f , a, b, x0 n. Output: the approximate solution to x0 = f (t, x) with initial guess x0 over interval [a, b]. • Step One: Initialization Set h = b−a n Set x0 = x0 Set t0 = a • Step Two: For i=1 to n do Step Three Step Three: Set xi = xi−1 − f (ti−1 , xi−1 ) ∗ h Set ti = ti−1 + h • Step Four return x. Notice, algorithm return an array of values, the ith element of the return array is an approximations of x(t) at t = a + ih. The following screen shot shows Mathcad code for implementing the algorithm. Notice, we also create a function tMesh(a, b, N) to compute the ending of subinterval for graphing purpose.

Figure 3. Mathcad code for Euler’s Method Notice the line to line corresponding between the Mathcad and the algorithm. Since Mathcad programming language is a scripting language, the translation between algorithm and code is straight forward, and you don’t need to worry about the variable type, io, etc. Also, without explicit return statement, the result of last is, by default, will be returned.


The next screen shot shows a call to the myEuler and tMesh for the equation x0 = t2 x + t2 sin(t3 ). It also shows the graph of approxi3 mate solution comparing with the exact solution x(t) = − 10 cos(t3 ) − 1 3 1 3 10 t sin(t3 ) + 10 e 10

Figure 4.

Mathcad Euler’s Approximation to x0 = t2 x + t2 sin(t3 )

2.3. Error analysis of Euler’s Method. There are two source of error in every numerical method used to approximate a solution of x0 (t) = f (t, x), • Local Truncation Error • The roundoff error. The local truncation error is due to the method and roundoff error is due to computer that is used. For Euler’s method, we use xi = xi−1 + f (ti−1 , xi−1 ) ∗ h to approximate the value of x(ti ), the difference, assuming no rounding is introduced, is |xi − x(ti )|, and is called the local truncation error. The following theorem shows how the local truncation error is depending on f (t, x) and h, (t,x) (t,x) , and ∂f∂x are continuous Theorem 2.1. Suppose f (t, x), ∂f∂t b−1 on [a, b], and h = n is the step size. Furthermore let x(t) be the solution of initial value problem x0 = f (t, x), x(a) = x0 and xi = xi−1 + f (ti−1 , xi−1 ) ∗ h be the approximate of x(ti ), and let ei = x(ti ) − xi be

3. RUNGE-KUTTA METHOD

7

the local truncation error, then 1 ei+1 ≤ (1 + hK)|ei | + h2 M, i = 1, ...n. 2 ∂f (t,x) 00 , where | ∂x | ≤ K, and |x (t)| ≤ M. Theorem 2.2 indicates that, without roundoff error, the smaller h gives better approximate solution. However, due the roundoff error introduced in each step of √ computation, (such as compute can only give approximate value for 2) we can’t choose h arbitrarily small. The following result gives the optimal step size h, Theorem 2.2. With the same assumption of Theorem 2.2,and suppose at each step the roundoff error is bounded by ² then the optimal value for the step length is r 2² hopt = M 3. Runge-Kutta Method 3.1. Runge-Kutta Method. There are several Runge-Kutta methods, but the fourth order Runge-Kutta method is most popular. Suppose x0 (t) = f (t, x), x(a) = x0 . and h = b−a is the step length, the n fourth order Runge-Kutta method is given below, k1 k2 k3 k4 xi+1

= = = = =

hf (ti , xi ) hf (ti + 12 h, xi + 12 k1 ) hf (ti + 12 h, xi + 12 k2 ) hf (ti + h, xi + k3 ) xi + 16 (k1 + 2k2 + 2k3 + k4 )

In general Runge-Kutta method gives more accurate result than Euler’s method at the same step length. However, Runge-Kutta method is more expensive to implement, that is, at each step, Runge-Kutta method requires more computation than Euler’s method( four function evaluations compare one in Euler’s method). Example 3.1. Example 3.2. Find approximate to x(1) if x0 (t) = t2 −ex sin(t), x(0) = 1 with h = 0.25. Solution Here the interval is [0, 1], so a = 0, b = 1. Since b−a h = n = 0.25 we have n = 4 and we need to compute x1 , x2 , x3 , x4 starting with x0 = x(0) = 1, t0 = 0


x1 : k1 = hf (t0 , x0 ) = 0.25 ∗ f (0, 1) = 0 k2 = hf (t0 + 12 h, x0 + 12 k1 ) = f (0.125, 1) = −0.373276 k3 = hf (t0 + 12 h, x0 + 12 k2 ) = 0.25f (0.125, 1 + 0.5(−0.373276)) = −0.373276 k4 = hf (t0 + h, x0 + k3 ) = 0.25f (0.25, 1 − 0.373276) = −0.309854 x1 = x0 + 61 (k1 + 2k2 + 2k3 + k4 ) = 1 + 16 (0 + 2 ∗ (−0.373276) + 2 ∗ (−0.373276) − 0.309854) = 0.923911 t1 = t0 + h = 0 + 0.25 = 0.25 x2 : k1 = hf (t1 , x1 ) = 0.25 ∗ f (.25, 0.923911) = −0.560741 k2 = hf (t1 + 21 h, x1 + 12 k1 ) = f (0.125, 0.923911 + 0.5(−0.560741)) = −0.719601 k3 = hf (t1 + 21 h, x1 + 12 k2 ) = 0.25f (0.125, 0.923911 + 0.5(−0.719601)) = −0.702688 k4 = hf (t1 + h, x1 + k3 ) = 0.25f (0.5, 0.923911 − 0.702688) = −0.763158 x2 = x1 + 16 (k1 + 2k2 + 2k3 + k4 ) = 0.923911 + 61 (−0.560741 + 2 ∗ (−0.719601) + 2 ∗ (−0.702688) − 0.763158) = 0.750224 t2 = t1 + h = 0.25 + 0.25 = 0.5 x3 : k1 = hf (t2 , x2 ) = 0.25 ∗ f (.5, 0.750224) = −0.765171 k2 = hf (t2 + 21 h, x2 + 12 k1 ) = f (0.625, 0.750224 + 0.5(−0.765171)) = −0.735295 k3 = hf (t2 + 21 h, x2 + 12 k2 ) = 0.25f (0.625, 0.750224 + 0.5(−0.735295)) = −0.739508 k4 = hf (t2 + h, x2 + k3 ) = 0.25f (0.75, 0.750224 − 0.739508) = −0.637224 x3 = x2 + 61 (k1 + 2k2 + 2k3 + k4 ) = 0.750224 + 61 (−0.765171 + 2 ∗ (−0.735295) + 2 ∗ (−0.739508) − 0.637224) = 0.568891 t3 = t2 + h = 0.5 + 0.25 = 0.75

3. RUNGE-KUTTA METHOD

9

x4 : k1 = hf (t3 , x3 ) = 0.25 ∗ f (0.75, 0.568891) = −0.641483 k2 = hf (t3 + 21 h, x3 + 12 k1 ) = f (0.875, 0.568891 + 0.5(−0.641483)) = −0.485628 k3 = hf (t3 + 21 h, x3 + 12 k2 ) = 0.25f (0.875, 0.568891 + 0.5(−0.485628)) = −0.510243 k4 = hf (t3 + h, x3 + k3 ) = 0.25f (1, 0.568891 − 0.510243) = −0.308297 x4 = x3 + 61 (k1 + 2k2 + 2k3 + k4 ) = 0.568891 + 61 (−0.641483 + 2 ∗ (−0.485628) + 2 ∗ (−0.510243) − 0.308297) = 0.446327 t4 = t3 + h = 0.75 + 0.25 = 1.0 So the approximation to x(1) is x(1) ≈ x4 = 0.446327.

a

3.2. Mathcad implementation. First, we have the following simple algorithm for the Euler’s method, Input f , a, b, x0 n. Output: the approximate solution to x0 = f (t, x) with initial guess x0 over interval [a, b]. • Step One: Initialization Set h = b−a n Set x0 = x0 Set t0 = a • Step Two: For i=1 to n do Step Three Step Three: Set k1 = hf (ti−1 , xi−1 ) Set k2 = hf (ti−1 + 12 h, xi−1 + 12 k1 ) Set k3 = hf (ti−1 + 12 h, xi−1 + 12 k2 ) Set k4 = hf (ti−1 + h, xi−1 + k3 ) Set xi = xi + 16 (k1 + 2k2 + 2k3 + k4 ) Set ti = ti−1 + h • Step Four return x. This algorithm returns an array of values, the ith element of the return array is an approximations of x(t) at t = a + ih. The following screen shot shows Mathcad code for implementing the algorithm. Again notice the line to line corresponding between the Mathcad and the algorithm.


Figure 5. Mathcad code for Euler’s Method The next screen shot shows a call to the myRungeKutta and tMesh for the equation x0 = t2 x+t2 sin(t3 ). It also shows the graph of approx3 imate solution comparing with the exact solution x(t) = − 10 cos(t3 ) − 1 3 1 3 10 t sin(t3 ) + 10 e , and with the approximate solution using Euler’s 10 method. In this case Runge-Kutta provides much superior result, an almost exact match!

Figure 6.

Mathcad Runge-Kutta’s Approximation to x0 = t2 x + t2 sin(t3 )

4. NUMERICAL METHOD FOR SYSTEM OF EQUATIONS

11

4. Numerical Method for System of Equations Both Euler’s method and Runge-Kutta method can be used to find the approximate solution to the system of first order differential equations. In fact, the Mathcad codes for system of first differential equations are exactly the same as Mathcad does not differentiate scalar and vectors when performs most computations. The only thing needs to be taken care is that at each step the result is vector instead of a scalar. Suppose x(t) is a vector-valued function that satisfies x0 (t) = F (t, x(t)), where F (t, x) is also vector-valued function, ti = a+ih then the Euler’s method compute the ith approximate, (2)

xi = xi−1 + F (ti−1 , xi−1 ). Example 4.1. Let x0 (t) = t2 sin(x(t)) + et cos(y(t)) y 0 (t) = 2tx(t) + ey(t) x(0) = 1, y(0) = −1

Find approximate solution for x(t), y(t) over interval [0, 1] with h = 0.1 · ¸ · 2 ¸ x(t) t sin(x) + et cos(y) Solution Set x(t) = and F (t, x) = y(t) 2tx + ey The equation can then be written, in vector form, · x(0) =

1 −1

¸

x0 (t) = F (t, x(t)), From (2) we can find, with t0 = 0,

x1 = · x0 + h¸∗ F·(t0 , x0 ) ¸ 1 t20 sin(x0 ) + et0 cos(y0 ) = + 2t0 x0 + ey0 · −1 ¸ cos(−1) = e−1 The following screen shot shows the results obtain by calling our Mathcad implementation of Euler’s method. Notice that in defining the vector-valued function F (t, X), for Mathcad to know that X is an vector, in the definition, we use subscript to access the component of X, as shown in the screen shot.


Figure 7.

Euler’s method for system of equations

a


13

Similarly, we can use the same Mathcad code of Runge-Kutta method to find approximate solution as shown in the following screen shot for the above system of equations,

Figure 8.

Runge-Kutta’s method for system of equations

4.1. Numerical Method for Higher Order Equations. To find numerical approximate for solutions of higher order differential equations using either Euler’s method or Runge-Kutta method, we first transform an higher order differential equations into a system of first order differential equations and then apply the methods. Example 4.2. Rayleigh Equation In modeling the oscillations of a clarinet reed. Lord Rayleigh introduced an equation of the form mx00 + kx = ax0 − b(x0 )3 Find approximate solution for m = 1, k = 1, a = 2, b = 3. and initial conditions x(0) = 1, x0 (0) = −2


Solution Suppose y(t) = x0 (t) is the velocity, then Rayleigh equation becomes x0 = y y 0 = 2y − 3y 3 − 2x · ¸ y So we apply the numerical method for F = and 2y − 3y 3 − 2x · ¸ · ¸ x(t) = x(t) 1 x0 (t) = F (t, x) with , x(0) = . The following y(t) −2 screen shot gives approximate solution using Runge-Kutta method,

Figure 9.

Runge-Kutta’s method for Rayleigh’s equation

a we also plot the solution in xy-plane, which is called the velocityposition phase plane, with y-axis represent the velocity and x-axis is the


15

position. The velocity-position plane shows that all velocity-position solution pairs are eventually being attracted to an closed orbit, the closed orbit is called the global attractor.