on sor waveform relaxation methods - CiteSeerX

ON SOR WAVEFORM RELAXATION METHODS JAN JANSSEN AND STEFAN VANDEWALLEy

Abstract. Waveform relaxation is a numerical method for solving large-scale systems of ordinary dierential equations on parallel computers. It diers from standard iterative methods in that it computes the solution on many time-levels or along a continuous time-interval simultaneously. This paper deals with the acceleration of the standard waveform relaxation method by successive overrelaxation (SOR) techniques. In particular, dierent SOR acceleration schemes based on matrixsplitting, extrapolation or convolution are described and their convergence properties are analysed. The theory is applied to a one-dimensional and two-dimensional model problem, and checked against results obtained by numerical experiments.

1. Introduction. Waveform relaxation, also called dynamic iteration or Picard{ Lindelof iteration, is a highly parallel iterative method for numerically solving largescale systems of ordinary dierential equations (ODEs). It is the natural extension to systems of dierential equations of the relaxation methods for solving systems of algebraic equations. In the present paper we will concentrate on linear ODE-systems of the form (1.1)

B u(t) _ + Au(t) = f(t) ; u(0) = u0 ;

where the matrices B and A belong to C dd , and B is assumed to be non-singular. Such a system is found, for example, after spatial discretisation of a constant-coecient parabolic partial dierential equation (PDE) using a conforming Galerkin niteelement method, [23]. Matrix B is then a symmetric positive de nite mass matrix and A is a stiness matrix. A spatial discretisation based on nite volumes or nite dierences leads to a similar system with B respectively a diagonal matrix or the identity matrix. Waveform relaxation has been applied successfully to more general, timedependent coecient problems and to non-linear problems. Yet, only problems of the form (1.1) seem to allow precise quantitative convergence estimates of the waveform iteration. Waveform relaxation methods for linear problems (1.1) are usually de ned by splittings of the coecient matrices of the ODE, i.e., B = MB ?NB and A = MA ?NA , and correspond to an iteration of the following form: d d ( ) (1.2) MB dt + MA u (t) = NB dt + NA u( ?1)(t) + f(t) ; u( )(0) = u0 : In [8], we have investigated the convergence of the above iteration, when (MB ; NB ) and (MA ; NA ) correspond to a standard Jacobi or Gauss{Seidel matrix-splitting. In that paper we assumed the resulting ODEs were solved exactly, i.e., the iteration is continuous in time. In [9] a similar iteration was studied for the discrete-time case, de ned by discretising (1.2) with a linear multistep method. The Jacobi and Gauss{ Seidel waveform methods proved to be convergent, with convergence rates very similar Katholieke Universiteit Leuven, Department of Computer Science, Celestijnenlaan 200A, B-3001 Heverlee, Belgium. This text presents research results of the Belgian Incentive Program "Information Technology" { Computer Science of the Future, initiated by the Belgian State { Prime Minister's Service { Federal Oce for Scienti c, Technical and Cultural Aairs. The scienti c responsibility is assumed by its authors. y California Institute of Technology, Applied Mathematics 217-50, Pasadena, CA 91125. This work was supported in part by the NSF under Cooperative Agreement No. CCR-9120008. 1

2

J. JANSSEN and S. VANDEWALLE

to the convergence rates obtained with corresponding relaxation methods for algebraic systems. In order to improve the convergence of the waveform relaxation methods, several acceleration techniques have been described: successive overrelaxation ([1], [2], [17], [18], [21]), Chebyshev iteration ([13], [22]), Krylov subspace acceleration ([15]), and multigrid acceleration ([8], [9], [14], [24]). This paper deals with acceleration methods based on the successive overrelaxation (SOR) idea. The SOR waveform relaxation method was introduced by Miekkala and Nevanlinna in [17], [18] for problems of the form (1.1) with B = I. They showed that SOR leads to some acceleration, although a much smaller one than the acceleration by the standard SOR method for algebraic equations. These somehow disappointing results led Reichelt, White and Allen to de ne a new convolution SOR waveform relaxation method, [21]. They changed the original algorithm by replacing the multiplication with an SOR parameter by a convolution with a time-dependent SOR kernel. This x appeared to be a very eective one. Application to a non-linear semiconductor device simulation problem showed the new method to be by far superior to the standard SOR waveform relaxation method. In this paper, we study three dierent SOR waveform relaxation techniques, based respectively on matrix-splitting, extrapolation and convolution, for general problems of the form (1.1). We extend the analyses of the above references, and put them into the framework developed in [8], [9]. The convergence properties of these SOR waveform methods are compared, and related to the convergence behaviour of the standard SOR method for algebraic systems. In particular, we show that for a broad class of problems the convolution SOR waveform method attains an identical acceleration as the standard SOR method does for the linear system Au = f. This is not so for the SOR waveform methods based on matrix-splitting or extrapolation. We start in x2 by describing the dierent types of SOR waveform algorithms. Their convergence as continuous-time methods is presented in x3. In x4, we brie y point out the corresponding discrete-time convergence results, and we show how the latter relate to the continuous-time ones. A model problem analysis and numerical results for the one- and two-dimensional heat equation are given in x5. While the latter paragraph deals with point-wise methods, x6 discusses results for line-wise relaxation. Finally, we end in x7 with some concluding remarks.

2. SOR waveform relaxation methods. 2.1. Continuous-time methods. The most obvious way to de ne an SOR

waveform method for systems of dierential equations is based on the natural extension of the standard SOR procedure for algebraic equations, [25], [27]. Let the elements of matrices B and A be denoted by bij and aij , 1 i; j d. First, a function u^(i )(t) is computed with a Gauss{Seidel waveform relaxation scheme, i?1 d X bij dt + aij u(j )(t) bii dtd + aii u^(i ) (t) = ? j =1 (2.1) d d X bij dt + aij u(j ?1)(t) + fi (t) ; ? j =i+1 with u^(i ) (0) = (u0)i . Next, the old approximation u(i ?1)(t) is updated by extrapolating the correction u^(i )(t) ? u(i ?1)(t) using an overrelaxation parameter !, (2.2) u(i )(t) = u(i ?1)(t) + ! u^(i )(t) ? u(i ?1)(t) :

3

SOR WAVEFORM RELAXATION METHODS

We shall refer to this method as the extrapolation SOR waveform relaxation (ESOR) method. This method was brie y considered for the B = I case in [21] and also studied in [1], [2] for the solution of non-linear systems of ODEs. A second approach is based on the idea of matrix-splitting. In [17], [18], Miekkala and Nevanlinna consider the case of ODE-systems of the form u_ + Au = f. They de ne an SOR waveform relaxation method in terms of the splitting MA = !1 DA ? LA ; NA = 1 ?! ! DA + UA ; (2.3) where A = DA ? LA ? UA , with DA , LA and UA respectively the diagonal, strictly lower triangular and strictly upper triangular part of A. Their iteration is of the form (1.2) with MB = I, NB = 0 and (2.3). This scheme can be extended to the more general problem (1.1), by splitting the matrix B in a similar way, (2.4) MB = !1 DB ? LB ; NB = 1 ?! ! DB + UB : We shall refer to the above schemes as splitting SOR waveform relaxation methods, with single splitting (SSSOR) or double splitting (DSSOR). The rst step of the convolution SOR waveform relaxation (CSOR) method is similar to the rst step of the ESOR scheme, and consists of the computation of a Gauss{Seidel iterate u^(i ) (t) using (2.1). Instead of multiplying the resulting correction by a scalar !, as in (2.2), the correction is convoluted with a function (t), [21], (2.5)

u(i )(t) = u(i ?1)(t) +

Z

t

0

(t ? ) u^(i )() ? u(i ?1)() d :

We will allow for fairly general convolution kernels of the form (2.6)

(t) = ! (t) + !c (t) ;

with ! a scalar parameter, (t) the delta function, and !c(t) a function in L1 . In that case, (2.5) can be rewritten as

(2.7)

u(i )(t) = u(i ?1)(t) + ! u^(i )(t) ? u(i ?1)(t) Z t + !c (t ? ) u^(i )() ? u(i ?1)() d : 0

Hence, it corresponds to (2.2) with an additional correction based on a Volterra convolution.

Property 2.1. The standard Gauss{Seidel waveform relaxation method is obtained by setting ! = 1 and !c (t) = 0 in any of the SOR waveform methods. Property 2.2. The extrapolation SOR waveform relaxation method is identical to the splitting SOR waveform method with double splitting. Proof. Let u( ?1), u( ) and u^ denote the d-vectors corresponding to, respectively,

the old approximation, the new approximation, and the intermediate approximation de ned by (2.1). It can be veri ed easily that (2.1) can be written in matrix-form as

DB dtd + DA u^( ) (t) = LB dtd + LA u( )(t) + UB dtd + UA u( ?1)(t) + f(t) ;

4


and that (2.2) is equivalent to

u( )(t) = u( ?1)(t) + ! u^( )(t) ? u( ?1)(t) :

Elimination of u^( )(t) from the former two expressions leads to 1D ?L d + 1D ?L B B dt A A u( ) (t) = ! ! (2.8) 1 ? !D + U d + 1 ? !D + U ( ?1) ! B B dt ! A A u (t) + f(t) ; which exactly equals the double-splitting method with (2.3) and (2.4). Property 2.3. For ODE-systems (1.1) with B = I , the extrapolation SOR

waveform relaxation method is dierent from the splitting SOR waveform method with single splitting, unless ! = 1. Proof. From Property 2.2 we know that, in the case B = I, the ESOR method

corresponds to a splitting of B with MB = !1 I and NB = 1?!! I, together with (2.3). On the other hand, the single-splitting scheme is de ned by MB = I, NB = 0 and (2.3). 2.2. Discrete-time methods. In an actual implementation, the continuoustime methods are replaced by their discrete-time variants. As in [9], we shall concentrate in this paper on the use of (irreducible, consistent, zero-stable) linear multistep formulae for time-discretisation. For the reader's convenience, we recall the general linear multistep formula for calculating the solution to the ODE y(t) _ = f(t; y) with y(0) = y[0], see e.g. [12, p. 11], k k X 1X l=0 l y[n + l] = l=0 l f[n + l] :

In this formula, l and l are real constants, denotes a constant step-size and y[n] denotes the discrete approximation of y(t) at t = n. We shall assume that k starting values y[0]; y[1]; : : :; y[k ? 1] are given. The characteristic polynomials of the linear multistep method are given by a(z) =

k X l=0

l z l and b(z) =

k X l=0

l z l :

The stability region S consists of those 2 C for which the polynomial a(z) ? b(z) (around = 1: ?1 a(z) ? b(z)) satis es the root condition: all roots satisfy jzl j 1 and those of modulus 1 are simple. The discrete-time extrapolation SOR waveform relaxation algorithm is obtained by application of the time-discretisation method to (2.1) and by extrapolating the resulting discrete correction in the manner of (2.2), (2.9)

u(i )[n] = u(i ?1)[n] + ! u^(i )[n] ? u(i ?1)[n] :

The discrete-time splitting SOR waveform relaxation algorithm is de ned by discretising (1.2). In the single-splitting case, for B = I, we use (2.3), with MB = I and NB = 0. Using (2.3) and (2.4) yields the double-splitting method.


5

The rst step of the discrete-time convolution SOR waveform relaxation algorithm is obtained by discretising (2.1). The second step replaces the convolution integral ?1, where N denotes the in (2.5) by a convolution sum with kernel = f [n]gNn=0 (possibly in nite) number of time-steps, (2.10)

u(i )[n] = u(i ?1)[n] +

n X l=0

[n ? l] u^(i ) [l] ? u(i ?1)[l] :

We obtain a discrete-time analogue of (2.7) if we set = ! + (!c ) , with the discrete delta function, [20, p. 409]. The right-hand side of (2.10) then becomes n X ( ? 1) ( ) ( ? 1) ui [n] + ! uî [n] ? ui [n] + !c [n ? l] u^(i ) [l] ? u(i ?1)[l] : l=0

Note that the convolution sum is a bounded operator if (or (!c ) ) is an l1 -sequence. We shall always assume that we do not iterate on k given starting values, i.e., ( ) uî [n] = u(i ) [n] = u(i ?1)[n] = ui [n], n < k. For implicit multistep methods, i.e., methods with k 6= 0, the linear systems arising from discretising (1.2) can be solved uniquely for every n if and only if the discrete solvability condition, k 62 (?M ?1 M ) ; B A k is satis ed, where () denotes the spectrum, [9, eq. (4.5)]. This simpli es to k 62 (?D?1 D ) ; (2.11) B A k in the case of (2.1) or (2.8). Remark 2.1. A discrete-time version of Property 2.1 is obtained in a straightforward manner. In complete analogy with Property 2.2, we can prove that the discretetime extrapolation SOR waveform relaxation method is identical to the discrete-time splitting SOR waveform method with double splitting. Also, a discrete equivalent of Property 2.3 yields that the discrete-time extrapolation SOR waveform relaxation method for ODE-systems (1.1) with B = I is dierent from the single-splitting SOR waveform method, unless ! = 1.

2.3. Block-wise methods. The point-wise SOR waveform relaxation methods described above can be adapted easily to the block-wise relaxation case. Matrices B and A are then partitioned into similar systems of db db rectangular blocks bij and aij with dimension mi nj , such that mi = ni ; 1 i db and

db X i=1

mi =

db X j =1

nj = d :

Correspondingly, DA , LA and UA (and DB , LB and UB ) are block diagonal, block lower triangular and block upper triangular matrices. All of the previous results can be shown to hold easily.

6


3. Continuous-time convergence analysis. In this paragraph, we will analyse the convergence properties of the continuous-time SOR waveform relaxation methods outlined in the previous paragraph. We will consider the general case of block-wise relaxation, which includes point-wise relaxation as a limiting case. This analysis will follow the framework of [8], [9] and extend and complete the results of [21] to systems of the form (1.1). We will concentrate on deriving the nature and properties of the iteration operators of the extrapolation and the convolution SOR waveform relaxation methods. For the splitting SOR waveform method with single splitting { which nds use only in the B = I case { we refer to [17], [18]. 3.1. Nature of the operators. In order to identify the nature of the SOR waveform iteration operators we need the following elementary result, which we formulate as a lemma. Lemma 3.1. The solution to the ODE bu(t) _ + au(t) = qv(t) _ + pv(t) + w(t) with constants b; a; q; p 2 C mm and b non-singular, is given by u(t) = Kv(t) + '(t), with Kx(t) = b?1qx(t) +

Z

0

t

kc (t ? )x()d

? '(t) = e?b? at u(0) ? b?1 q v(0) + 1

kc(t) = e?b? atb?1 (p ? ab?1 q) ;

with

Z

t

0

1

e?b? a(t? ) b?1w()d : 1

The function kc (t) 2 L1 (0; 1) if Re() > 0 for all 2 (b?1 a). If, in addition, w(t) 2 Lp (0; 1), then '(t) 2 Lp (0; 1). Proof. The lemma is an immediate consequence of the well-known solution for-

mula for the ODE u_ + au = f, see e.g. [4, p. 119]. The ESOR and CSOR waveform relaxation algorithm implicitly de ne a classical successive iteration scheme of the form u( )(t) = KESORu( ?1)(t) + 'ESOR (t) and u( )(t) = KCSORu( ?1)(t)+'CSOR (t) ; where 'ESOR (t) and 'CSOR (t) depend on the ODE's right-hand side f(t) and the initial condition, and where KESOR and KCSOR are the continuous-time ESOR and CSOR waveform relaxation operators. The next two theorems identify the nature of these operators. Theorem 3.2. The continuous-time extrapolation SOR waveform relaxation operator is of the form

KESOR = K ESOR + KcESOR ;

(3.1)

with K ESOR 2 C dd and with KcESOR a linear Volterra convolution operator, whose matrix-valued kernel kcESOR (t) 2 L1(0; 1) if all eigenvalues of DB?1 DA have positive real parts. Proof. Introductory Lemma 3.1 can be applied to equation (2.1) to give i?1

db X

j =1

j =i+1

X (3.2) u^(i )(t) = (Hij + (hc )ij ?)u(j ) (t) +

(Hij + (hc )ij ?)u(j ?1)(t) + i (t) :

Here, we have used the symbol \?" to denote convolution. The (matrix-)constants Hij and (matrix-)functions (hc )ij (t) can be derived from the result of the lemma with b = bii, a = aii, q = ?bij and p = ?aij . Note that (hc )ij (t) 2 L1 (0; 1) if Re() is positive for all 2 (b?ii 1 aii), while i(t) 2 Lp (0; 1) if, in addition, fi (t) belongs to Lp (0; 1).


7

We shall prove the existence of constants KijESOR and L1 -functions (kcESOR )ij (t), i; j = 1; : : :; db, such that (3.3) u(i ) (t) =

db X j =1

KijESOR u(j ?1)(t)

+

db X j =1

(kcESOR )ij ? u(j ?1)(t) + 'i (t) :

The case i = 1 follows immediately from the combination of (3.2) with (2.2). More ESOR = (1 ? !)I, K ESOR = !H1j , (kESOR )11(t) = 0 and (kESOR )1j (t) = precisely, K11 c c 1j !(hc )1j (t), 2 j db, with I the identity matrix whose dimension equals the dimension of b11 and a11, i.e., m1 m1 , in the case of block-relaxation. Also, '1(t) = !1 (t). The general case follows by induction on i. This step involves computing u(i )(t) from (3.2) and (2.2) and using (3.3) to substitute u(j )(t), j < i. The result then follows from the knowledge that linear combinations and products of operators of the form (m + mc ?), i.e., multiplication by a constant plus a convolution with an L1 -kernel, leads to an operator of the same type. A similar result can be proven for the CSOR waveform operator. Theorem 3.3. The continuous-time convolution SOR waveform relaxation op-

erator is of the form

KCSOR = K CSOR + KcCSOR ; with K CSOR 2 C dd and with KcCSOR a linear Volterra convolution operator, whose matrix-valued kernel kcCSOR (t) 2 L1 (0; 1) if all eigenvalues of DB?1 DA have positive real parts and if (t) is of the form (2.6) with !c (t) 2 L1 (0; 1). (3.4)

Proof. The proof is similar to the proof of Theorem 3.2. It only diers in that the expression (3.2) for u^(i ) (t) is now inserted into (2.5) or, equivalently, into (2.7). Since !c(t) 2 L1 (0; 1), the induction argument based on the closedness of the space of operators of the form (m + mc ?) with mc (t) 2 L1 (0; 1) still applies. 3.2. Symbols. The symbols KESOR(z) and KCSOR(z) of the continuous-time ESOR and CSOR waveform relaxation operators are obtained by Laplace-transforming their respective iterative schemes. For the ESOR waveform relaxation scheme, we transform (2.1){(2.2), or, equivalently, (2.8). This gives the equation u~( )(z) = KESOR (z)~u( ?1)(z)+ '(z) ~ in Laplacetransform space, with ?1 1 1 ESOR K (z) = z ! DB ? LB + ! DA ? LA (3.5) 1 ? ! 1 ? ! z ! DB + UB + ! DA + UA :

Here, we have used the \~"-notation ? to R denote a Laplace-transformed variable, as, for instance, in u~( ) (z) = L u( )(t) = 01 u( )(t)e?zt dt. Alternatively, we can write ? ESOR ESOR KESOR (z) as K ESOR + KESOR (z), with K (z) = L k (t) , the Laplacec c c transform of the Volterra convolution part of operator (3.1). Laplace-transforming the CSOR waveform relaxation scheme (2.1){(2.5) yields ?1 1 1 CSOR K (z) = z ~ DB ? LB + ~ DA ? LA

(z)

(z) ! !! ~ (z) ~ (z) 1 ?

1 ?

; D +U + ~ D +U z ~

(z) B B

(z) A A

8


where ~ (z) = L ( (t)). Note that ~ (z) = ! + !~ c(z), where !~ c(z) = L (!c (t)). With reference to (3.4), write this as KCSOR (z) = K CSOR + KCSOR (z), with c ? we can also CSOR CSOR Kc (z) = L kc (t) . 3.3. Spectral radii and norms. The convergence properties of the continuoustime SOR waveform relaxation operators can be expressed in terms of their symbols, as explained in our general theoretical framework for operators consisting of a matrix multiplication and convolution part, [8, x2]. 3.3.1. Finite time-intervals. The following two theorems follow immediately from [8, Lemma 2.1], applied to KESOR and KCSOR. In the latter case, the proof is completed by noting that !c(t) is an L1 -kernel, and, consequently, limz!1 !~ c(z) = 0. Theorem 3.4. Consider KESOR as an operator in C[0; T]. Then, KESOR is a bounded operator and

(3.6) (3.7)

? ? ? KESOR = KESOR (1) = K ESOR ?1 ! 1 1 ? ! = ! DB ? L B ! DB + UB :

Theorem 3.5. Consider KCSOR as an operator in C[0; T]. Then, KCSOR is a

bounded operator and

? ? ? KCSOR = KCSOR (1) = K CSOR :

(3.8)

If (t) is of the form (2.6) with !c (t) 2 L1 (0; T), we have

(3.9)

?

KCSOR =

! 1 D ? L ?1 1 ? ! D + U ! B B ! B B :

3.3.2. In nite time-intervals. If all eigenvalues of DB?1 DA have positive real parts, Theorems 3.2 and 3.3 imply that the convolution kernels of both SOR waveform relaxation operators KESOR and KCSOR are in L1 (0; 1). Consequently, we can apply [8, Lemmas 2.3 and 2.4] in order to obtain the following two convergence theorems. Theorem 3.6. Assume all eigenvalues of DB?1 DA have positive real parts. Consider KESOR as an operator in Lp (0; 1), 1 p 1. Then, KESOR is a bounded operator and

(3.10)

(KESOR ) = sup KESOR (z) = sup KESOR (i) : ?

Re(z)0

R

?

2

Denote by jjjj2 the L2-norm and by jjjj the standard Euclidean vector-norm. Then, (3.11) jjKESORjj2 = sup jjKESOR(z)jj = sup jjKESOR(i)jj : Re(z)0

R

2

Theorem 3.7. Assume all eigenvalues of DB?1 DA have positive real parts, and let (t) be of the form (2.6) with !c (t) 2 L1 (0; 1). Consider KCSOR as an operator

in Lp (0; 1), 1 p 1. Then, KCSOR is a bounded operator and ? ? (3.12) (KCSOR ) = sup KCSOR (z) = sup KCSOR (i) : Re(z)0

R

2


9

Denote by jjjj2 the L2-norm and by jjjj the standard Euclidean vector-norm. Then, (3.13) jjKCSORjj2 = sup jjKCSOR(z)jj = sup jjKCSOR(i)jj : Re(z)0

R

2

In Theorem 3.7, we require ~ (z) to be the Laplace-transform of a function of the form (2.6) with !c (t) 2 L1 (0; 1). For this, a sucient (but not necessary) condition is that ~ (z) is a bounded and analytic function in an open domain containing the closed right half of the complex plane, [10, Prop. 2.3]. 3.4. On the relation between the Jacobi and SOR symbols. At the heart of classical SOR theories for determining the optimal overrelaxation parameter lies usually the existence of a Young-relation, i.e., a relation between the eigenvalues of the Jacobi and the SOR iteration matrices. The following lemma reveals the existence of such a relation between the eigenvalues of the ESOR symbol KESOR (z) and the eigenvalues of the Jacobi symbol (3.14) KJAC (z) = (zDB + DA)?1 (z(LB + UB ) + (LA + UA )) : It uses the notion of a block-consistent ordering, for which we refer to [27, p. 445, Def. 3.2]. Lemma 3.8. Assume the matrices B and A are such that zB + A is a blockconsistently ordered matrix with non-singular diagonal blocks, and ! 6= 0. If (z) is an eigenvalue of KJAC (z) and (z) satis es (3.15) ((z) + ! ? 1)2 = (z)(!(z))2 ; then (z) is an eigenvalue of KESOR(z). Conversely, if (z) 6= 0 is an eigenvalue of KESOR (z) which satis es (3.15), then (z) is an eigenvalue of KJAC (z). Proof. Using the shorthands D? = zDB +DA , L? = zLB +LA and U ? = zUB +UA , we can write (3.14) and (3.5) as KJAC (z) = (D? )?1 (L? + U ? ) KESOR(z) = (D? ? !L? )?1((1 ? !)D? + !U ?) : Thus, KJAC (z) and KESOR (z) are the standard Jacobi and SOR iteration matrices for the matrix zB + A = D? ? L? ? U ? . The result of the lemma then follows immediately from classical SOR theory, [25, Thm. 4.3], [27, p. 451, Thm. 3.4]. A similar result holds for the eigenvalues of KCSOR (z). Lemma 3.9. Assume the matrices B and A are such that zB + A is a blockconsistently ordered matrix with non-singular diagonal blocks, and ~ (z) 6= 0. If (z) is an eigenvalue of KJAC (z) and (z) satis es (3.16) ((z) + ~ (z) ? 1)2 = (z)( ~ (z)(z))2 ; then (z) is an eigenvalue of KCSOR (z). Conversely, if (z) 6= 0 is an eigenvalue of KCSOR (z) which satis es (3.16), then (z) is an eigenvalue of KJAC (z). Under the assumption of a block-consistent ordering of zB + A, we have that if (z) is an eigenvalue of KJAC (z) then also ?(z) is an eigenvalue of KJAC (z), see e.g. [27, p. 451, Thm. 3.4]. Therefore, if (z) is an eigenvalue of KJAC (z) and (z) satis es (3.15) or (3.16), we can choose p p (3.17) (z) + ! ? 1 = (z)!(z) or (z) + ~ (z) ? 1 = (z) ~ (z)(z) : Remark 3.1.

10


The next lemma determines the optimal (complex) value ~ (z) which minimises the spectral radius of KCSOR (z) for a given value of z. The result follows immediately from complex SOR theory, see e.g. [11, Thm. 4.1] or [19, eq. (9.19)]. The result was rediscovered in [21, Thm. 5.2], and presented in a waveform relaxation context for the B = I case. Lemma 3.10. Assume the matrices B and A are such that zB + A is a blockconsistently ? ordered matrix with non-singular diagonal blocks. Assume the spectrum

KJAC (z) lies on a line-segment [?1 (z); 1(z)] with 1(z) 2 C and j1(z)j = (KJAC (z)) < 1. The spectral radius of KCSOR (z) is then minimised for a given value of z by the unique optimum ~ opt (z), given by (3.18)

~ opt (z) = p 2 2 ; 1 + 1 ? 1 (z)

p

where denotes the root with the positive real part. In particular, ? (3.19) KCSOR;opt (z) = j ~ opt(z) ? 1j < 1 :

Remark 3.2. Following Kredell in [11, Lemma 4.1], the condition on the collinearity of the eigenvalues of the Jacobi symbol can be weakened. For (3.18) to hold, it is sucient that there exists a critical eigenvalue pair 1 (z) of KJAC (z), i.e., for all values of the overrelaxation parameter the pair 1 (z) corresponds to the dominant eigenvalue of KCSOR (z). The existence of such a pair may be dicult to verify in practice, though. The results of the above lemmas rely on the assumption of a block-consistent ordering of the matrix zB + A. In the case of semi-discrete parabolic PDEs in two dimensions, this assumption is, for example, satis ed if B and A correspond to a vepoint star and the relaxation is point-wise with lexicographic, diagonal or red/black ordering of the grid points. It is also satis ed if B and A correspond to a 3 3-stencil { a discretisation with linear or bilinear nite elements on a regular triangular or rectangular mesh, for example { and the relaxation is line-wise. The assumption that the eigenvalues of the Jacobi symbol are on a line is rather severe. Yet, it is satis ed for certain important classes of problems. For example, this is so when B = I and A is a symmetric positive de nite consistently ordered matrix with constant positive diagonal DA = da I (da > 0). In that case the spectrum of KJAC (z) equals da=(z + da )(KJAC (0)). It is the spectrum of the standard Jacobi iteration matrix, which is real and has maximumsmaller than 1, [27, p. 147, Thm. 3.5], scaled and rotated around the origin. If the assumptions of the theorem are violated, a more general complex SOR theory, allowing the eigenvalues of the Jacobi symbol to be in a certain ellipse, should be used, see e.g. [6], [19], [27]. Alternatively, one could decide to continue to use (3.18). Although one gives up optimality then, this practice may lead to a good enough convergence, as is illustrated in x6. If j1(z)j < 1 and 1(z) is analytic for Re(z) 0, then ~ opt (z) is bounded and analytic for Re(z) 0. According to Remark 3.1, Theorem 3.7 then may be applied to calculate (KCSOR;opt ). In general however 1(z) is only known to be piecewise analytic, with possible discontinuities or lack of smoothness where the maximum switches from one eigenvalue branch to another. In that case, one might opt for using (3.18) with an analytic function 1(z) that approximates the \correct" one at certain values of z (e.g., near the values that correspond to the slowest converging error components).

11


3.5. Optimal point-wise overrelaxation in the B=I case. In [17, Thm. 4.2], Miekkala and Nevanlinna derived an analytical expression for the spectral radius of the single-splitting SOR waveform relaxation operator KSSSOR . For comparison purposes, we recall their result. Theorem 3.11. Consider (1.1) with B = I . Assume A is a consistently ordered matrix with constant positive diagonal DA = da I (da > 0), the eigenvalues of KJAC (0) are real with 1 = (KJAC (0)) < 1 and 0 < ! < 2. Then, if we consider KSSSOR as an operator in Lp (0; 1), 1 p 1, we have 8 >
(3.20) (K 8(! ? 1)2 : ! > !d ; 8(! ? 1) ? (!1 )2 p where !d = (4=3)(2 ? 4 ? 321)=21 . Furthermore, we have !opt = 4=(4 ? 21) > !d . Based on Lemma 3.8 we can derive an analogous theorem for the extrapolation SOR waveform relaxation operator. Theorem 3.12. Consider (1.1) with B = I . Assume A is a consistently ordered matrix with constant positive diagonal DA = da I (da > 0), the eigenvalues of KJAC (0) are real with 1 = (KJAC (0)) < 1 and 0 < ! < 2. Then, if we consider KESOR as an operator in Lp (0; 1), 1 p 1, we have (3.21)

(KESOR ) =

8 > >
> :

p

1

! !d ! > !d ;

where !d = (4 ? 2 4 ? 221 )=21. Furthermore, we have that !opt > !d . Proof. Since B = I and since DA = da I, we have that

(KJAC (z)) = z +dad (KJAC (0)) :

(3.22)

a

With (0) denoting an arbitrary eigenvalue of KJAC (0), we can rewrite (3.17) as p z = ?1 + (z)!(0) : da (z) + ! ? 1 Setting (z) = j(z)jeit, we nd p j(z)j!(0) : z = ?1 + i da j(z)je t + (! ? 1)e?i t The supremum of j(z)j along the imaginary axis is attained at the point where the equilibrium lines of j(z)j are tangent to the imaginary axis, i.e., when Re z(t) = 0 and Re z 0(t) = 0. If the supremum in (3.10) is attained for z = i = 0, we get the same value for (KESOR (0)) as in the algebraic SOR method, [25], [27], 2

(KESOR (0)) = (3.23)

(

2

q

1 ? ! + 21 (!1 )2 + (!1 ) 1 ? ! + 14 (!1 )2 !?1

alg ! !opt alg ; ! > !opt

12

J. JANSSEN and S. VANDEWALLE p

alg = 2=(1 + 1 ? 2). where !opt 1 If the supremum in (3.10) is attained at the complex point z = i, 6= 0 (and by symmetry at z = ?i), condition Re(z) = 0 yields p 4j(z)j(! ? 1) cos2 2t ? j(z)j!(0)(j(z)j +! ? 1) cos 2t +(j(z)j? !+1)2 = 0 and condition Re(z 0 ) = 0 gives p 4j(z)j(! ? 1) cos 2t ? 12 j(z)j!(0)(j(z)j + ! ? 1) = 0 : ? Elimination of cos 2t from the above two expressions gives 2 ? 2!2 (! ? 1)2(0) ? 32(! ? 1) 2 j(z)j + (! ? 1)2 = 0 ; j(z)j + 16(! ? 1) ? !2 2 (0) whose largest solution for ! > 1 equals

j(z)j = (! ? 1)

(3.24)

1 + 41 p!!(0) ?1 : 1 ? 41 p!!(0) ?1

The value of !d may be obtained by determining the least value of ! for which alg . the osculation point moves away from the origin. It turns out that 1 < !d < !opt Hence, the proof is completed by combining (3.23) and (3.24), the latter of which is an increasing function of (0). Theorem 3.13. Consider (1.1) with B = I . Assume A is a consistently ordered matrix with constant positive diagonal DA = da I (da > 0) and the eigenvalues of KJAC (0) are real with 1 = (KJAC (0)) < 1. Then, if we consider the CSOR operator with optimal kernel, KCSOR;opt, as an operator in Lp (0; 1), 1 p 1, we have

(KCSOR;opt) =

(3.25)

21 : (1 + 1 ? 21)2 p

Proof. Under the assumptions of the theorem we have (3.22). Hence, we may apply Lemma 3.10 with 1 (z) = z +dad 1 ; a in order to derive the optimum complex overrelaxation parameter ~ opt (z). Since

~ opt (z) is a bounded analytic function in the complex right-half plane, including the imaginary axis, we know from Remark 3.1 that it is the Laplace-transform of a function of the form (2.6), with !c(t) in L1 (0; 1). Thus, we may apply Theorem 3.7, which, when combined with (3.19), yields j1(i)j2 p : (KCSOR;opt ) = sup (KCSOR;opt (i)) = sup j ~ opt (i) ? 1j = sup 2 2 2 j1 + 1 ? 21(i)j2 Since the numerator is maximal for = 0, and the denominator is minimal for = 0, the latter supremum is obtained for = 0. This completes the proof.

R

R

R


13

Table 3.1

Spectral radii of optimal single-splitting (SSSOR), extrapolation (ESOR) and convolution (CSOR) SOR waveform relaxation as a function of 1 . The value of the optimal parameter !opt is given in parenthesis.

1 0.9 0.95 0.975 0.9875 (KSSSOR;!opt ) 0.681 (1.2539) 0.822 (1.2913) 0.906 (1.3117) 0.952 (1.3224) (KESOR;!opt ) 0.750 (1.1374) 0.867 (1.1539) 0.932 (1.1626) 0.965 (1.1671) (KCSOR;opt ) 0.393 0.524 0.636 0.728

Since the maximum of (KCSOR;opt (z)) is found at the complex origin, we can easily state and prove the following properties.

Property 3.1. Under the assumptions of Theorem 3.13, point-wise optimal convolution SOR waveform relaxation for the ODE system u_ + Au = f attains the same asymptotic convergence rate as optimal algebraic SOR for the linear system Au = f . alg . Proof. This follows from ~ opt (0) = !opt Property 3.2. Under the assumptions of Theorem 3.13, we have that opt (t) = alg , the optimal overrelaxation parameter for the system Au = f (t) + !c (t). Also !Ropt alg satis es !opt = 1 + 01 !c (t)dt . Proof. We have that limz?>1 1 (z) = 0. Hence, limz?>1 ~ opt (z) = 1 and the

alg and the de nition rst result follows. The second result follows from ~ opt (0) = !opt of the Laplace-transform.

Property 3.3. Under the assumptions of Theorem 3.13, we have that KCSOR;opt

is a convolution operator. Proof. The matrix multiplication part of KCSOR;opt satis es K CSOR;opt = lim KCSOR;opt (z) = 0 : z?>1

Hence, by Theorem 3.3, KCSOR;opt is a convolution operator with an L1 -kernel. Using formulae (3.20), (3.21) and (3.25), we can compute the spectral radii of the optimal single-splitting , extrapolation and convolution SOR waveform relaxation methods as a function of 1 . These values are presented in Table 3.1, together with the values of the optimal parameter !opt for the SSSOR and ESOR method. 4. Discrete-time convergence analysis. In this paragraph we will brie y analyse the discrete-time SOR waveform relaxation methods outlined in x2.2. As in the previous paragraph, we will identify the nature of the iteration operators, present their symbols and give the spectral radii and norm formulae. The operators are very similar to those analysed in [9]. Hence most results follow in a straightforward way. 4.1. Nature of the operators. Since we do not iterate on the k starting values, we use a shifted subscript -notation for sequences u of which the initial values u[n], ?1 . The discrete-time version of Lemma 3.1 n < k are known, i.e., u = fu[k + n]gNn=0 reads as follows. Lemma 4.1. The solution to the dierence equation

k 1 X l=0

l b + l a u[n + l] =

k 1 X l=0

l q + l p v[n + l] +

k X l=0

l w[n + l] ; n 0;

14


with b; a; q; p 2 C mm and b non-singular is given by u = K v + ' , with K a discrete convolution operator:

(K x )[n] = (k ? x )[n] =

n X l=0

k[n ? l]x[l] ; n 0 ;

and ' depending on w[l], l 0 and the initial values u[l]; v[l], l = 0; : : :; k ? 1. The sequence k 2 l1 (1) if (?b?1a) intS , where S is the stability region of the linear multistep method. If, in addition, w 2 lp (1), then ' 2 lp (1). Proof. The proof of the lemma is based on a Z-transform argument and the use

of Wiener's inversion theorem for discrete l1 -sequences. It is analogous to the proof of [9, Lemma 4.3]. The discrete-time ESOR and CSOR schemes can both be written as classical successive approximation methods, u( ) = KSOR u( ?1) + ' . Here, KSOR denotes the ESOR or CSOR iteration operator and ' is a sequence which depends on the dierence equation's right-hand side f and the initial conditions. We shall rst identify the nature of KESOR .

Theorem 4.2. The discrete-time ESOR waveform relaxation operator KESOR is a discrete convolution operator, whose matrix-valued kernel kESOR 2 l1 (1) if (?D?1 DA ) int S . B

Proof. Discretisation of (2.1) and (2.2) leads to the discrete-time ESOR scheme,

given by

k 1 X

(4.1) l=0

( ) l bii + l aii uî [n + l] = ?

i?1 X k 1 X

( ) l bij + l aij uj [n + l]

j =1 l=0 k 1 b + a u( ?1)[n + l] + X l fi [n + l] ; ? l ij l ij j j =i+1 l=0 l=0 db X k X

n0;

and (2.9). Application of Lemma 4.1 to (4.1) gives (4.2)

(û(i )) =

i?1 X

db X

j =1

j =i+1

(hij ) ? (u(j )) +

(hij ) ? (u(j ?1)) + (i) ;

with (hij ) 2 l1 (1) if (? b?ii 1 aii) int S. If the latter assumption is satis ed and (fi ) 2 lp (1), the sequence (i ) belongs to lp (1). It is now easy to prove that there exist l1 -sequences (kijESOR ) such that (4.3)

(u(i ) ) =

db X j =1

(kijESOR ) ? (u(j ?1)) + ('i ) ;

ESOR ) = (1 ? !)I and Indeed, the combination of (4.2) and (2.9) for i = 1 gives (k11 ESOR (k1j ) = ! (h1j ) , 2 j db, with I the identity matrix of order m1 in the case of block-relaxation, and ('1 ) = !(1 ) . The general case involves the computation of (u(i )) from (4.2) and (2.9). It follows by induction on i and is based on the elimination of the sequences (u(j )) , j < i from (4.2) using (4.3). We nd that

(kijESOR ) = !

i?1 X r=1

(hir ) ? (krj ) + ij (1 ? !)I + ij ! (hij ) ;

15


with ij =1 if i=j and zero otherwise, ij =1 if i>j and zero otherwise, and I the identity matrix of order mi in the case of block-relaxation. Since linear combinations and convolutions of l1 -sequences lead to other l1 -sequences we have that (kijESOR ) 2 l1 (1). Theorem 4.3. The discrete-time CSOR waveform relaxation operator KCSOR is a discrete convolution operator, whose matrix-valued kernel kCSOR 2 l1 (1) if (?DB?1 DA ) int S and 2 l1 (1).

Proof. The proof is similar to the proof of Theorem 4.2, and diers only in that every occurrence of a multiplication by ! is replaced by a convolution with the l1 -sequence as in (2.10). 4.2. Symbols. Discrete Laplace- or Z-transformation of the iterative schemes of the discrete-time SOR waveform relaxation methods yields anPiteration of the ( ?1)(z) + '~ (z) with y~ (z) = Z (y ) = 1 y[i]z ?i and form u~( )(z) = KSOR (z)~u i=0 SOR K (z) the discrete-time ESOR or CSOR symbol. These symbols are given by

KESOR (z) = 1 a (z) 1 DB ? LB + 1 DA ? LA b

?1

! ! 1 a (z) 1 ? ! D + U + 1 ? ! D + U b ! B B ! A A

and

KCSOR (z) = 1 ab (z) ~ 1 DB ? LB + ~ 1 DA ? LA

(z)

(z) !

?1

1 a (z) 1 ? ~ (z) D + U + 1 ? ~ (z) D + U b

~ (z) B B

~ (z) A A

!!

respectively, with ~ (z) = Z ( ).

4.3. Spectral radii and norms. The theorems in this paragraph follow immediately from the general theory in [9, x2], where we analysed the properties of discrete convolution operators. For the iteration on nite intervals we have the following result from [9, Lemma 2.1].

Theorem 4.4. Assume that the discrete solvability condition (2.11) is satis ed, and consider the discrete-time ESOR or CSOR operator K as an operator in lp (N) with 1 p 1 and N nite. Then, K is a bounded operator and

(4.4)

(K ) = (K (1)) :

If (?DB?1 DA ) int S, Theorems 4.2 and 4.3 imply that the kernels of the ESOR and CSOR operators are in l1 (1). Hence, we may apply [9, Lemmas 2.2 and 2.3] in order to derive the following in nite-interval results. Theorem 4.5. Assume (?DB?1 DA ) int S and 2 l1 (1). Consider the discrete-time ESOR or CSOR iteration operator K as an operator in lp (1), 1 p 1. Then, K is a bounded operator and (4.5)

(K ) = jmax (K (z)) = jmax (K (z)) : zj1 zj=1

16


If we denote by jj jj2 the l2 -norm and by jj jj the Euclidean vector-norm, we have

jjK jj2 = jmax jjK (z)jj = max jjK (z)jj : zj1 jzj=1

(4.6)

~ (z) to be the Z-transform of an Remark 4.1. In Theorem 4.5, we require

l1 -kernel . For this, a sucient (but not necessary) condition is that ~ (z) is a bounded and analytic function in an open domain containing fz 2 C j jz j 1g. A tighter set of conditions can be found in [5, p. 71]. The following lemma is the discrete-time equivalent of Lemma 3.10. Lemma 4.6. Assume the matrices B and A are such that 1 ab (z)B + A is a

block-consistently ? ordered matrix with non-singular diagonal blocks. Assume the spectrum KJAC?(z) lies on a line-segment [?(1 ) (z); (1) (z)] with (1 ) (z) 2 C and j(1) (z)j = KJAC (z) < 1. The spectral radius of KCSOR (z) is then minimised ~ for a given value of z by the unique optimum ( opt ) (z), given by

( ~ opt ) (z) =

(4.7)

p

2 p ; 1 + 1 ? (1 )2 (z)

where denotes the root with the positive real part. In particular, ;opt (z)) = j(

~ opt ) (z) ? 1j : (4.8) (KCSOR Proof. In analogy with Lemma 3.10, the result follows from standard complex SOR theory applied to the complex matrix 1 ab (z)B + A. 4.4. Continuous-time versus discrete-time results. The discrete-time and continuous-time ESOR symbols are related by 1 a (z) : ESOR (4.9) KESOR (z) = K b As a result, we have the following two theorems which provide the spectral radius of the discrete-time operator in terms of the symbol of the continuous-time operator. They correspond to Theorems 4.1 and 4.4 in [9], so we can omit their proofs. Theorem 4.7. Assume that the discrete solvability condition (2.11) is satis ed, and consider KESOR as an operator in lp (N) with 1 p 1 and N nite. Then,

(4.10)

(KESOR ) =

KESOR 1 k

k

:

Theorem 4.8. Assume (?DB?1 DA ) int S , and consider KESOR as an op-

erator in lp (1), 1 p 1. Then, (4.11) (KESOR ) = supf(KESOR (z)) j z 2 C n int S g = sup (KESOR (z)) : z2@S

Also, in complete analogy to the result in [9, x4.3], we can show that ? ? (4.12) lim KESOR = KESOR ; !0 for both the nite and in nite time-interval computation.


17

In the CSOR case, we have a more restricted analogue of (4.9), i.e., 1 a a 1 CSOR CSOR ~ ~ if (z) = b (z) : K (z) = K b (z) The latter equality is not necessarily satis ed. It does hold, however, in the important case explained below. The discrete-time and continuous-time Jacobi symbols are obviously related as 1 a JAC JAC K (z) = K b (z) : Therefore, a similar? relation holds for their respective eigenvalues with largest modulus: (1 ) (z) = 1 1 ab (z) . By comparing the formulae for the optimal convolution kernels in Lemmas 3.10 and 4.6, we nd ( ~ opt ) (z) = ~ opt 1 ab (z) : Thus, in the case of convolution with kernels determined by formulae (3.18) and (4.7), we can formulate for the CSOR waveform operator similar results as in Theorems 4.7 and 4.8, and equality (4.12).

5. Point-wise SOR waveform relaxation: model problem analysis and numerical results. We consider the m-dimensional heat equation @ u ? u = 0 ; x 2 [0; 1]m ; t > 0 ; (5.1)

m @t with Dirichlet boundary conditions and a given initial condition. We discretise using nite dierences or nite elements on a regular grid with mesh-size h. 5.1. Finite-dierence discretisation. A nite-dierence discretisation of (5.1) with central dierences leads to a system of ODEs of the form (1.1) with mass matrix B = I and stiness matrix A, which, in the one-dimensional and two-dimensional case, is given respectively by the stencils 2 3 ?1 1 1 A = h2 [?1 2 ? 1] and A = h2 4 ?1 4 ?1 5 : ?1

Matrix A is consistently ordered for an iteration with point-wise relaxation in a lexicographic, red/black or diagonal ordering. The eigenvalues of the Jacobi iteration matrix corresponding to matrix A are well-known. They are real and one has independent of the spatial dimension that (5.2) 1 = (KJAC (0)) = cos(h) : Hence, the assumptions of Theorems 3.11, 3.12 and 3.13 are satis ed. In Figure 5.1, we illustrate formulae (3.20) and (3.21) by depicting (KSSSOR ) and (KESOR ), SOR ), the spectral radius of the standard SOR method for the linear together with (Kalg system Au = f, as a function of !. In [17, p. 473], Miekkala and Nevanlinna derived the following result for the optimal single-splitting SOR waveform method from Theorem 3.11.

18


3.0

h = 1=8

2.5

1.0 0.5 0.0

... ...

2.0 1.5

... ...

...

... ...

... ...

. .. ..... .... ... ..... .... . ... . . . ... .. .... .... ... ..... ... ... ..... . .... . . . . . . . ..................................................... ... ...... ............................. .. .................... .... .. ...... .................... .. ..................... .. .. .. .. . . . .. . ..

+

+ + + + + +

0.0

0.5

1.0

1.5

2.0

3.0

3.0

2.5

2.5

2.0

2.0

1.5

1.5

1.0

1.0

0.5

0.5

0.0

0.0

h = 1=32

.. ...

.. ...

.. ...

... ...

.. ...

.. .... .... ... .... .... . . ... .... .... ... .... ... .... .... . . . . . . .. ..... ... ..... ...... .................................................................................................................................................... . . . . . . . .. .. . ... ... ...

+ ... ...

+ + + + + +

3.0 2.5 2.0 1.5 1.0 0.5

0.0

0.5

1.0

1.5

2.0

0.0

SSSOR ) (solid), (KESOR ) (dashed) and (KSOR ) (dots) vs. ! for model problem Fig. 5.1. (K alg (5.1) with nite-dierence discretisation. The '+'- and ''-symbols indicate measured values from numerical experiments (x5.4).

Property 5.1. Consider model problem (5.1), discretised with nite dierences. Then, if we consider KSSSOR;!opt as an operator in Lp (0; 1), 1 p 1, we have for small h that

(5.3)

(KSSSOR;!opt ) 1 ? 22 h2 ; !opt 34 ? 94 2 h2 :

A similar result can be proven for the optimal ESOR waveform relaxation method. The calculation is standard, but rather lengthy and tedious. It was performed by using the formula manipulator Mathematica, [26]. The computation is based on rst dierentiating (3.21) with respect to ! and then nding the zeros of the resulting expression. The formula for !opt is then substituted back into (3.21). Finally, entering (5.2) and calculating a series expression for small h leads to the desired result. Property 5.2. Consider model problem (5.1), discretised with nite dierences. Then, if we consider KESOR;!opt as an operator in Lp (0; 1), 1 p 1, we have for small h that

(5.4)

(KESOR;!opt ) 1 ?

p

p 2 2 h2 ; !opt (4 ? 2 2) + (3 ? 9 4 2 )2 h2 :

p

For the CSOR waveform relaxation method with optimal overrelaxation kernel we may invoke Property 3.1. The operator's spectral radius equals that of the optimal SOR iteration matrix for the discrete Laplace operator. Property 5.3. Consider model problem (5.1), discretised with nite dierences. Then, if we consider KCSOR;opt as an operator in Lp (0; 1), 1 p 1, we have for small h that

(5.5)

(KCSOR;opt ) 1 ? 2 h :

Numerical values of these spectral radii, together with the corresponding !opt are presented in Table 5.1 as a function of the mesh-size h. They are computed from (3.20), (3.21) and (3.25).


19

Table 5.1

Spectral radii of optimal single-splitting (SSSOR), extrapolation (ESOR) and convolution (CSOR) SOR waveform relaxation for model problem (5.1) with nite-dierence discretisation. The value of the optimal parameter !opt is given in parenthesis.

h 1/8 1/16 1/32 1/64 (KSSSOR;!opt ) 0.745 (1.2713) 0.927 (1.3166) 0.981 (1.3291) 0.995 (1.3323) (KESOR;!opt ) 0.804 (1.1452) 0.947 (1.1647) 0.986 (1.1698) 0.997 (1.1711) (KCSOR;opt ) 0.446 0.674 0.821 0.906

5.2.Finite-element discretisation. A nite-element discretisation of (5.1) does not in general lead to a matrix zB+A that is consistently ordered for point-relaxation. This precludes the use of Lemma 3.8. An exception is the one-dimensional model problem (5.1) discretised with linear nite elements. In that case one obtains a system of ODEs of the form (1.1) where B and A are given by the stencils (5.6) B = h6 [ 1 4 1 ] and A = h1 [ ?1 2 ? 1 ] : The spectral radius of the ESOR waveform relaxation operator can be found based on the use of Lemma 3.8. We obtain a result similar to the result of Theorem 3.12. Theorem 5.1. Consider (1.1) with B and A given by (5.6). Assume 0 < ! < 2. Then, if we consider KESOR as an operator in Lp (0; 1), 1 p 1, we have with 1 = cos(h) that 8 > >
> :

(5.7)

q

1 ? ! + 21 (!1 )2 + !1 1 ? ! + 14 (!q1 )2 ! !d 1 2 17(!1 ) + 64(! ? 1) + 48!1 ! ? 1 + 8 (!1 )2 ! > !d ; (! ? 1) 64(! ? 1) ? (!1 )2

with !d the value of ! for which the two expressions in (5.7) are equal. Proof. The spectrum of the Jacobi symbol is given by 2 (5.8) (KJAC (z)) = ?2zh + 12 j 1 j 1 ? 1 with j = cos(jh) :

4zh2 + 12 h Since the conditions of Lemma 3.8 are satis ed, (3.17) can be written as p 2zh2 + 12 ; (z) + ! ? 1 = (z)! ?4zh 2 + 12 j or, after setting (z) = j(z)jeit, i t + (! ? 1)e?i t ? pj(z)j!j j (z) j e ? 3 p (5.9) z = h2 : j(z)jei t + (! ? 1)e?i t + 21 j(z)j!j If the supremum in (3.10) is attained for z = 0, we get (3.23). If the supremum in (3.10) is attained at a complex point z = i, 6= 0, we impose the conditions Re z(t) = 0 and Re z 0 (t) = 0 on (5.9) to obtain p 4j(z)j(! ? 1) cos2 2t ? 21 j(z)j!j (j(z)j + ! ? 1) cos 2t (5.10) +(j(z)j + ! ? 1)2 ? 21 j(z)j!22j = 0 2

2

2

2

20


4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0

h = 1=8

+

+

+ + + +

. ... ..

+

.. .......... . ......... . .......... . ........ . .... . .......... . ......... . ......... . . .

.. ....

.. ....

. ..

... ...

... ...

... ...

... ...

. .. .. . ... .. .

..

.. ...

4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0

... ..

. ..

..

0.0

4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0

h = 1=32

. .. ..

... ..

.

+

+

+

... ...

.. ...

... ...

... ...

... ..

.. ...

... ... ... . . . . .......... . ......... . .......... . ......... . .......... . ......... . .......... . . . . . . . . . . . . . . . ... . . . . .

+ + + +

4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0

0.5 1.0 1.5 2.0 0.0 0.5 1.0 1.5 2.0 ESOR ) (dashed) and (KSOR ) (dots) vs. ! for model problem (5.1) with m = 1 Fig. 5.2. (K alg and linear nite-element discretisation. The '+'-symbols indicate measured values from numerical experiments (x5.4).

and (5.11)

p 4j(z)j(! ? 1) cos 2t ? 41 j(z)j!j (j(z)j + ! ? 1) = 0 : ? Eliminating cos 2t from (5.10) and (5.11) gives a second-order equation in j(z)j whose largest solution equals q 17(!j )2 + 64(! ? 1) + 48!j ! ? 1 + 18 (!j )2 (5.12) j(z)j = (! ? 1) : 64(! ? 1) ? (!j )2 The proof is completed by combining (3.23) and (5.12), p which obtains its maximum alg value for j = 1, and noting that !d < !opt = 2=(1 + 1 ? 21 ). Equation (5.7) is illustrated in Figure 5.2, where the theoretical values of the spectral radius are plotted against !. The spectral radius of the CSOR waveform relaxation operator with optimal kernel is calculated in the following theorem. As in the nite-dierence case, it equals the spectral radius of the optimal standard SOR method for the system Au = f where A is the discrete Laplacian. Theorem 5.2. Consider (1.1) with B and A given by (5.6). Then, if we consider KCSOR;opt as an operator in Lp (0; 1), 1 p 1, we have for small h that (5.13) (KCSOR;opt ) 1 ? 2h :

Proof. Because of (5.8), we have that the eigenvalues of KJAC (z) lie on the line-

segment [?1(z); 1 (z)] with j1(z)j < 1. Therefore, the conditions of Lemma 3.10 are satis ed. Since 1 (z) is an analytic function for Re(z) 0, so is ~ opt (z). Hence, we may apply Theorem 3.7 and Lemma 3.10 to nd (KCSOR;opt ) = sup j ~ opt(i) ? 1j = j ~ opt (0) ? 1j ;

R

2

from which the result follows. Table 5.2 shows some values of !d , !opt and (KESOR;!opt ) calculated by means of (5.7). For comparison purposes we have also added the spectral radii of optimal convolution SOR waveform relaxation. Note that the latter are identical to the ones in Table 5.1.


21

Table 5.2

Parameters !d and !opt , together with spectral radii of optimal extrapolation (ESOR) and convolution (CSOR) SOR waveform relaxation for model problem (5.1) with m = 1 and linear nite-element discretisation.

h !d !opt

1/8 1/16 1/32 1/64 1.0599 1.0687 1.0710 1.0716 1.0625 1.0694 1.0712 1.0716 (KESOR;!opt ) 0.834 0.956 0.989 0.997 (KCSOR;opt ) 0.446 0.674 0.821 0.906 Table 5.3

Spectral radii of discrete-time optimal extrapolation (ESOR) and convolution (CSOR) SOR waveform relaxation for model problem (5.1) with m = 1 and linear nite-element discretisation (h = 1=16, = 1=100).

multistep method CN BDF(1) BDF(2) BDF(3) BDF(4) BDF(5) ? (KESOR;! ) 0.956 0.956 0.956 0.991 1.236 2.113 (KCSOR;opt ) 0.674 0.674 0.674 0.674 0.674 0.674

5.3. Eect of time-discretisation. We analyse the use of the Crank{Nicolson (CN) method and the backward dierentiation (BDF) formulae of order 1 up to 5, for the one-dimensional model problem (5.1) with linear nite-element discretisation on a mesh with mesh-size h = 1=16. The results for nite dierences or more-dimensional problems are qualitatively similar. We computed (KESOR;!? ) by direct numerical evaluation of (4.5), with = 1=100 and with !? equal to the optimal ! for the continuous-time iteration. Since (1 ) (z) is analytic and smaller than one in magnitude for jz j 1, we have that ( ~ opt ) (z), given by (4.7), is also analytic. Hence, we can also compute (KCSOR;opt ) by evaluation of (4.5). The results for the ESOR and CSOR iteration are reported in Table 5.3. They can be illustrated by a so-called spectral picture, see [9, x6], in which the scaled stability region boundaries of the linear multistep methods are plotted on top of the contour lines of the spectral radius of the continuous-time waveform relaxation symbol. Two such pictures are given in Figure 5.3, with contour lines drawn for the values 0.8, 1.0, 1.2, 1.4, 1.6, 1.8, 2.0 and 2.2 in the ESOR case and 0.6, 0.7, 0.8 and 0.9 in the CSOR case. According to Theorem 4.8 and its optimal CSOR equivalent, the values of Table 5.3 can be veri ed visually by looking for the supremum of the symbol's spectral radius along the plotted scaled stability region boundaries. 5.4. Numerical results. In this paragraph we will present the results of some numerical experiments. We will show that the observed convergence behaviour agrees very well with the theory. For long enough time-windows the averaged convergence factors closely match the theoretical spectral radii on in nite time-intervals. For an explanation of why they do not agree with the nite-interval ones, we refer to [9, x7.1] and to the pseudo-spectral analysis of [7, 16]. To determine the averaged convergence factor, we rst computed the -th iteration convergence factor by calculating the l2 norm of the discrete error of the -th approximation u( ), and by dividing the result for successive iterates. This factor takes a nearly constant value after a suciently large number of iterations. The averaged convergence factor is then de ned as the geometric average of these iteration convergence factors in the stable regime.

22

J. JANSSEN and S. VANDEWALLE 600 400 200 0

?200 ?400

.. .. . . .. . . . .. . .. .. . .. ......... .. . .. .... .. .. . . .. .. .. ..... .. .. .. ... ........... .. .. .... .. .. .. .. .. ..... .. .. .... .. .. . . ........ .. .. ....... .. .. .. ........ .. .. ..... .. .. . .. ..... ... .. . ..... .. .. .. .. ...... .. .. ........................................... .... . . . . .. . .. . ...... ......... ...... ....... .. . .. ..... .... . ............ ....... ....... .. .. .. ... ..... ......... ..... .... ...... .. .. .. ..... .... ....... .... ..... . . . ... .. ... ... . . . . . . . . . . . .... . ..... . . .... ..... . .. ..... . . .. ... . ................................ ... ...... ... .. ........... .. .... ....... ............ . . . .. ..... . .. ............. ...... ........ ....... . .. . . ... .. . . . ....... .. ... ... .. . . . . . . . . . .. . . . . . . ... . . .. . . ............. .................... ... . . . . . . ..... ..... ...... . . .. .. . . . . . ....... . .. .... . . . . . . . . . . . . . . . .. .. ... .. .. .. .. .. .. ....... .. .. . . .. . . . .. .. .. .. .. ...... .. . . . . . . .. .. . .. .. .. .. .. .. .......... .. . . . . . . .. . .. .. .. .. .. .. ............................. ............. . . . . . . . . . ................ . .... .. .. . . . . . . . . . . . . . . . . . . . . . . . . .. ... .. . . ....... .. ... .... .... ... . .. .......... .. .... ...... ........ ..... .. .. ......... .. .. .... ........ .. .............. ...... ... ....................... ... ......... .. .. ..... ........ .. ... . . . ... ... .... . . . . . . . . . ... .... . . . . .. .... .... ........ .. .. .. ... ..... ......... ..... ....... . .. .. .... .... .. .......... ...... ...... ....... ........ .. .. .. ..... ..... .. .......... ...... ........................................ ...... .. .. .. .. ..... ... .. .. .... .. .. . .. ..... ... . .. ....... .. .. .. ........ .. .. .. .... .. .. .. .. ....... .. .. .. .... .. .. .. .. ... ......... . . .. .. ... .. . .. ... . ...... .. . .. ..... . .. .. .. .. ........ ..... .. .. . ... . . .. .

=0:8

..... ...... .......

..... ...... ...... .

600 400

=0:6

=2:2

?600 ?400 ?200 0

. . .. ...... . .. . ..... . .. ..... . . ....... .. . . ....... .. . . . . . .. .. . ..... . . . ... . . . . . .. . . ..................................... . .. .......... ....... ... . . ....... ...... . .. . .. . ...... ....... . .. . .. . .... ...... . .. ..... . ...... . . . . .... ... . . . . .... . .... ... . . . ... . . . ... .... . ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ....... ... ... .. .. ... ..... ..... .... ...... . .. . . . . . . ..... .... .... .. . ...... ... . ..... ..... . . .. . . .. . . . ... ...... . . . ... .. .. ... . ...... ...... ................... .. . . .. ... ........................ .. . ... ...... .... . . ... ....... . . . . . . .. .. .... . . .... ........ .... . . . . . . . .. ...... ....... .. .. .. ... . .. ........ .. . ..... .. . .......... . . . . . .. ..... . . . . . . . . . . . . . . . . . . . .. ...... .... ... ... . . . ............. . . . . . . . . . . . . . . . . . . .. . ........ . . .... .... . ... ..... ...... . . .... ...... . . . . . . . . . . . . .... ........... .. .. .............. .. .. .... . . . . . . . . . . . . . . . . . . . . . .. .... .. . .. .... ..... ... . .... .... .. .... . ..... ..... .... . ... ... ....... .... ..... .... .. ... .. ... ... ................................ .. . . ... ... ..... .. . .... . . ... . . . . . . . . . ..... . . .. ..... . . . ... . ........ ... . . . .. . ...... ...... .. .. . . ...... ...... .. .. . . ....... ......... . .. . .. ........................................... . . .. . .. . ... . . . .... . . . .. .. ..... . .. . . ....... .. . . . ........ . . . .... . .. . ...... .. . . ..

200

=0:9

0

?200 ?400

?600 200 400 600 800?400 ?200 0 200 400 600 800 Fig. 5.3. Spectral pictures of optimal extrapolation (left) and convolution (right) SOR waveform relaxation for model problem (5.1) with m = 1 and linear nite-element discretisation (h=1/16, =1/100). ..... ..... ......

. ..... .... ......

For the problems discussed in this paragraph, the Z-transform of the optimal convolution kernel is known to be analytic for jz j 1 and is given by (4.7). The computation of the components opt [n] of the optimal discrete convolution kernel, which satisfy (5.14)

( ~ opt ) (z) =

1 X

n=0

opt [n]z ?n ;

involves the use of an inverse Z-transform technique. The method we used is based on a Fourier-transform method, and is justi ed by the following observation. Setting

(t) = ( ~ opt ) (e?it ), (5.14) becomes 1

X

(t) = opt [n]eint :

n=0

Thus, opt [n] is the n-th Fourier-coecient of the 2-periodic function (t). More precisely, Z 2 1

(t)e?intdt ;

opt [n] = 2 0 which can be approximated numerically by the nite sum MX ?1 2

num[n] = M1

k M e?ink M : 2

k=0

Consequently, numerical approximations to the M values f opt [n]gnM=0?1 can beofound n M ?1 by computing the discrete Fourier-transform of the sequence ( ~ opt ) (e?ik M ) k=0 . This can be performed very eciently by using the FFT-algorithm. In the numerical experiments the number of time-steps N is always nite. Hence, we only use the ?1. To compute the discrete kernel, we took numerical approximations of f opt [n]gNn=0 M N and large enough to anticipate possible aliasing eects. 2


23

Table 5.4

Averaged convergence factors of optimal single-splitting (SSSOR), extrapolation (ESOR) and convolution (CSOR) SOR waveform relaxation for model problem (5.1) with m = 1, nite-dierence discretisation and Crank{Nicolson method ( = 1=100). Compare with Table 5.1.

h 1/8 1/16 1/32 1/64 SSSOR 0.713 0.919 0.979 0.995 ESOR 0.783 0.942 0.985 0.996 CSOR 0.441 0.676 0.820 0.907 Table 5.5

Averaged convergence factors of optimal single-splitting (SSSOR), extrapolation (ESOR) and convolution (CSOR) SOR waveform relaxation for model problem (5.1) with m = 2, nite-dierence discretisation and Crank{Nicolson method ( = 1=100). Compare with Table 5.1.

h 1/8 1/16 1/32 1/64 SSSOR 0.718 0.921 0.980 0.995 ESOR 0.788 0.944 0.986 0.996 CSOR 0.442 0.670 0.822 0.909

The correctness of formulae (3.20) and (3.21) is illustrated in Figure 5.1 by the '+'- and the ''-symbols. They correspond to the measured averaged convergence factors of the single-splitting and extrapolation SOR waveform relaxation method respectively, applied to the one-dimensional model problem (5.1). In this computation the time-window equals [0; 1], the time-step is 1=1000 and the time-discretisation is by the Crank{Nicolson method. Averaged convergence factors as a function of h are given in Tables 5.4 and 5.5 for the one- and two-dimensional model problem (5.1). They agree very well with the theoretical values given in Table 5.1, and they illustrate the correctness of formulae (5.3), (5.4) and (5.5). Note that we take the overrelaxation parameters ! in the numerical experiments equal to the optimal parameters !opt of the corresponding continuous-time iterations, see also x5.3. In order to illustrate the dramatic improvement of convolution SOR over the other SOR waveform relaxation methods we included Figure 5.4. There, we depict the evolution of the l2 -norm of the error as a function of the iteration index. The results for standard Gauss{Seidel waveform relaxation are also given. The correctness of (5.7) is illustrated in Figure 5.2 where the '+'-symbols correspond to observed averaged convergence factors for the extrapolation SOR waveform relaxation method, applied to the one-dimensional model problem (5.1) with linear nite-element discretisation, time-window [0; 1], = 1=1000 and Crank{Nicolson time-discretisation. In Table 5.6 we present numerical results as a function of h for optimal extrapolation and convolution SOR waveform relaxation. These values should be compared to the ones given in Table 5.2. Moreover, the CSOR results illustrate the correctness of (5.13). Finally, averaged convergence rates obtained with dierent time-discretisation formulae are given in Table 5.7. They match the theoretical values of Table 5.3 very well. 6. Line-wise CSOR waveform relaxation. A nite-element discretisation of model problem (5.1) does not in general lead to matrices zB +A that are consistently ordered for point-wise relaxation. These matrices may, however, be consistently ordered for block-wise or line-wise relaxation. As an illustration, we investigate the performance of line-wise convolution SOR waveform relaxation for the nite-element

24

J. JANSSEN and S. VANDEWALLE 3 2 1 0 ?1 ?2 ?3 ?4 ?5 ?6 ?7 ?8 ?9

...... ......... . . ................... . .......... . ... ...... .... .... . . ..... .... ..... . ...... .. ... . . ...... ..... ........ . ...... . ..... ....... . ....... . ...... . . .... ...... . .... . ...... ..... . . ........ . . . ....... . . . ... ...... . .... . . ..... ...... ...... . ........ . ..... . . ..... ...... ....... . ... . . . . . . ...... ...... ...... ..... . . ..... . ...... . .... . ....... ..... . ....... ...... . . .... . . ...... ..... . .... . ..... . . ........ ..... ...... . . . ..... . . . . . . ..... ... . ...... . ..... ....... . . ..... . . ... ...... ..... . ..... . . ..... . ....... ....... ..... . .. . ..... . ...... . ...... ..... . ... .. . .... ..... ..... . . . . . .. . ..... ..... . ........ .... . .... ...... .... . ..... . . . . . . .... . .. . ..... .... ...... ... . .... .. . ..... ... .... ..... . ... ... . ... .... . .. ... ... .. . .. ... ... . .. .. ..

0

200 400 600 800 1000 1200 1400 1600 1800 2000

. log jje( ) jj2 vs. iteration index for model problem (5.1) with m = 2, nite-dierence discretisation and Crank{Nicolson method (h = 1=32, = 0:01), using Gauss{Seidel (dash-dotted), single-splitting SOR (solid), extrapolation SOR (dashed) and convolution SOR (dotted) waveform relaxation. Fig. 5.4

Table 5.6

Averaged convergence factors of optimal extrapolation (ESOR) and convolution (CSOR) SOR waveform relaxation for model problem (5.1) with m = 1, linear nite-element discretisation and Crank{Nicolson method ( = 1=100). Compare with Table 5.2.

h 1/8 1/16 1/32 1/64 ESOR 0.817 0.952 0.988 0.997 CSOR 0.441 0.676 0.819 0.908 Table 5.7

Averaged convergence factors of optimal extrapolation (ESOR) and convolution (CSOR) SOR waveform relaxation for model problem (5.1) with m = 1, linear nite-element discretisation, (h = 1=16, = 1=100). Compare with Table 5.3.

multistep method CN BDF(1) BDF(2) BDF(3) BDF(4) BDF(5) ESOR 0.953 0.953 0.953 0.965 1.221 2.142 CSOR 0.637 0.637 0.632 0.630 0.630 0.629

discretisation of the two-dimensional heat equation. The relevant stencils are given by 2 3 2 3 1 1 ?1 2 h B = 12 4 1 6 1 5 and A = 4 ?1 4 ?1 5 1 1 ?1 in the linear nite-element case, and by 2 2 3 3 1 4 1 ? 1 ? 1 ? 1 2 B = h36 4 4 16 4 5 and A = 13 4 ?1 8 ?1 5 1 4 1 ?1 ?1 ?1 in the bilinear nite-element case. We also study the line-wise CSOR method for the nite-dierence discretisation of the two-dimensional model problem (5.1). The resulting matrices zB + A are block-consistently ordered and, therefore, the Young-relation (3.16) holds. Unfortunately, however, the eigenvalues of the Jacobi


25

Table 6.1

Averaged convergence factors of line-wise convolution SOR waveform relaxation for model problem (5.1) with m = 2 and Crank{Nicolson time-discretisation (h = 1=32, = 1=100). The theoretical spectral radii of the corresponding standard line-wise SOR method are given in parenthesis.

h 1/8 1/16 1/32 1/64 nite dierences 0.318 (0.322) 0.568 (0.572) 0.756 (0.757) 0.871 (0.870) linear nite elements 0.320 (0.322) 0.569 (0.572) 0.757 (0.757) 0.870 (0.870) bilinear nite elements 0.312 (0.317) 0.567 (0.571) 0.760 (0.757) 0.870 (0.870)

symbols KJAC (z) are in general not collinear (except for z = 0 and z = 1). Consequently, formula (3.18) is not guaranteed to give the optimal convolution kernel. Moreover, we cannot use Theorem 3.7 to estimate the spectral radii of the line-wise CSOR waveform relaxation methods since 1 (z) and, hence, ~ (z) are not known to be analytic for Re(z) 0. Despite similar violations of the assumptions of Theorem 4.5 and Lemma 4.6, we ?1, used formula (4.7) to compute the convolution sequence ( num ) = f num[n]gNn=0 following the procedure explained in x5.4. That is, we compute (1) (z) for M values of z located equidistantly along the unit circle. We apply (4.7) to compute the corresponding values of ~ (z), or, (t), and we compute the Fourier-transform of this sequence to arrive at M values num [n], n = 0; : : :; M ? 1. As we take M such that M N, we truncate the sequence after the rst N components. This corresponds to using an l1 (1)-kernel of the form (6.1) ( num ) = f num [0]; num[1]; : : :; num[N ? 1]; 0; 0; :: :; 0; : : :g : Numerical results, based on Crank{Nicolson time-discretisation with time-step = 1=100, are reported in Table 6.1. We also included the theoretical spectral radii of the optimal line-wise SOR method for the correspondingplinear systems Au = f. Observe that the latter, which can be approximated by 1 ? 2 2h for small mesh-size h, [3, p. 152], agree very well with the averaged convergence factors of the line-wise CSOR waveform relaxation methods. To illustrate and explain this behaviour we provide Figure 6.1, where we depict (KCSOR (ei )), 2 [?; ], for the two-dimensional heat equation, discretised using linear nite elements on a mesh with mesh-size h = 1=32. From this picture it is clear that the maximum in (4.5) is found very close to = 0 (or z = 1). The maximum is not exactly at the origin though, as we see from Table 6.2 where values of (KCSOR (ei )) are presented for close to 0. By construction, (0) equals the optimal overrelaxation parameter for the linewise SOR method applied to the problem Au = f. By de nition of the inverse Fourier-transform, we have that (6.2)

(0) =

MX ?1 n=0

num[n] :

Because of the rapid decay of the Fourier-coecients, the latter is a very good approximation of the Z-transform of (6.1) at z = 1, ( ~ num) (1), if M > N. In particular, (6.2) equals ( ~ num) (1) if M = N. Hence, in Figure 6.1, the value of the spectral radius of the optimal line-wise SOR method for the stationary problem (or a very good approximation to it) is found at the origin. Since the curve peaks close to the origin, we may expect a similar convergence rate for the line-wise CSOR waveform relaxation method as for the optimal line-wise SOR method for the system Au = f.

26

J. JANSSEN and S. VANDEWALLE 1.0 0.8 0.6 0.4 0.2 0.0

........................................ ........... ........... ............ ........... ........... ............. ............. .............. ............. .............. . . . . . . . . . . . . . . . ............. ............ .............. . . . . . . . . . . . . . . .............. ............. . ............. . . . . . . . . . .......... ............ . . .......... . . . . . . . . .. .......... ......... . ........ . . . . . ...... ....... . . ...... . . . . ....... ...... . . . . ...... . ...... ....... . . . ..... . .... ..... . . . . .... ..... ....... . . . . . ...... .. . ......

?

?=2

0

=2

1.0 0.8 0.6 0.4 0.2 0.0

CSOR (ei )) for line-wise relaxation vs. for model problem (5.1) with m = 2 Fig. 6.1. (K and linear nite-element discretisation (h = 1=32). (KCSOR (ei )) for line-wise relaxation as a function of for model problem (5.1) with m = 2 and linear nite-element discretisation (h = 1=32). Table 6.2

0 0.01 0.02 0.03 0.04 0.05 0.06 i )) (KCSOR ( e 0.757 0.759 0.761 0.761 0.760 0.759 0.756

7. Conclusions. In this paper, we studied dierent SOR waveform relaxation methods for general ODE systems of the form B u_ + Au = f. The methods based on matrix-splitting and extrapolation, which use a single scalar parameter, were shown to lead to some acceleration. This acceleration is, however, only a marginal one. The method based on convolution, using a frequency-dependent overrelaxation parameter, proved to be vastly superior, leading to a convergence acceleration similar to the convergence acceleration of the optimal SOR method for solving stationary problems. It was our aim in this paper to provide the theoretical framework in which to study the dierent SOR waveform methods, and to illustrate the potential convergence acceleration of the convolution method. We realise that in order to cast the latter into a practical procedure more research is required, in particular on how to derive the optimal overrelaxation kernel. For stationary problems, the determination of a good overrelaxation parameter ! is already non-trivial. Finding a good convolution kernel for time-dependent problems is expected to be even far more dicult. Yet, the problem does not seem to be unsurmountable. Some promising results have already been reported in [21], where an automatic procedure is developed for determining a good (t). In addition, our results in x6 show that the lemmas concerning the optimal convolution kernel, i.e., Lemmas 3.10 and 4.6, appear to be \robust". That is to say, even though some assumptions are violated, the use of a convolution kernel based on the former lemmas leads to an excellent convergence acceleration. Acknowledgments. The authors would like to thank Min Hu, Ken Jackson, Andrew Lumsdaine and Mark W. Reichelt for many helpful discussions. REFERENCES [1] A. Bellen, Z. Jackiewicz, and M. Zennaro. Contractivity of waveform relaxation Runge-Kutta iterations and related limit methods for dissipative systems in the maximum norm. SIAM J. Numer. Anal., 31(2):499{523, April 1994. [2] A. Bellen and M. Zennaro. The use of Runge-Kutta formulae in waveform relaxation methods. Appl. Numer. Math., 11:95{114, 1993.


27

[3] G. Birkho and R. E. Lynch. Numerical Solution of Elliptic Systems, volume 6 of SIAM Studies in Applied Mathematics. SIAM, Philadelphia, 1984. [4] R. Curtain and A. Pritchard. Functional Analysis in Modern Applied Mathematics. Academic Press, London, 1977. [5] K. Homan. Banch Spaces of Analytic Functions. Prentice-Hall, Englewood Clis, N.J., 1962. [6] M. Hu, K. Jackson, and B. Zhu. Complex optimal SOR parameters and convergence regions. Technical report, Dept. of Computer Science, University of Toronto, Canada, 1995. (preprint). [7] Z. Jackiewicz and B. Owren. Convergence analysis of waveform relaxation methods using pseudospectra. Numerics no. 2/1995, Dept. of Mathematics, University of Trondheim, Norway, 1995. [8] J. Janssen and S. Vandewalle. Multigrid waveform relaxation on spatial nite-element meshes: The continuous-time case. SIAM J. Numer. Anal., 33(2), April 1996. (to appear). [9] J. Janssen and S. Vandewalle. Multigrid waveform relaxation on spatial nite-element meshes: The discrete-time case. SIAM J. Sci. Comput., 17(1), Januari 1996. (to appear). [10] G. S. Jordan, O. J. Staans, and R. L. Wheeler. Local analyticity in weighted L1 -spaces and applications to stability problems for Volterra equations. Trans. Am. Math. Soc., 274(2):749{782, December 1982. [11] B. Kredell. On complex successive overrelaxation. BIT, 2:143{152, 1962. [12] J. D. Lambert. Computational Methods in Ordinary Dierential Equations. John Wiley and Sons, Chichester, 1973. [13] C. Lubich. Chebyshev acceleration of Picard-Lindelof iteration. BIT, 32:535{538, 1992. [14] C. Lubich and A. Ostermann. Multi-grid dynamic iteration for parabolic equations. BIT, 27:216{234, 1987. [15] A. Lumsdaine. Theoretical and Practical Aspects of Parallel Numerical Algorithms for Initial Values Problems, with Applications. Ph.D.-thesis, Dept. of Electrical Engineering and Computer Science, M.I.T., USA, Januari 1992. [16] A. Lumsdaine and D. Wu. Spectra and pseudospectra of waveform relaxation operators. Csetr-95-14, Dept. of Computer Science and Engineering, University of Notre Dame, USA, June 1995. [17] U. Miekkala and O. Nevanlinna. Convergence of dynamic iteration methods for initial value problems. SIAM J. Sci. Statist. Comput., 8(4):459{482, July 1987. [18] U. Miekkala and O. Nevanlinna. Sets of convergence and stability regions. BIT, 27:554{584, 1987. [19] W. Niethammer and R. Varga. The analysis of k-step iterative methods for linear systems from summability theory. Numer. Math., 41:177{206, 1983. [20] A. D. Poularakis and S. Seely. Elements of Signals and Systems. PWS-Kent Series in Electrical Engineering. PWS-Kent Publishing Company, Boston, 1988. [21] M. W. Reichelt, J. K. White, and J. Allen. Optimal convolution SOR acceleration of waveform relaxation with application to parallel simulation of semiconductor devices. SIAM J. Sci. Comput., 16(5):1137{1158, September 1995. [22] R. Skeel. Waveform iteration and the shifted Picard splitting. SIAM J. Sci. Stat. Comput., 10(4):756{776, July 1989. [23] V. Thomee. Galerkin Finite Element Methods for Parabolic Problems, volume 1054 of Lecture Notes in Mathematics. Springer-Verlag, Berlin, 1984. [24] S. Vandewalle. Parallel Multigrid Waveform Relaxation for Parabolic Problems. B.G. Teubner, Stuttgart, 1993. [25] R. S. Varga. Matrix Iterative Analysis. Prentice-Hall, Englewood Clis, N.J., 1965. [26] S. Wolfram. Mathematica, a System for Doing Mathematics by Computer. Addison-Wesley, New York, 1988. [27] D. M. Young. Iterative Solution of Large Linear Systems. Academic Press, New York, 1971.