Implementing Scrambled Digital Sequences

Implementing Scrambled Digital Sequences Hee Sun Hong and Fred J. Hickernell Hong Kong Baptist University

Random scrambling of deterministic (t, m, s)-nets and (t, s)-sequences eliminates their inherent bias while retaining their low discrepancy properties. This article describes an implementation of two types of random scrambling, one proposed by Owen and another proposed by Faure and Tezuka. The four different constructions of digital sequences implemented are those proposed by Sobol’, Faure, Niederreiter, and Neiderreiter and Xing. Because the random scrambling involves manipulating all digits of each point, the code must be written carefully to minimize the execution time. Computed root mean square discrepancies of the scrambled sequences are compared to known theoretical results. Furthermore, the performances of these sequences on various test problems are discussed. Categories and Subject Descriptors: F.2.1 [Theory of Computation]: Numerical Algorithms and Problems; G.1.4 [Mathematics of Computing]: Quadrature and Numerical Differentiation; G.4 [Mathematics of Computing]: Mathematical Software Additional Key Words and Phrases: Scrambling, Digital net

1. INTRODUCTION Scrambled (t, s)-sequences were proposed by Art Owen [Owen 1995; Owen 1997a; Owen 1997b; Owen 1998; Owen 1999] as a hybrid of Monte Carlo and quasiMonte Carlo methods. Quasi-Monte Carlo methods [Niederreiter 1992; Tezuka 1995; Hellekalek and Larcher 1998; Fox 1999] use (deterministic) low discrepancy sequences to obtain typically higher convergence rates than Monte Carlo methods. However, it is easier to estimate the error of Monte Carlo methods because one can perform a number of replications and compute the variance. Clever randomizations of quasi-Monte Carlo methods combine the best of both by yielding higher accuracy with practical error estimates. There is a cost in scrambling (t, s)-sequences. The manipulation of each digit different addresses Address: Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong SAR, China different addresses Email: [email protected], [email protected] Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or direct commercial advantage and that copies show this notice on the first page or initial screen of a display along with the full citation. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, to redistribute to lists, or to use any component of this work in other works, requires prior specific permission and/or a fee. Permissions may be requested from Publications Dept, ACM Inc., 1515 Broadway, New York, NY 10036 USA, fax +1 (212) 869-0481, or [email protected].

2

·

Hee Sun Hong and Fred J. Hickernell

of each point requires more time than for a non-scrambled sequence. This article proposes an efficient method of scrambling that minimizes this cost. The algorithm given here builds upon the algorithms of [Bratley and Fox 1988; Bratley et al. 1992] and includes the digital Niederreiter-Xing sequences [Niederreiter and Xing 1996; Niederreiter and Xing 1998; Pirsic 2001]. To save time a somewhat restricted form of Owen’s scrambling is used as explained below. However, this restriction does not appear to have any serious drawbacks. Faure and Tezuka’s scrambling method is also implemented [Faure and Tezuka 2001]. The scrambling algorithms described here apply only to digital (t, s)-sequences. However, all known general constructions of (t, m, s)-nets and (t, s)-sequences are digital constructions [Sobol’ 1967; Faure 1982; Niederreiter 1988; Larcher 1998a; Larcher 1998b; Niederreiter and Xing 1998]. The remainder of this section introduces digital sequences and Owen’s original scrambling. Section 2 explains the scrambling method used here, and the section after that describes the implementation details. Section 4 discusses the time and space complexity of the scrambling algorithm. Section 5 shows the root mean square discrepancies of scrambled sets. Two practical examples are explored in Section 6. The last section contains some concluding remarks. Let b ≥ 2 denote a prime power, and let Zb = {0, 1, . . . , b − 1} be the finite (Galois) field of order b. For any non-negative integer i = · · · i3 i2 i1 (base b), define the ∞ × 1 vector ψ(i) as the vector of its digits, i.e., ψ(i) = (i1 , i2 , . . . )T . For any point z = 0.z1 z2 · · · (base b) ∈ [0, 1), let φ(z) = (z1 , z2 , . . . )T denote the ∞×1 vector of the digits of z. Let C1 , . . . , Cs denote predetermined ∞ × ∞ generator matrices. The digital sequence in base b is {y0 , y1 , y2 , . . . }, where each yi = (yi1 , . . . , yis )T ∈ [0, 1)s is defined by φ(yij ) = Cj ψ(i),

j = 1, . . . , s, i = 0, 1, . . . .

(1)

Here all arithmetic operations take place in the finite field Zb . Thus, the right side of (1) should not give a vector ending in an infinite trail of b − 1s almost surely. To insure this, it is assumed that any linear combination of columns of any C j cannot be a vector ending in an infinite trail of b − 1s. The quality of a digital sequence is expressed by its t-value, which can be determined from the generator matrices. For any positive integer m let cTjmk be the row vector containing the first m columns of the k th row of Cj . Let t be an integer, 0 ≤ t ≤ m, such that for all s-vectors d = (d1 , . . . , ds ) of non-negative integers with d1 + · · · + ds = m − t, the vectors cjmk , k = 1, . . . , dj , j = 1, . . . , s are linearly independent over Zb .

(2)

Then for any non-negative integer ν and any λ = 0, . . . , b − 1 with λ ≤ b − (ν mod b), the set {yνbm , . . . , y(ν+λ)bm −1 } defined by (1) is a (λ, t, m, s)-net in base b. (Note that a (1, t, m, s)-net is the same as a (t, m, s)-net.) If the same value of t holds for all non-negative integers m, then the digital sequence is a (t, s)-sequence. Let {x0 , x1 , x2 , . . . } denote the randomly scrambled sequence proposed by Owen.

Scrambled Digital Sequences

·

3

Let xijk denote the k th digit of the j th component of xi , and similarly for yijk . Then xij1 = πj (yij1 ),

xij2 = πyij1 (yij2 ), xij3 = πyij1 ,yij2 (yij3 ), xijk = πyij1 ,yij2 ,··· ,yijk−1 (yijk ), . . .

... ,

where the πa1 a2 ... are random permutations of the elements in Zb chosen uniformly and mutually independently. A proof that a randomized net preserves the property of (t, m, s)-net almost surely can be found in [Owen 1995]. 2. THE METHOD OF SCRAMBLING Owen’s scrambling can be rather tedious because of the bookkeeping required to store the many permutations. Here an alternative is offered that is only slightly less general than Owen’s proposal. This is called an Owen-scrambling to recognize that it is done in the spirit of Owen’s original proposal. Another type of scrambling proposed by Faure and Tezuka [Faure and Tezuka 2001] is also described below. Let L1 , · · · , Ls , be nonsingular lower triangular ∞×∞ matrices and let e1 , · · · , es be an ∞×1 vector. Assume that for all j any linear combination of columns of L j Cj plus ej does not end in an infinite trail of b − 1s. A particular Owen-scrambling {xi } of a digital sequence {yi } is defined as φ(xij ) = Lj φ(yij ) + ej = Lj Cj ψ(i) + ej ,

j = 1, . . . , s, i = 0, 1, . . . .

(3)

The left multiplication of each generator matrix Cj by a nonsingular lower triangular matrix Lj gives a new generator matrix that satisfies the same condition (2) as the original. The addition of a ej just acts as a digital shift and also does not alter the t-value. Therefore the scrambled sequence is also a digital (t, s)-sequence. To get a randomly scrambled sequence one chooses the elements of the L j and ej randomly. The resulting randomized sequence has properties listed in the theorem below. Although these properties are not equivalent to the Owen’s original randomization, they are sufficient for the analysis of scrambled net quadrature rules given in [Owen 1995; Hickernell 1996; Owen 1997a; Owen 1997b; Owen 1998; Hickernell and Hong 1999; Yue 1999; Yue and Mao 1999; Hickernell and Yue 2001]. Theorem 1. Let {xi } be an Owen-scrambling of a digital (t, s)-sequence where the elements of L1 , · · · , Ls , e1 , · · · , es are all chosen randomly and independently. The diagonal elements of L1 , · · · , Ls are chosen uniformly on {1, . . . , b − 1}, and the other elements are chosen uniformly on Zb . Then this randomly scrambled sequence satisfies the following properties almost surely: i. xijk distributed uniformly on Zb ; ii. if yijh = y`jh for h < k and yijk 6= y`jk , then a. xijh = x`jh for h < k b. (xijk , x`jk ) are uniformly distributed on {(x, y) ∈ Z2b : x 6= y}; c. xijh , x`jh are independent for h > k iii. (xij , x`j ) is independent from (xir , x`r ) for j 6= r Proof. The qualification “almost surely” is required to rule out the zero probability event that for some j there exists a linear combination of columns of L j Cj plus ej that ends in an infinite trail of b − 1s. Assertion iii. follows from the fact

4

·


that the elements of Lj , ej , Lr , and er are chosen independently of each other. Now the other two assertions are proved. Note from (3) that xijk = lTjk φ(yij ) + ejk , where lTjk is the k th row of Lj , and all arithmetic operations are assumed to take place in the finite field. Since e jk is distributed uniformly on Zb , it follows that xijk is also distributed uniformly on Zb . To prove ii. consider φ(xij )−φ(x`j ) = Lj [φ(yij )−φ(y`j )]. Under the assumption of ii. the first k−1 rows of [φ(yij )−φ(y`j )] are zero, which implies that the first k−1 rows of φ(xij ) − φ(x`j ) are also zero, since Lj is lower triangular. This immediately implies iia. Furthermore, xijk − x`jk = lTjk [φ(yij ) − φ(y`j )] = ljkk (yijk − y`jk ), where ljkk is the k th diagonal element of Lj . Since yijk − y`jk 6= 0, and ljkk is a random nonzero element of Zb , it follows that xijk − x`jk is a random nonzero element of Zb . Combined with i., this implies iib. For h > k it follows that xijh − x`jh = lTjh [φ(yij ) − φ(y`j )] = · · · + ljhk (yijk − y`jk ) + · · · . The fact that yijk − y`jk 6= 0, and ljhk is uniformly distributed on Zb , combined with i. now imply iic. To understand why the randomization given by (3) is not as rich as that originally prescribed by Owen, consider randomizing yi11 , the first digit of yi1 for different values of i. The yi11 take on b different values, and there are b! possible permutations of these values. However, the formula given by (3) is xi11 = l111 yi11 + e11 . Since there are only b − 1 possible values for l111 and only b possible values for e11 , this formula cannot give at most b(b − 1) permutations. The Faure-Tezuka scrambling [Faure and Tezuka 2001] scrambles the digits of i before multiplying by the generator matrices. Let L be a nonsingular lower triangular ∞ × ∞ matrix and let e be an ∞ × 1 vector with a finite number of nonzero elements. A particular Faure-Tezuka-scrambling, {zi }, of a digital sequence is defined as φ(zij ) = Cj [LT ψ(i) + e],

j = 1, . . . , s, i = 0, 1, . . . .

(4)

A particular Owen-Faure-Tezuka-scrambling, {xi }, of a digital sequence is defined as φ(xij ) = Lj Cj [LT ψ(i) + e] + ej ,

j = 1, . . . , s, i = 0, 1, . . . .

(5)

To get random Faure-Tezuka-scrambling the elements of L and e are chosen randomly. The effect of the Faure-Tezuka-scrambling can be thought of as re-ordering the original sequence, rather than permuting its digits like the Owen-scrambling. For any m, λ = 0, 1, . . . , let i = λbm , . . . , (λ + 1)bm − 1. Then, ψ(i) takes on all possible values in its top m rows. By the same token LT ψ(i) + e takes on all possible values in its top m rows, but not necessarily in the same order. Therefore, the Faure-Tezuka-scrambled (t, m, s)-net, {zλbm , . . . , z(λ+1)bm −1 }, given by (4) is just the same set as the original (t, m, s)-net, {yνbm , . . . , y(ν+1)bm −1 }, given by (1) for some integer ν. 3. IMPLEMENTATION OF THE SCRAMBLED NETS Generators for scrambled Sobol’ [Sobol’ 1967], Faure [Faure 1982], and Niederreiter [Niederreiter 1988] points have been programmed following the Fortran codes of


·

5

[Bratley and Fox 1988; Bratley et al. 1992]. The differences and improvements that we have made are highlighted and explained below. The code for the Niederreiter ¨ points has been extended to higher dimension by Giray Okten (private communication), and we have employed this extension. A generator for Niederreiter-Xing points has been coded based on the generator matrices provided by [Pirsic 2001]. The different scrambled or non-scrambled sequences given by (1), (3), (4), and (5) are implemented with different choices given to the user. In practice one cannot store all the entries of an ∞ × ∞ matrix. Suppose that bM is the maximum amount of points in one run that is required, and that K is the number of digits to be scrambled. Then, one can restrict L1 , . . . , Ls to K × M matrices, C1 , . . . , Cs , L to M × M matrices, e1 , . . . , es to K × 1 vectors, and e to an M × 1 vector. One might think that it is best to scramble all the digits available, but Section 5 provides evidence that this is not necessary. The value of Kmax , the maximum possible value of K, is 31, which corresponds to the number of bits in the largest integer available. When b = 2, the algorithms for generating digital sequences may be implemented more efficiently by storing an array of bits as an integer. For the scrambled Sobol’ generator the value of S, the maximum dimension is 40, the same as the in [Bratley and Fox 1988]. For the scrambled Faure sequence, S = 500, and for the scrambled Niederreiter generator S = 318. For the Niederreiter-Xing points S = 16 based on the generator matrices available so far. The program may be modified to allow for higher dimensions as the generator matrices become available. To save time some matrices and vectors are premultiplied so that (5) becomes ˜ j ψ(i) + e ˜j , φ(xij ) = C

j = 1, . . . , s, i = 0, 1, . . . ,

(6)

˜ j = Lj Cj LT , and e ˜j = Lj Cj e + ej . Another time saving feature is to use where C ˜ a gray code [Lichtner 1998]. Recursively define the ∞ × 1 vector function ψ(i) as follows: ˜ ψ(0) = (0, 0, . . . )T , ˜ + 1) = ψ(i) ˜ − δ ˆ , where kˆi = min{k : bib−k c 6= ib−k }, ψ(i k,ki+1

(7) (8)

δkl is the Kroneker delta function, and where again all arithmetic operations take place in the finite field. For example, if b = 3, then ˜ ψ(1) = (2, 0, 0 . . . )T ,

˜ ψ(2) = (1, 0, 0 . . . )T ,

˜ ψ(3) = (1, 2, 0 . . . )T , . . .

˜ changes as the argument is increased by one. Note that only one element of ψ ˜ Replacing ψ(i) by ψ(i) in (6) still gives one a scrambled digital sequence; it just has the effect of re-ordering the points. The efficiency advantage is that the digits of the i + 1st point can be obtained from those of the i + 1th point by the iteration: ˜ ˆ , φ(xi+1,j ) = φ(xij ) − C j ki+1

(9)

˜ jl is the lth column of C ˜ j. where C The implementations of the scrambled Sobol’ and Niederreiter sequences follow closely the structures of the original programs, and our changes are rather minor. However, the structure of the Faure sequence generator has been altered more substantially to improve efficiency. The scrambled Sobol’, Niederreiter, and

6

·


Niederreiter-Xing generators all have base b = 2, which allows the use of logical operations instead of additions and multiplications. 3.1 Scrambled Sobol’ Generator: SSOBOL INSSOBL is a modified version of INSOBL in [Bratley and Fox 1988] that initializes the values needed by SSOBOL. This subroutine is called whenever a different realization of scrambled points to be generated. The variable iflag passed from the main program determines the choice of scrambling or no scrambling to be utilized. The original generator matrices C1 , . . . , Cs , are stored in a two-dimensional array v. Each entry of v(j,k) is a 31-bit integer, whose digits form the k th column of Cj . Similarly, the randomly generated Owen-scrambling matrices L1 , . . . , Ls are stored in the array lsm, except that now lsm(j,i) is an integer whose bits form the ith row of Lj . The i, k element of the product Lj Cj can then be performed by multiplying the digits of lsm(j,i) and v(j,k) which are extracted by the Fortran command IBITS. The randomly generated Faure-Tezuka-scrambling matrix, L T , is stored in the array usm(i,k). GOSSOBL is identical to the routine GOSOBL except that the generator matrices v are replaced by the scrambled generator ma˜ j . GENSCRML trices sv. The element sv(j,k) represents the k th column of C and GENSCRMU are the routines that generates the scrambling matrices lsm and usm respectively. 3.2 Scrambled Niederreiter Generator: SNIED Similar to INSSOBL in the Sobol’ generator, CALCC2 generates the scrambled generator matrices. The subroutine GENSCRML and GENSCRMU are the same as in SSOBOL. 3.3 Scrambled Faure Generator: SFAURE INSFAUR is a modification of the routine INFAUR in [Bratley and Fox 1988] that initializes the values needed for the scrambled Faure generator. Similar to INSSOBOL, the subroutine is called whenever a different randomization of scrambled points is required. The generator matrices for the Faure sequence take the form Cj = Pj−1 , where P is the matrix of binomial coefficients (Pascal’s triangle). Thus, the original routine INFAUR initializes just the values P. However, the new routine INSFAUR initializes Cj for j = 1, . . . , s, and stores the results in the integer array coef(j,·,·). This routine calls either GENSCRML or GENSCRMU to generate the user choice of the scrambling matrices lsm(j,·,·) or usm(·,·), which are then multiplied by the generator matrices to obtain the scrambled generator matrices. GOSFAUR is a modification of the routine of GOFAUR that uses the gray code mentioned above. 3.4 Scrambled Niederreiter-Xing Generator: SNIEDX The structure of the program is same as the SSOBOL. Similar to INSSOBOL in the Sobol’ generator, INSNIEDX generates the scrambled generator matrices. The original generator matrices C1 , . . . , Cs , are read from data files. The subroutine GENSCRML and GENSCRMU are the same as in SSOBOL.


Algorithm Time Algorithm Time

Table 1. Time required to compute low discrepancy points SOBOL SSOBOL NIED SNIED NIEDX 1 2 2 2 2 Old FAURE 10

FAURE 3

·

7

SNIEDX 2

SFAURE 6

4. TIME AND SPACE COMPLEXITY This section considers the number of operations and amount of memory required to generate the first N points of a scrambled (t, s)-sequence with bM −1 < N ≤ bM . The space required to store the scrambled generator matrices is O(sKM ), where K ≥ M is the number of digits scrambled. The time required to generate and pre-multiply the matrices is O(sKM 2 ) = O(sK[log N ]2 ). To generate one new point from the scrambled sequence requires just O(sK) operations according to (9), so the total number of operations to to generate all points is O(sKN ). Thus, the preparations steps take proportionally negligible time as N → ∞. Note that for non-scrambled points the corresponding time and space complexities can be obtained by replacing K by M . Table 1 compares the approximate time required to generate the different kinds of non-scrambled and scrambled digital point sets using K ≈ M + 15, and M ≈ 10 to 20. The times have been normalized so that the time to generate the Sobol’ set equals 1. For non-scrambled Sobol’ points the matrices Cj are upper triangular, so one only needs to generate M binary digits for each coordinate of each point. Thus, this is the fastest algorithm. For non-scrambled Niederreiter and Niederreiter-Xing points the Cj are not upper triangular and one must generate 31 and 30 binary digits respectively for each coordinate of each point. Thus, these methods are about twice as slow as generating non-scrambled Sobol’ points. For scrambled Sobol’, Niederreiter and Niederreiter-Xing points the scrambled generator about 31 rows also. The new Faure generator takes about 1/3 the time of the original one due to the use of the gray code.

5. DISCREPANCY The discrepancy is a measure of how uniformly a set of points is distributed. It can also be interpreted as the maximum possible quadrature error over a unit ball of integrands [Hickernell 1998; Hickernell 1999a; Hickernell 1999b]. Depending on how one measures uniformity, or which space of integrands one is interested in, one gets different, but related, definitions of the discrepancy. The root mean square discrepancy of randomly scrambled (t, m, s)-nets and subsets of (t, s)-sequences has been studied theoretically [Hickernell 1996; Hickernell and Hong 1999; Hickernell and Yue 2001; Yue and Hickernell 2001]. In this section the root mean square discrepancies of the scrambled Sobol’, Faure, Niederreiter, and Niederreiter-Xing nets are computed numerically and compared with these theoretical results. For a set {x0 , . . . , xN −1 } one may define a discrepancy as [Hickernell and Hong

·

8


1999] D(P ) =  " #1/2 s α N −1 Y  α+1 X X 1 (−1) Br (xij )Br (xlj )  B2α ({xij − xlj }) 1 + , −1 + 2   N j (2α)! (r!)2 r=0 j=1 i,l=0

where the Br (x) are Bernoulli polynomials [Abramowitz and Stegun 1964, Chapter 23], and the notation {x} means the fractional part of a number or vector x. The positive integer α indicates the degree of smoothness of integrands in the underlying Hilbert space. The discrepancy plotted in Figures 1–3 has been scaled by a dimension-dependent constant so that the root mean square scaled discrepancy of a simple random sample is N −1 . The root mean square discrepancies of randomly scrambled (t, m, s)-nets in base b with N = bm points have been calculated using 100 different replications. The choices of dimension are s = 1 and 10. Since the Faure-Tezuka-scrambling does not affect the root mean square discrepancy of nets, only Owen’s scrambling is performed. The number of scrambled digits, K, is chosen to be 31. 0

10

−1

10

N−0.5

−2

10

−3

Discrepancy

10

−1

N −4

10

−5

10

−6

10

−7

10

−8

10

0

10

1

10

2

10 N

3

10

4

10

Fig. 1. The scaled root mean squared discrepancy with α = 1 as a function of N for Owenscrambled Sobol’ (·) and Niederreiter-Xing (∗) nets for s = 1 and 10.

Theory predicts that the root mean square discrepancy of nets should decay as O(N −1+ ) for α = 1, and as O(N −3/2+ ) for α = 2. Figures 1–3 display this behavior, although for larger s the asymptotic decay rate is only attained for larger enough N . The discrepancy of non-scrambled Sobol’, Faure, and Niederreiter nets does not attain the O(N −3/2+ ) decay rate for α = 2. However, the non-scrambled Niederreiter-Xing nets do appear to attain an O(N −3/2+ ) decay rate for α = 2,


·

9

0

10

−1

10

N−0.5

−2

10

−3

Discrepancy

10

−4

10

−5

10

N−1.5 −6

10

−7

10

N−2 −8

10

0

10

1

10

2

10 N

3

4

10

10

Fig. 2. The scaled root mean squared discrepancy with α = 2 as a function of N for Owenscrambled Sobol’ (·) and Niederreiter-Xing (∗) nets for s = 1 and 10.

1

10

0

10

−1

10

N−0.5

−2

Discrepancy

10

−3

10

N−1

−4

10

−5

10

N−1.5 −6

10

−7

10

N−2 −8

10

0

10

1

10

2

10 N

3

10

4

10

Fig. 3. The scaled discrepancy with α = 2 as a function of N for non-scrambled Sobol’ (·) and Niederreiter-Xing (∗) nets for s = 1 and 10.

for reasons not yet clear. Although for these experiments all possible digits were scrambled, our experience suggests that scrambling about K = M + 10 digits is enough to obtain the benefits of scrambling. Scrambling more than this number does not improve the discrepancy but increases the time required to generate the scrambled points.

·

10


6. QUADRATURE EXAMPLES Now the performance of scrambled nets is tested using two quadrature examples from the literature. The first example is the computation of multivariate normal probabilities, and the second is the evaluation of a multidimensional integral arising in physics problems. 6.1 Multivariate Normal Probabilities Consider the following multivariate normal probability: Z −1 1 T 1 p e− 2 θ Σ θ dθ. |Σ|(2π)s [a,c]

(10)

where a and c are known s-dimensional vectors that define the interval of integration, and Σ is a given s × s positive definite covariance matrix. One or more of the components of a and c may be infinite. Since the original form is not well-suited for numerical quadrature, Alan Genz [Genz 1992; Genz 1993] proposed a transformation of variables that result in an integral over the s − 1 dimensional unit cube. This transformation is used here. The choices of the parameters in (10) are made as in [Genz 1993; Hickernell and Hong 1997]: √ a1 = · · · = as = −∞, cj i.i.d. uniformly on [0, s], Σ generated randomly as in [Genz 1993]. Tol = 0.0001

1

10

0

Scaled Error

10

−1

10

−2

10

−3

10

3

6

9

12

15

s

Fig. 4. Box and whisker plots of scaled errors,E/, for for 50 randomly chosen test problems (10). For each dimension s results from left to the four algorithms as listed in the text.

Four different types of algorithms are compared for this problem: i.

the adaptive algorithm MVNDNT of Alan Genz, which can be found at http://www.sci.wsu.edu/math/faculty/genz/software/mvndstpack.f

·


11

Tol = 0.0001

5

10

Nubmer of Points

4

10

3

10

2

10

3

6

9

12

15

s

Fig. 5. Box and whisker plots of the computation times in seconds for 50 randomly chosen test problems (10). For each dimension s results from left to the four algorithms as listed in the text. Tol = 0.0001

1

10

0

Computation Time

10

−1

10

−2

10

−3

10

3

6

9

12

15

s

Fig. 6. Box and whisker plots of the number of points used for solving 50 randomly chosen test problems (10). For each dimension s results from left to the four algorithms as listed in the text.

ii. a randomly-shifted extensible rank-1 lattice sequence of Korobov type [Hickernell and Hong 1997; Hickernell et al. 2001] with generating vector (1, 17797, . . . , 17797 s−1 ), iii. the non-scrambled Niederreiter-Xing sequence described here, and iv. the Owen-scrambled Sobol’ sequence described here. The periodizing transformation x0j = |2xj − 1|, j = 1, . . . , s is applied to the

12

·


integrand for algorithms ii., as it appears to increase the accuracy of the algorithm. The computations were carried out in Fortran on a Unix work station in double precision. An absolute error tolerance of ε = 10−4 was chosen and compared to the absolute error E. Error estimation and stopping criteria for these algorithms was done as described in [Hickernell et al. 2001]. Because the true value of the integral is unknown for this test problem, the Korobov algorithm, implemented as part of the NAG library, with ε = 10−8 was used to compute the “exact” value. For each s, 50 test problems were generated randomly. Figures 4, 5 and 6 show the box and whisker plots of the scaled absolute error E/, the number of points used, and the computation time in seconds for the four methods with various dimensions. The boxes are divided by the median and contain the middle half of the values. The whiskers show the full range of values, and the outliers are plotted as ∗. The scaled error indicates how conservative the error estimate is. Ideally the scaled error should be close to but not more than one. If the scaled error is greater than one, then the error estimation is not careful enough. However, if the scaled error is much less than one, then the algorithm is wasting time by being too conservative. Figures 4, 5 and 6 show that all four methods do reasonably well in error estimation. The scrambled Sobol’ points, however, use fewer function evaluations and less computation time. Also, their performance is less dependent on the dimension. We also tested this problem with the scrambled Niederreiter-Xing points. For the lower dimension which is less than 9, the scrambled Niederreiter-Xing points performs as well as the scrambled Sobol’ points. However as dimension increases the performance deteriorates. 6.2 Multidimensional Integral from Physics The following multidimensional integral arising in physics problems is considered by Keister [Keister 1996]:  v uX Z Z s −1 (y )2 ] u 2 [Φ j  dy, (11) cos t cos(kxk)e−kxk dx = π s/2 2 [0,1]s Rs j=1 where k · k denotes the Euclidean norm in Rs , and Φ denotes the standard multivariate Gaussian distribution function. Keister provides an exact formula for the answer and compares the quadrature methods of McNamee and Stenger [McNamee and Stenger 1967] and Genz and Patterson for [Genz 1982; Patterson 1968] evaluating this integral. Papageorgiou and Traub [Papageorgiou and Traub 1997] apply the generalized Faure sequence in their FINDER library to this problem. The exact value of the integral is reported in [Keister 1996; Papageorgiou and Traub 1997]. Since the methods of McNamee and Stenger and Genz and Patterson performed much worse than quasi-Monte Carlo methods, they are ignored here. Instead we compare the Owen-scrambled Sobol’ sequence with FINDER’s generalized Faure algorithm and the extensible lattice rule described above. The other scrambled sequences performed a bit worse than the Sobol’ sequence. To be consistent with the results reported in [Papageorgiou and Traub 1997] the relative error is computed as a function of N . Figure 7 shows the results of the numerical experiments for dimension s = 25. Box and Whisker plots show how well 50 randomized nets and lattice rules perform for each N . The two algorithms appear to be quite competitive


·

13

1

10

0

10

−1

10

Relative Error

−2

10

−3

10

−4

10

−5

10

−6

10

0

10

l 1 10

l 2 10

l 3 10 N

l 4 10

l 5 10

6

10

Fig. 7. The relative error obtained in approximating (11) for s = 25 for FINDER’s generalized Faure algorithm (◦). The box and whisker plots show the performance of the randomly shifted extensible rank-1 lattice and randomly scrambled Sobol’ points. The values of N are 4, 8, . . . , 2 18 .

to each other. In some cases randomized Sobol’ sequence performs better than the lattice rule.

7. CONCLUSION The random scrambling of Owen and of Faure and Tezuka have been implemented for the best known digital (t, s)-sequences. Both types of scrambling involve multiplying the original generator matrices by randomly chosen matrices and adding random digital shifts. Base 2 sequences can be generated faster by taking the advantage of the computer’s binary arithmetic system. Using a gray code also speeds up the generation process. The cost of producing a scrambled sequence is about twice the cost of producing a non-scrambled one. Scrambled sequences often have smaller discrepancies than their non-scrambled counterparts. Moreover, random scramblings facilitate error estimation. On test problems taken from the literature scrambled digital sequences performed as well as or better than other quadrature methods.

ACKNOWLEDGMENTS

Several people have contributed to this article. The authors would like to thank Paul Bratley, Ben Fox, and Harald Niederreiter for allowing us to modify they digital sequence generators. We would also like to thank Alan Genz for the use of MVNDNT and Anargyros Papageorgiou and Joe Traub for the use of FINDER. Discussions with Harald Niederreiter, Gerhard Larcher and Wolfgang Schmid helped us to better understand digital sequences.

14

·


REFERENCES Abramowitz, M. and Stegun, I. A. Eds. 1964. Handbook of Mathematical Functions with Formulas, Graphs and Mathematical Tables. U. S. Government Printing Office, Washington, DC. Bratley, P. and Fox, B. L. 1988. Algorithm 659: Implementing Sobol’s quasirandom sequence generator. ACM Trans. Math. Software 14, 88–100. Bratley, P., Fox, B. L., and Niederreiter, H. 1992. Implementation and tests of lowdiscrepancy sequences. ACM Trans. Model. Comput. Simul. 2, 195–213. Faure, H. 1982. Discr´ epance de suites associ´ ees a ` un syst` eme de num´ eration (en dimension s). Acta Arith. 41, 337–351. Faure, H. and Tezuka, S. 2001. A new generation of digital (0, s)-sequences. Fox, B. L. 1999. Strategies for Quasi-Monte Carlo. Kluwer Academic Publishers, Boston. Genz, A. 1982. A Lagrange extrpolation algorithm for sequences of approximations to multiple integrals. SIAM J. Sci. Stat. Comput. 3, 160–172. Genz, A. 1992. Numerical computation of multivariate normal probabilities. J. Comput. Graph. Statist. 1, 141–150. Genz, A. 1993. Comparison of methods for the computation of multivariate normal probabilities. Computing Science and Statistics 25, 400–405. Hellekalek, P. and Larcher, G. Eds. 1998. Random and Quasi-Random Point Sets, Volume 138 of Lecture Notes in Statistics. Springer-Verlag, New York. Hickernell, F. J. 1996. The mean square discrepancy of randomized nets. ACM Trans. Model. Comput. Simul. 6, 274–296. Hickernell, F. J. 1998. A generalized discrepancy and quadrature error bound. Math. Comp. 67, 299–322. Hickernell, F. J. 1999a. Goodness-of-fit statistics, discrepancies and robust designs. Statist. Probab. Lett. 44, 73–78. Hickernell, F. J. 1999b. What affects the accuracy of quasi-Monte Carlo quadrature? In H. Niederreiter and J. Spanier Eds., Monte Carlo and Quasi-Monte Carlo Methods 1998 (1999), pp. 16–55. Springer-Verlag, Berlin. Hickernell, F. J. and Hong, H. S. 1997. Computing multivariate normal probabilities using rank-1 lattice sequences. In G. H. Golub, S. H. Lui, F. T. Luk, and R. J. Plemmons Eds., Proceedings of the Workshop on Scientific Computing (Hong Kong, 1997), pp. 209– 215. Springer-Verlag, Singapore. Hickernell, F. J. and Hong, H. S. 1999. The asymptotic efficiency of randomized nets for quadrature. Math. Comp. 68, 767–791. ´ Hickernell, F. J., Hong, H. S., L’Ecuyer, P., and Lemieux, C. 2001. Extensible lattice sequences for quasi-Monte Carlo quadrature. SIAM J. Sci. Comput. 22, 1117–1138. Hickernell, F. J. and Yue, R. X. 2001. The mean square discrepancy of scrambled (t, s)sequences. SIAM J. Numer. Anal. 38, 1089–1112. Keister, B. D. 1996. Multidimensional quadrature algorithms. Computers in Physics 10, 119–122. Larcher, G. 1998a. Digital point sets: Analysis and applications. In P. Hellekalek and G. Larcher Eds., Random and Quasi-Random Point Sets, Volume 138 of Lecture Notes in Statistics, pp. 167–222. New York: Springer-Verlag. Larcher, G. 1998b. On the distribution of digital sequences. In H. Niederreiter, P. Hellekalek, G. Larcher, and P. Zinterhof Eds., Monte Carlo and quasi-Monte Carlo methods 1996 , Volume 127 of Lecture Notes in Statistics (1998), pp. 109–123. Springer-Verlag, New York. Lichtner, J. 1998. Iterating an α-ary gray code. SIAM J. Discrete Math. 11, 3, 381–386. McNamee, J. and Stenger, F. 1967. Construction of fully symmetric numerical integration formulas. Numer. Math. 10, 327–344. Niederreiter, H. 1988. Low discrepancy and low dispersion sequences. J. Number Theory 30, 51–70.


·

15

Niederreiter, H. 1992. Random Number Generation and Quasi-Monte Carlo Methods. CBMS-NSF Regional Conference Series in Applied Mathematics. SIAM, Philadelphia. Niederreiter, H. and Spanier, J. Eds. 1999. Monte Carlo and Quasi-Monte Carlo Methods 1998 (1999). Springer-Verlag, Berlin. Niederreiter, H. and Xing, C. 1996. Quasirandom points and global function fields. In S. Cohen and H. Niederreite Eds., Finite Fields and Applications, Number 233 in London Math. Society Lecture Note Series (1996), pp. 269–296. Cambridge University Press. Niederreiter, H. and Xing, C. 1998. Nets, (t, s)-sequences and algebraic geometry. In P. Hellekalek and G. Larcher Eds., Random and Quasi-Random Point Sets, Volume 138 of Lecture Notes in Statistics, pp. 267–302. New York: Springer-Verlag. Owen, A. B. 1995. Randomly permuted (t, m, s)-nets and (t, s)-sequences. In H. Niederreiter and P. J.-S. Shiue Eds., Monte Carlo and Quasi-Monte Carlo Methods in Scientific Computing, Volume 106 of Lecture Notes in Statistics (1995), pp. 299–317. Springer-Verlag, New York. Owen, A. B. 1997a. Monte Carlo variance of scrambled equidistribution quadrature. SIAM J. Numer. Anal. 34, 1884–1910. Owen, A. B. 1997b. Scrambled net variance for integrals of smooth functions. Ann. Stat. 25, 1541–1562. Owen, A. B. 1998. Scrambling Sobol’ and Niederreiter-Xing points. J. Complexity 14, 466– 489. Owen, A. B. 1999. Monte Carlo, quasi-Monte carlo, and randomized quasi-Monte Carlo. In H. Niederreiter and J. Spanier Eds., Monte Carlo and Quasi-Monte Carlo Methods 1998 (1999). Springer-Verlag, Berlin. Papageorgiou, A. and Traub, J. F. 1997. Faster evaluation of multidimensional integrals. Computers in Physics 11, 574–578. Patterson, T. N. L. 1968. The optimum addition of points to quadrature formulae. Math. Comp. 22, 847–856. Pirsic, G. 2001. A software implementation of the Niederreiter-Xing sequences. Fourth International Conference on Monte Carlo and Quasi-Monte Carlo Methods in Scientific Computing, November 27 – December 1, 2000, Hong Kong. submitted to proceedings. Sobol’, I. M. 1967. The distribution of points in a cube and the approximate evaluation of integrals. U.S.S.R. Comput. Math. and Math. Phys. 7, 86–112. Tezuka, S. 1995. Uniform Random Numbers: Theory and Practice. Kluwer Academic Publishers, Boston. Yue, R. X. 1999. Variance of quadrature over scrambled unions of nets. Statist. Sinica 9, 451–473. Yue, R. X. and Hickernell, F. J. 2001. The discrepancy of digital nets. submitted to J. Complexity. Yue, R. X. and Mao, S. S. 1999. On the variance of quadrature over scrambled nets and sequences. Statist. Probab. Lett. 44, 267–280.