Block Toeplitz Matrices - COMONSENS

R Foundations and Trends in Communications and Information Theory Vol. 8, No. 3 (2011) 179–257 c 2012 J. Guti´ errez-Gutiérrez and P. M. Crespo DOI: 10.1561/0100000066

Block Toeplitz Matrices: Asymptotic Results and Applications By Jes´ us Gutiérrez-Gutiérrez and Pedro M. Crespo

Contents 1 Introduction

180

2 Mathematical Preliminaries

184

2.1 2.2

184 187

Matrix Norms Functions of Matrices

3 Asymptotically Equivalent Sequences of Matrices 3.1 3.2

Definition and Basic Properties Asymptotically Equivalent Sequences of Hermitian Matrices

191 191 197

4 Block Toeplitz Matrices

201

4.1 4.2

201

4.3

Definition and Examples Sequence of Block Toeplitz Matrices Generated by a Continuous Function Examples of Continuous Generating Functions

204 218

5 Block Circulant Matrices

221

5.1 5.2

221

Definition Sequence of Block Circulant Matrices Generated by a Continuous Function

6 Asymptotic Behaviour of Block Toeplitz Matrices 6.1 6.2

Products and Functions of Large Block Toeplitz Matrices The Szegö Theorem for Block Toeplitz Matrices

225

232 232 238

7 Applications to Vector Random Processes

242

7.1 7.2

242

Vector Asymptotically WSS Processes Differential Entropy Rate of Certain Vector AWSS Processes

247

A Asymptotically 2-Equivalent Sequences of Matrices

249

B The Wiener Class

251

Acknowledgments

255

References

256

R Foundations and Trends in Communications and Information Theory Vol. 8, No. 3 (2011) 179–257 c 2012 J. Guti´ errez-Gutiérrez and P. M. Crespo DOI: 10.1561/0100000066

Block Toeplitz Matrices: Asymptotic Results and Applications Jes´ us Guti´ errez-Guti´ errez1 and Pedro M. Crespo2 1 2

CEIT and Tecnun (University of Navarra), Manuel Lardiz´ abal 15, San Sebasti´ an, Spain, [email protected] CEIT and Tecnun (University of Navarra), Manuel Lardiz´ abal 15, San Sebasti´ an, Spain, [email protected]

Abstract The present monograph studies the asymptotic behaviour of eigenvalues, products and functions of block Toeplitz matrices generated by the Fourier coefficients of a continuous matrix-valued function. This study is based on the concept of asymptotically equivalent sequences of non-square matrices. The asymptotic results on block Toeplitz matrices obtained are applied to vector asymptotically wide sense stationary processes. Therefore, this monograph is a generalization to block Toeplitz matrices of the Gray monograph entitled “Toeplitz and circulant matrices: A review”, which was published in the second volume of Foundations and Trends in Communications and Information Theory, and which is the simplest and most famous introduction to the asymptotic theory on Toeplitz matrices.

1 Introduction

The Gray monograph entitled “Toeplitz and circulant matrices: A review” [8], which was published in the second volume of Foundations and Trends in Communications and Information Theory, is the simplest and most famous introduction to the asymptotic theory on Toeplitz matrices. The secret of the success of that monograph lies in the simplicity of the mathematical tools that Gray used to prove important results on Toeplitz matrices. Specifically, he proved asymptotic results on eigenvalues, products and inverses of Toeplitz matrices by using mainly the concept of asymptotically equivalent sequences of matrices, which he introduced in [7]. The present monograph is a generalization of the Gray monograph to block Toeplitz matrices, which is a type of matrices frequently used in Communications, Information Theory and Signal Processing, because, for instance, matrix representations of discrete-time causal finite impulse response (FIR) multiple-input multiple-output (MIMO) filters and correlation matrices of vector wide sense stationary (WSS) processes are block Toeplitz. Therefore, the present monograph deals with the asymptotic behaviour of eigenvalues, products and inverses of 180

181 block Toeplitz matrices, and, as in [8], the concept of asymptotically equivalent sequences of matrices is the key concept in it. However, since the blocks of block Toeplitz matrices are not, in general, square, we extend here the Gray definition of asymptotically equivalent sequences of matrices to sequences of non-square matrices. Unlike in [8], where powers and inverses of Toeplitz matrices were the only functions of Toeplitz matrices studied, in the present monograph we cover any function of block Toeplitz matrices. Furthermore, while the Toeplitz matrices that Gray considered in [8] were generated by the Fourier coefficients of a function in the Wiener class (which is a special subset of the set of continuous functions) we consider here block Toeplitz matrices generated by the Fourier coefficients of any continuous matrix-valued function. Observe that even for the case of Toeplitz matrices (that is, block Toeplitz matrices with 1 × 1 blocks) the type of matrices considered here is more general, since they are Toeplitz matrices generated by the Fourier coefficients of continuous functions without imposing the Wiener condition. Since the continuous functions are Riemann integrable, the integral that we use in the monograph is the Riemann integral. Thus, as in [8], it is not required to use the Lebesgue integral, an integral that is not included in a typical engineering background. We recall here that continuous functions are the most representative type of Riemann integrable functions, since the functions that are Riemann integrable on a closed interval are those which are continuous except on a set of Lebesgue measure zero and which are bounded. Although there are some new results in the present monograph, most of them were given by the authors in [12] and [13], however, here they are presented in a tutorial manner and proved in detail. The rest of the monograph is organized as follows. In Section 2 the mathematical preliminaries are given. We review the two matrix norms that are used in Section 3 to define the concept of asymptotically equivalent sequences of matrices: the spectral norm and the Frobenius norm. Furthermore, we review the concept of function of a matrix and we give some examples of functions of matrices, such as, the powers, the exponential or the inverse of a matrix.

182

Introduction

Section 3 is devoted to the concept of asymptotically equivalent sequences of non-square matrices. We give its definition, its basic properties and we study its relation to the concept of function of a matrix. In Section 4 we review the definition of a block Toeplitz matrix and show that matrix representations of discrete-time causal FIR MIMO filters and correlation matrices of vector WSS processes are block Toeplitz. However, most of the section is devoted to the presentation of several non-asymptotic properties of a sequence of block Toeplitz matrices generated by the Fourier coefficients of a continuous matrixvalued function. In Section 5 we study a special type of block Toeplitz matrices: block circulant matrices. Moreover, we define the sequence of block circulant matrices generated by a continuous matrix-valued function, and we prove several non-asymptotic properties of this sequence. Section 6 is devoted to the study of the asymptotic behaviour of block Toeplitz matrices. We begin by proving that the sequence of block Toeplitz matrices and the sequence of block circulant matrices generated by the same continuous matrix-valued function are asymptotically equivalent. Based on this result and the results obtained in the previous sections, we analyse the asymptotic behaviour of eigenvalues, products and functions of block Toeplitz matrices generated by the Fourier coefficients of a continuous matrix-valued function. In particular, we prove the most famous asymptotic result on block Toeplitz matrices: the Szegö theorem for block Toeplitz matrices.1 This theorem deals with the arithmetic mean of the eigenvalues of functions of large Hermitian block Toeplitz matrices, and it has found different applications in Information Theory and Signal Processing (see, e.g., [6, 12, 13, 15, 19, 23]). As a matter of fact, in Section 6 we prove that the Szeg¨ o theorem for block Toeplitz matrices is also true if the sequence of Hermitian block Toeplitz matrices is replaced by any other asymptotically equivalent sequence of Hermitian matrices. Finally, the theory on block Toeplitz matrices developed in this monograph is applied in Section 7 to study some vector non-stationary 1 The

Szeg¨ o theorem for block Toeplitz matrices is the generalization to block Toeplitz matrices of the famous result on Toeplitz matrices given by Szeg¨ o in [9, p. 64].

183 processes. Specifically, we define there the concept of vector asymptotically WSS (AWSS) process, which is based on the concept of asymptotically equivalent sequences of matrices, and we give several interesting properties and examples of this kind of process. Moreover, we compute the differential entropy rate of certain vector AWSS processes.

2 Mathematical Preliminaries

2.1

Matrix Norms

In this monograph N, Z, R, and C denote the set of natural numbers (i.e., the set of positive integers), the set of integer numbers, the set of (finite) real numbers, and the set of (finite) complex numbers, respectively. We begin this section by reviewing the concept of matrix norm on Cm×n (see, e.g., [2]), where Cm×n , with m, n ∈ N, is the set of all m × n complex matrices. Definition 2.1. A matrix norm on Cm×n is a function · : Cm×n → [0, ∞) that satisfies the three following properties: (1) A = 0 if and only if A = 0m×n , where 0m×n is the m × n zero matrix. (2) αA = |α|A for all α ∈ C and A ∈ Cm×n . (3) A + B ≤ A + B for all A, B ∈ Cm×n . A matrix norm · on Cm×n is unitarily invariant if U AV = A for all A ∈ Cm×n and for all unitary1 matrices U ∈ Cm×m and V ∈ Cn×n . n × n invertible matrix U is unitary if U −1 = U ∗ , where U ∗ = U with denoting transpose (that is, ∗ denotes conjugate transpose).

1 An

184

2.1 Matrix Norms

185

Property (3) of Definition 2.1 is called triangle inequality. We now review the definition of two well-known unitarily invariant matrix norms that we will need for the definition of asymptotically equivalent sequences of matrices in Section 3: the spectral norm and the Frobenius norm. Moreover, in Propositions 2.1, 2.2 and 2.3 we review the properties of these two matrix norms that will be used in the monograph. All those properties can be found, e.g., in [2]. 2.1.1

The Spectral Norm

Definition 2.2. The spectral norm of A ∈ Cm×n is defined as 1 ∗ ∗ x A Ax 2 . A2 := max x∗ x x ∈ Cn×1 x = 0n×1

Thus, for instance, In 2 =

max n×1

x∈C x = 0n×1

x∗ x x∗ x

1 2

=

max

x ∈ Cn×1 x = 0n×1

1 = 1,

∀n ∈ N,

where In denotes the n × n identity matrix. The following proposition provides some useful properties of the spectral norm. In particular, in that proposition the spectral norm is written in terms of singular values and eigenvalues. Here and subsequently, σj (A), 1 ≤ j ≤ min{m, n}, are the singular values of A ∈ Cm×n numbered in a non-increasing order, and λk (B), 1 ≤ k ≤ n, denote the eigenvalues of a diagonalizable matrix B ∈ Cn×n . Proposition 2.1. The spectral norm is a unitarily invariant matrix norm, and it satisfies 1 2 ∗ = σ1 (A) (2.1) A2 = A 2 = A = max λk (A A) 2

1≤k≤n

and AB2 ≤ A2 B2

(2.2)

186

Mathematical Preliminaries

for all A ∈ Cm×n and B ∈ Cn×p . If A is Hermitian then A2 = max |λk (A)|. 1≤k≤n

Observe that if we consider an n × n diagonal matrix  α1 0   0 α2 A = diag(α1 , . . . , αn ) = diag1≤k≤n (αk ) :=   .. . .  . . 0 ···

··· .. . .. . 0

 0 ..  .  ,  0 αn

then from (2.1) we obtain 1

2 2 2 A2 = max λk diag |α1 | , . . . , |αn | = 2.1.2

1≤k≤n

1

max |αk |

2

1≤k≤n

2

= max |αk |. 1≤k≤n

The Frobenius Norm

Definition 2.3. The Frobenius norm of an m × n matrix A = (aj,k )1≤j≤m,1≤k≤n is defined as  1 2 n m 2  |aj,k | . AF := j=1 k=1

We now give some useful properties of the Frobenius norm. Proposition 2.2. The Frobenius norm is a unitarily invariant matrix norm, and it satisfies AF = AF = A F for all A ∈ Cm×n . If A is square then √ |tr(A)| ≤ nAF , where tr denotes trace.

2.2 Functions of Matrices

187

The following proposition provides two inequalities that relate the Frobenius norm with the spectral norm. Proposition 2.3. If A ∈ Cm×n and B ∈ Cn×p , then A2 ≤ AF

(2.3)

ABF ≤ min{A2 BF , AF B2 }.

(2.4)

and

From (2.3) and (2.4) we deduce that (2.2) is also true if the spectral norm is replaced by the Frobenius norm. However, we will not need this property in the monograph.

2.2

Functions of Matrices

We now review the two concepts of functions of matrices that we will use in the monograph. We begin with the concept of polynomial function of a square matrix. Definition 2.4. Let p(x) = αq xq + · · · + α1 x + α0 be a polynomial with coefficients in C. If A ∈ Cn×n then p(A) is defined as the n × n matrix given by p(A) := αq Aq + · · · + α1 A + α0 In . Therefore, powers of matrices are examples of polynomial functions of matrices. Secondly, we review the concept of function of a diagonalizable matrix. Definition 2.5. If A ∈ Cn×n is diagonalizable, and g is a complex function on the set of eigenvalues of A, g(A) is defined as the n × n matrix given by g(A) := U diag(g(λ1 (A)), . . . , g(λn (A)))U −1

(2.5)

188


where A = U diag(λ1 (A), . . . , λn (A))U −1 is any eigenvalue decomposition of A. It can be proved (see, e.g., [16]) that Definition 2.5 is well defined (that is, (2.5) is independent of the chosen eigenvalue decomposition of the matrix A) and agrees with Definition 2.4. An interesting example of function of a matrix is the exponential of a matrix that plays an important role in solving systems of first-order linear differential equations (see, e.g., [4, 17]). Observe that if A ∈ Cn×n is an invertible diagonalizable matrix then A−1 = g(A) with g(x) = x1 , because 1 U −1 Ag(A) = U diag1≤k≤n (λk (A))U −1 U diag1≤k≤n λk (A) 1 U −1 = U diag1≤k≤n (λk (A))In diag1≤k≤n λk (A) 1 U −1 = U diag1≤k≤n (λk (A))diag1≤k≤n λk (A) = U In U −1 = U U −1 = In . Thus, the inverse of a matrix is another interesting example of function of a matrix. We now present a result that gives two simple properties of functions of a diagonalizable matrix. Lemma 2.4. If g and h are two complex functions on the set of eigenvalues of an n × n diagonalizable matrix A, then

(1) tr(g(A)) = nk=1 g(λk (A)). (2) (αg + βh)(A) = αg(A) + βh(A) for all α, β ∈ C.

2.2 Functions of Matrices

189

Proof. Let A = U diag(λ1 (A), . . . , λn (A))U −1 be an eigenvalue decomposition of A. Then tr(g(A)) = tr(U diag1≤k≤n (g(λk (A)))U −1 ) = tr(U −1 U diag1≤k≤n (g(λk (A)))) = tr(diag1≤k≤n (g(λk (A)))) =

n

g(λk (A)),

k=1

and (αg + βh)(A) = U diag1≤k≤n ((αg + βh)(λk (A)))U −1 = U diag1≤k≤n (αg(λk (A)) + βh(λk (A)))U −1 = U (diag1≤k≤n (αg(λk (A))) + diag1≤k≤n (βh(λk (A))))U −1 = U (αdiag1≤k≤n (g(λk (A))) + βdiag1≤k≤n (h(λk (A))))U −1 = αU diag1≤k≤n (g(λk (A)))U −1 + βU diag1≤k≤n (h(λk (A)))U −1 = αg(A) + βh(A). We finish Section 2 with a result on the spectral norm and the Frobenius norm of a function of a Hermitian matrix. Lemma 2.5. If g is a complex function on the set of eigenvalues of an n × n Hermitian matrix A, then g(A)2 = max |g(λk (A))|

(2.6)

1≤k≤n

and g(A)F =

n

1 2

|g(λk (A))|

2

.

(2.7)

k=1

Proof. As A is Hermitian there exists an eigenvalue decomposition of A A = U diag(λ1 (A), . . . , λn (A))U −1 ,

190


where the eigenvector matrix U is unitary. Since g(A) = U diag(g(λ1 (A)), . . . , g(λn (A)))U ∗ , U and U ∗ are unitary matrices, and the spectral norm and the Frobenius norm are unitarily invariant, we have g(A)2 = diag(g(λ1 (A)), . . . , g(λn (A)))2 and g(A)F = diag(g(λ1 (A)), . . . , g(λn (A)))F .

3 Asymptotically Equivalent Sequences of Matrices

3.1

Definition and Basic Properties

Based on the two matrix norms reviewed in Section 2 we can now proceed to give the key concept of the monograph: the concept of asymptotically equivalent sequences of matrices. This concept was introduced by Gray for sequences of square matrices in [7] and we extended it to sequences of non-square matrices in [13]. The definition of asymptotically equivalent sequences of matrices given in [13] is the one that we review here. Definition 3.1. Consider two strictly increasing sequences of natural (1) (2) (1) (2) numbers {dn } and {dn }. Let An and Bn be dn × dn matrices for all n ∈ N. We say that the sequences {An } and {Bn } are asymptotically equivalent, and write {An } ∼ {Bn }, if ∃M ∈ [0, ∞) :

An 2 , Bn 2 ≤ M,

and lim

n→∞

An − Bn F √ = 0. n 191

∀n ∈ N

(3.1)

192

Asymptotically Equivalent Sequences of Matrices

The original definition of asymptotically equivalent sequences of matrices given by Gray in [7] can be obtained from Definition 3.1 by taking (1) (2) {dn } = {dn } = {n}. In Appendix A we introduce a definition different from Definition 3.1 that also extends the Gray concept of asymptotically equivalent sequences of matrices to sequences of non-square matrices. The next lemma, which was given in [13] without a proof, provides some basic properties of asymptotically equivalent sequences of matrices. Lemma 3.1. Consider two strictly increasing sequences of natural (1) (2) (1) (2) numbers {dn } and {dn }. Let An , Bn , Cn and Dn be dn × dn matrices for all n ∈ N. If {An } ∼ {Bn } then {Bn } ∼ {An }. If {An } ∼ {Bn } and {Bn } ∼ {Cn }, then {An } ∼ {Cn }. If {An } ∼ {Bn } then {αAn } ∼ {αBn } for all α ∈ C. If {An } ∼ {Bn } and {Cn } ∼ {Dn }, then {An + Cn } ∼ {Bn + Dn }. (5) If {An } ∼ {Bn } then {An } ∼ {Bn }. (6) If {An } ∼ {Bn } then {A n } ∼ {Bn }. (7) If {An } ∼ {Bn } then {A∗n } ∼ {Bn∗ }.

(1) (2) (3) (4)

Proof. (1) This property is true because Bn − An F | − 1|An − Bn F An − Bn F √ √ √ = lim = lim = 0. n→∞ n→∞ n→∞ n n n lim

(2) This property holds since 0≤

An − Bn + Bn − Cn F An − Cn F √ √ = n n ≤

An − Bn F + Bn − Cn F √ n

=

Bn − Cn F An − Bn F √ √ + n n

→ 0.

3.1 Definition and Basic Properties

193

(3) If An 2 , Bn 2 ≤ M < ∞ for all n ∈ N, then αAn 2 = |α|An 2 ≤ |α|M < ∞ and αBn 2 = |α|Bn 2 ≤ |α|M for all n ∈ N. On the other hand, αAn − αBn F |α|An − Bn F √ √ = lim n→∞ n→∞ n n lim

= |α| lim

n→∞

An − Bn F √ = 0. n

(4) If An 2 , Bn 2 ≤ M1 < ∞ and Cn 2 , Dn 2 ≤ M2 < ∞ for all n ∈ N, then An + Cn 2 ≤ An 2 + Cn 2 ≤ M1 + M2 < ∞ and Bn + Dn 2 ≤ Bn 2 + Dn 2 ≤ M1 + M2 for all n ∈ N. On the other hand, 0≤

An + Cn − (Bn + Dn )F √ n

≤

An − Bn F + Cn − Dn F √ n

=

Cn − Dn F An − Bn F √ √ + → 0. n n

(5) This property is true because An = An 2 ≤ M 2 and Bn = Bn 2 ≤ M 2 for all n ∈ N, and An − Bn An − Bn An − Bn F F F √ √ √ = lim = lim = 0. lim n→∞ n→∞ n→∞ n n n

194


(6) This property holds since An = An 2 ≤ M 2 and Bn = Bn 2 ≤ M 2 for all n ∈ N, and An − Bn (An − Bn ) An − Bn F F F √ √ √ = lim = lim = 0. lim n→∞ n→∞ n→∞ n n n (7) This basic property of asymptotically equivalent sequences of matrices is a direct consequence of the two previous properties. The following lemma was given in [13] and it provides a fundamental property of asymptotically equivalent sequences of matrices. Lemma 3.2 and property (2) of Lemma 3.1 were first proved by Gray in [8] for the (1) (2) (3) case in which {dn } = {dn } = {dn } = {n}. Lemma 3.2. Consider three strictly increasing sequences of natural (1) (2) (3) (1) (2) numbers {dn }, {dn } and {dn }. Let An and Bn be dn × dn matri(2) (3) ces for all n ∈ N. Suppose that Cn and Dn are dn × dn matrices for all n ∈ N. If {An } ∼ {Bn } and {Cn } ∼ {Dn }, then {An Cn } ∼ {Bn Dn }. Proof. If An 2 , Bn 2 ≤ M1 < ∞ and Cn 2 , Dn 2 ≤ M2 < ∞ for all n ∈ N, then An Cn 2 ≤ An 2 Cn 2 ≤ M1 M2 < ∞ and Bn Dn 2 ≤ Bn 2 Dn 2 ≤ M1 M2 for all n ∈ N. On the other hand, 0≤ =

An Cn − Bn Dn F √ n An Cn − An Dn + An Dn − Bn Dn F √ n

3.1 Definition and Basic Properties

≤

An Cn − An Dn F + An Dn − Bn Dn F √ n

≤

An 2 Cn − Dn F + An − Bn F Dn 2 √ n

≤

M1 Cn − Dn F + M2 An − Bn F √ n

= M1

195

Cn − Dn F An − Bn F √ √ + M2 → 0. n n

From the proof of Lemma 3.1, we know that Lemma 3.1 is also true if we do not require (3.1) in Definition 3.1. However, as we will now prove, this does not happen with Lemma 3.2, and that is the reason why (3.1) has been included in Definition 3.1. Observe that if {An } = {Cn } = {diag(n + 1, 0, . . . , 0)}, and {Bn } = {Dn } = {diag(n, 0, . . . , 0)}, with (2) (3) {d(1) n } = {dn } = {dn } = {n},

then the sequences {An 2 } = {Cn 2 } = {n + 1} and {Bn 2 } = {Dn 2 } = {n} are not bounded, lim

n→∞

An − Bn F Cn − Dn F 1 √ √ = lim = lim √ = 0, n→∞ n→∞ n n n

and An Cn − Bn Dn F diag((n + 1)2 − n2 , 0, . . . , 0)F √ √ = lim n→∞ n→∞ n n lim

(n + 1)2 − n2 2n + 1 √ = lim √ = ∞. n→∞ n→∞ n n

= lim

196


We finish this subsection with a lemma that gives three additional properties of asymptotically equivalent sequences of matrices. Lemma 3.3. Consider two strictly increasing sequences of natural (1) (2) (1) (2) numbers {dn } and {dn }. Let An be a dn × dn matrix for all n ∈ N. is bounded. (1) {An } ∼ {An } if and only if {An 2 } (2) If {An F } is bounded then {An } ∼ 0d(1) ×d(2) . n n (1) (2) (1) (1) (3) Suppose that dn = {dn } and let Bn be a dn × dn matrix for all n ∈ N. If {An } ∼ {Bn } then {p(An )} ∼ {p(Bn )} for any polynomial p(x) with coefficients in C.

Proof. (1) Based on Definition 3.1, if {An } ∼ {An } then {An 2 } is bounded. The reciprocal is also true because 0 (1) (2) An − An F 0 dn ×dn F √ √ = lim = lim √ = 0. lim n→∞ n→∞ n→∞ n n n (2) If An F ≤ M < ∞ for all n ∈ N, then An 2 ≤ An F ≤ M and

0d(1) ×d(2) = 0 ≤ M n

n

2

for all n ∈ N, and An − 0d(1) ×d(2) An F M n n F √ = √ ≤ √ → 0. 0≤ n n n (3) We will proceed by induction on the degree q of the polynomial p(x). We begin with the case q = 0, that is, p(x) = α0 with α0 ∈ C. Since I | I = |α = {|α0 | · 1} = {|α0 |} α 0 d(1) 0 d(1) n

2

n

2

3.2 Asymptotically Equivalent Sequences of Hermitian Matrices

197

is bounded, from property (1) of the present lemma we obtain {p(An )} = α0 Id(1) ∼ α0 Id(1) = {p(Bn )}. n

n

Now suppose that the result is true for q0 ≥ q ≥ 0, and consider the case q = q0 + 1. In this case, p(x) = αq0 +1 xq0 +1 + pq0 (x) where αq0 +1 ∈ C and pq0 (x) is a polynomial of degree at most q0 with coefficients in C. By the induction hypothesis we have {pq0 (An )} ∼ {pq0 (Bn )}

(3.2)

{Aqn0 } ∼ {Bnq0 }.

(3.3)

and

From (3.3), Lemma 3.2, and the fact that {An } ∼ {Bn }, we obtain {Aqn0 +1 } = {Aqn0 An } ∼ {Bnq0 Bn } = {Bnq0 +1 }, and consequently, applying property (3) of Lemma 3.1 yields {αq0 +1 Aqn0 +1 } ∼ {αq0 +1 Bnq0 +1 }.

(3.4)

Finally, from (3.2), (3.4), and property (4) of Lemma 3.1, we conclude that {p(An )} = {αq0 +1 Aqn0 +1 + pq0 (An )} ∼ {αq0 +1 Bnq0 +1 + pq0 (Bn )} = {p(Bn )}.

3.2

Asymptotically Equivalent Sequences of Hermitian Matrices

In this subsection, we give three results on asymptotically equivalent sequences of Hermitian matrices. Lemma 3.4. Consider a strictly increasing sequence of natural numbers {dn }. Let An and Bn be dn × dn Hermitian matrices for all n ∈ N, and suppose that {An } ∼ {Bn }. Then there exists a closed interval [a, b] ⊂ R containing the eigenvalues of all the matrices of these two matrix sequences.

198


Proof. Since {An } and {Bn } are sequences of Hermitian matrices, from (3.1) there exists M ∈ [0, ∞) such that max |λk (An )|, max |λk (Bn )| ≤ M,

1≤k≤dn

∀n ∈ N.

1≤k≤dn

(3.5)

From (3.5) and the fact that all the eigenvalues of a Hermitian matrix are real, we conclude that the closed interval [−M, M ] contains the eigenvalues of all the matrices of the sequences {An } and {Bn }. Lemma 3.4 was given in [8] for the case in which {dn } = {n}. The next result deals with asymptotically equivalent sequences of Hermitian matrices and functions of matrices. Theorem 3.5. Let {dn }, {An }, {Bn } and [a, b] be as in Lemma 3.4. If { dnn } is bounded then {g(An )} ∼ {g(Bn )},

∀g ∈ C[a, b],

where C[a, b] denotes the set of all continuous complex functions on [a, b]. Proof. Fix g ∈ C[a, b]. Applying (2.6) yields g(An )2 , g(Bn )2 ≤ max |g(x)|, a≤x≤b

∀n ∈ N,

where the existence of maxa≤x≤b |g(x)| is guaranteed by the continuity of the real function |g(x)| on the closed interval [a, b]. Let ∈ (0, ∞). Since g is a continuous complex function on [a, b], from the Stone–Weierstrass theorem (see, e.g., [20, p. 159]), there exists a polynomial p(x) with coefficients in C such that |g(x) − p(x)| < ,

∀x ∈ [a, b].

(3.6)

Applying property (3) of Lemma 3.3 gives {p(An )} ∼ {p(Bn )}, and consequently, there exists n0 ∈ N such that p(An ) − p(Bn )F √ < , n

∀n ≥ n0 .

(3.7)

3.2 Asymptotically Equivalent Sequences of Hermitian Matrices

199

From the triangle inequality, property (2) of Lemma 2.4, (2.7), (3.6) and (3.7) we conclude that g(An ) − g(Bn )F √ n =

g(An ) − p(An ) + p(An ) − p(Bn ) + p(Bn ) − g(Bn )F √ n

≤

g(An ) − p(An )F + p(An ) − p(Bn )F + p(Bn ) − g(Bn )F √ n

p(An ) − p(Bn )F (p − g)(Bn )F (g − p)(An )F √ √ √ + + n n n 1 dn 2 2 |(g − p)(λ (A ))| n k k=1 p(An ) − p(Bn )F √ √ + = n n 1 dn 2 2 k=1 |(p − g)(λk (Bn ))| √ + n 1 1 dn dn 2 2 2 2 k=1 k=1 √ √ ++ < n n √ √ dn dn = √ ++ √ n n dn +1 = 2 n √ ≤ (2 M + 1), ∀n ≥ n0 , =

where M ∈ (0, ∞) is an upper bound of the bounded sequence dn n . We finish this section with a result that can be obtained as a corollary of Theorem 3.5.

200


Theorem 3.6. Let {dn }, {An }, {Bn } and [a, b] be as in Lemma 3.4. Consider g ∈ C[a, b] and suppose that dnn is bounded. Then n 1 (g(λk (An )) − g(λk (Bn ))) = 0. n→∞ n k=1 d n n g(λk (Bn )) is convergent then n1 dk=1 g(λk (An )) Hence if n1 k=1 is also convergent and

d

lim

n n 1 1 lim g(λk (An )) = lim g(λk (Bn )). n→∞ n n→∞ n

d

d

k=1

k=1

Proof. If dnn ≤ M < ∞ for all n ∈ N, then d n tr(g(A )) − tr(g(B )) 1 n n (g(λk (An )) − g(λk (Bn ))) = 0≤ n n k=1 √ dn g(An ) − g(Bn )F |tr(g(An ) − g(Bn ))| ≤ = n n dn g(An ) − g(Bn )F √ g(An ) − g(Bn )F √ √ ≤ M . = n n n Taking into account that from Theorem 3.5 we have lim

n→∞

g(An ) − g(Bn )F √ = 0, n

we deduce that

d n 1 (g(λk (An )) − g(λk (Bn ))) = 0, lim n→∞ n k=1

or equivalently, n 1 (g(λk (An )) − g(λk (Bn ))) = 0. n→∞ n

d

lim

k=1

Theorems 3.5 and 3.6 were introduced for the case in which {dn } = {n} in [12] and [8], respectively. Observe that if {dn } = {n} then the sequence dnn = {1} is bounded.

4 Block Toeplitz Matrices

4.1

Definition and Examples

We begin this section by reviewing the concept of block Toeplitz matrix. Definition 4.1. An m × n block Toeplitz matrix with M × N blocks is an mM × nN matrix of the form   F−1 F−2 · · · F1−n F0  F1 F0 F−1 · · · F2−n     F2 F1 F0 · · · F3−n  (4.1) ,    . . . . . .. .. .. ..   .. Fm−1

Fm−2

Fm−3

···

F0

where Fk ∈ CM ×N with 1 − n ≤ k ≤ m − 1. When M = N = 1 the matrix in (4.1) is simply an m × n Toeplitz matrix.

201

202

Block Toeplitz Matrices

A matrix



A1,1  .. A= .

···

Am,1

···

 A1,n ..  ∈ CmM ×nN , . 

(4.2)

Am,n

with Aj,k ∈ CM ×N ,

1 ≤ j ≤ m, 1 ≤ k ≤ n,

is denoted by A = (Aj,k )1≤j≤m,1≤k≤n , and by A = (Aj,k )nj,k=1 if m = n. The submatrix Aj,k is also denoted by [A]j,k , and by aj,k when M = N = 1. The matrix in (4.2) is an m × n block Toeplitz matrix with M × N blocks if Aj1 ,k1 = Aj2 ,k2 whenever j1 − k1 = j2 − k2 . We now present two interesting examples of block Toeplitz matrices that are frequently used in Communications, Information Theory and Signal Processing. For those two examples, we need to introduce the following notation. Let {xn }n∈Z be a sequence in CN ×1 , i.e., xn ∈ CN ×1 for all n ∈ Z. If j, k ∈ Z with j ≤ k, then   xk xk−1    xk−2  xk:j :=  .  .   ..  xj Example 4.1. In this first example we consider a discrete-time causal finite impulse response (FIR) multiple-input multiple-output (MIMO) filter, that is, a filter of the form yn =

m

H−k xn−k ,

∀n ∈ Z,

(4.3)

k=0

where the filter taps H−k , with 0 ≤ k ≤ m, are M × N complex matrices, and the input {xn }n∈Z and the output {yn }n∈Z of the filter satisfy that xn ∈ CN ×1 and yn ∈ CM ×1 for all n ∈ Z. From (4.3) we have yn:1 = Hn xn:1−m ,

∀n ∈ N,

203

4.1 Definition and Examples

where Hn is the n × (n + m) block Toeplitz matrix with M × N blocks given by   H0 H−1 H−2 · · · H−m 0M ×N 0M ×N · · · 0M ×N 0M ×N H0 H−1 · · · H1−m H−m 0M ×N · · · 0M ×N    0M ×N 0M ×N  H · · · H H H · · · 0 0 2−m 1−m −m M ×N  ,  . .. .. ..  ..  .. . . . .  0M ×N

0M ×N

0M ×N

···

···

H−m

i.e., Hn = (Hj−k )1≤j≤n,1≤k≤n+m

(4.4)

with Hj−k = 0M ×N when j − k ∈ {−m, . . . , −1, 0}. Thus, matrix representations of discrete-time causal FIR MIMO filters are block Toeplitz.

Example 4.2. In this second example, we consider a vector wide sense stationary (WSS) process. Let xn be a (column) random vector of dimension N for all n ∈ Z. The N -dimensional vector random process1 {xn : n ∈ Z} is said to be WSS or weakly stationary if it has constant mean, that is, E(xn1 ) = E(xn2 ),

∀n1 , n2 ∈ Z,

and if it satisfies E(xn1 x∗n2 ) = E(xn3 x∗n4 )

(4.5)

whenever n1 − n2 = n3 − n4 . From (4.5) we have E(xn:1 x∗n:1 ) = (Xj−k )nj,k=1 ,

∀n ∈ N,

where Xj−k = E(xk x∗j ) ∈ CN ×N ,

∀j, k ∈ N.

Thus, the correlation matrix E(xn:1 x∗n:1 ) of the random vector xn:1 is an n × n block Toeplitz matrix with N × N blocks for all n ∈ N. 1 Vector

random processes are also called multivariate random processes.

204


4.2

Sequence of Block Toeplitz Matrices Generated by a Continuous Function

Consider a matrix-valued function F : [0, 2π] → CM ×N . Suppose that F is continuous, that is, the complex function [F ]r,s , with [F ]r,s (ω) := [F (ω)]r,s , ω ∈ [0, 2π], is continuous for all 1 ≤ r ≤ M and 1 ≤ s ≤ N . 2π Then 0 F (ω)dω denotes the M × N matrix given by 2π 2π F (ω)dω := [F (ω)]r,s dω . (4.6) 0

0

1≤r≤M,1≤s≤N

The following lemma provides three simple properties of the integral introduced in (4.6). Lemma 4.1. Let F, G : [0, 2π] → CM ×N be two continuous matrixvalued functions. (1) If α, β ∈ C then 2π (αF (ω) + βG(ω))dω = α 0

2π 0

0

G(ω)dω. 0

(2) If A ∈ CL×M and B ∈ CN ×K then 2π AF (ω)Bdω = A

2π

F (ω)dωB. 0

(3) If M = N = 1 then 2π F (ω)Adω = 0

2π

F (ω)dω + β

2π

F (ω)dωA ∀A ∈ CL×K .

0

Proof. (1) Since F and G are continuous, the matrix-valued function αF (ω) + βG(ω), with ω ∈ [0, 2π], is also continuous. Moreover, we have 2π (αF (ω) + βG(ω))dω 0

2π

[αF (ω) + βG(ω)]r,s dω

= 0

1≤r≤M,1≤s≤N

2π

= 0

(α[F (ω)]r,s + β[G(ω)]r,s )dω 1≤r≤M,1≤s≤N

4.2 Sequence of Block Toeplitz Matrices Generated by a Continuous Function

= α

[F (ω)]r,s dω + β

0

[G(ω)]r,s dω 1≤r≤M,1≤s≤N

[F (ω)]r,s dω

0

+β 0

1≤r≤M,1≤s≤N

2π

[G(ω)]r,s dω 1≤r≤M,1≤s≤N

2π

=α

2π 0

2π

=α

2π

205

2π

F (ω)dω + β 0

G(ω)dω. 0

(2) Since F is continuous, the matrix-valued function AF (ω)B, with ω ∈ [0, 2π], is also continuous. Furthermore,

2π

AF (ω)Bdω = 0

2π

[AF (ω)B]r,s dω

0

1≤r≤L,1≤s≤K

=

[A]r,k [F (ω)B]k,s dω

0

k=1

N [A]r,k [F (ω)]k,l [B]l,s dω

0

= = =

1≤r≤L,1≤s≤K

M 2π

=

M 2π

k=1

l=1

N M [A]r,k k=1

l=1

1≤r≤L,1≤s≤K

F (ω)dω 0

=A

1≤r≤L,1≤s≤K

F (ω)dωB

2π

k,s

1≤r≤L,1≤s≤K

F (ω)dωB

2π

F (ω)dωB. 0

[B]l,s k,l

2π

0

2π

0

k=1

= A

1≤r≤L,1≤s≤K

[F (ω)]k,l dω[B]l,s

0

l=1

M [A]r,k

2π

N M [A]r,k k=1

r,s

1≤r≤L,1≤s≤K

206


(3) Since F is continuous, the matrix-valued function F (ω)A, with ω ∈ [0, 2π], is also continuous. Moreover, we have 2π 2π F (ω)Adω = [F (ω)A]r,s dω 0

0

1≤r≤L,1≤s≤K

=

F (ω)[A]r,s dω

0

0

1≤r≤L,1≤s≤K

F (ω)dω[A]r,s 1≤r≤L,1≤s≤K

2π

=

2π

=

2π

F (ω)dω([A]r,s )1≤r≤L,1≤s≤K

0 2π

=

F (ω)dωA. 0

We can now define the sequence of block Toeplitz matrices generated by a continuous matrix-valued function. Definition 4.2. Consider a matrix-valued function of a real variable F : R → CM ×N . Suppose that F is continuous and 2π-periodic, that is, the complex function [F ]r,s , with [F ]r,s (ω) := [F (ω)]r,s , ω ∈ R, is continuous and 2π-periodic for all 1 ≤ r ≤ M and 1 ≤ s ≤ N . For every n ∈ N, we define Tn (F ) as the n × n block Toeplitz matrix with M × N blocks given by Tn (F ) := (Fj−k )nj,k=1 , where {Fk }k∈Z is the sequence of Fourier coefficients of F (i.e., {[Fk ]r,s }k∈Z is the sequence of Fourier coefficients of [F ]r,s for all 1 ≤ r ≤ M and 1 ≤ s ≤ N ): 2π 1 e−kωi F (ω)dω, ∀k ∈ Z, Fk = 2π 0 where i is the imaginary unit. The sequence {Tn (F )} is called the sequence of block Toeplitz matrices generated by F , and the continuous matrix-valued function F is called the generating function or the symbol of the sequence {Tn (F )}.


207

When the sequence of block Toeplitz matrices {Tn (F )} is the sequence of correlation matrices {E(xn:1 x∗n:1 )} of Example 4.2, the generating function F is called the power spectral density of the vector WSS process {xn : n ∈ Z}. The following lemma provides four properties of sequences of block Toeplitz matrices generated by continuous matrix-valued functions. Lemma 4.2. Let F, G : R → CM ×N , and suppose that they are continuous and 2π-periodic. Then (1) {Tn (F ∗ )} = {(Tn (F ))∗ }, where (F ∗ )(ω) := (F (ω))∗ , ω ∈ R. 2π T (F )2 1 2 (2) n n F ≤ 2π 0 F (ω)F dω for all n ∈ N. (3) {Tn (αF + βG)} = {αTn (F ) + βTn (G)} for all α, β ∈ C, where (αF + βG)(ω) := αF (ω) + βG(ω), ω ∈ R. (4) {Tn (F )} = {Tn (G)} if and only if F = G. Proof. (1) Since F : R → CM ×N is continuous and 2π-periodic, F ∗ : R → CN ×M is also continuous and 2π-periodic. Moreover, we have  n  1 2π ∗ −(j−k)ωi {Tn (F )} = [(F )(ω)]r,s e dω  2π 0 1≤r≤N,1≤s≤M

 

 n  1 2π [(F (ω))∗ ]r,s e−(j−k)ωi dω =  2π 0 1≤r≤N,1≤s≤M

 

∗



j,k=1

  =



1 2π

2π

−(j−k)ωi

[F (ω)]s,r e

0

  

1 2π

dω 1≤r≤N,1≤s≤M

2π 0

(j−k)ωi

[F (ω)]s,r e

j,k=1

 1≤r≤N,1≤s≤M

dω 1≤s≤M,1≤r≤N



 

n

n (([Tn (F )]k,j )∗ )j,k=1 ∗ = {(Tn (F ))∗ }. = ([Tn (F )]j,k )n j,k=1

=

 

n

 2π  1 (j−k)ωi [F (ω)]s,r e dω =   2π 0

=



j,k=1

j,k=1

∗ n

   

j,k=1

208


(2) Let {Fk }k∈Z be the sequence of Fourier coefficients of F . From the Parseval theorem (see, e.g., [20, p. 191]) we obtain 2π ∞ 1 |[F (ω)]r,s |2 dω = |[Fk ]r,s |2 , 1 ≤ r ≤ M, 1 ≤ s ≤ N. 2π 0 k=−∞

Consequently, 2π 2π M N 1 1 2 F (ω)F dω = |[F (ω)]r,s |2 dω 2π 0 2π 0 r=1 s=1 =

=

2π M N 1 |[F (ω)]r,s |2 dω 2π 0 r=1 s=1 N ∞ M

|[Fk ]r,s |2

r=1 s=1 k=−∞ M ∞ N

=

|[Fk ]r,s |2 =

k=−∞ r=1 s=1

Therefore, nF0 2F + Tn (F )2F = n

n−1 ≤

≤

n−1

k=1 (n

2 k=−(n−1) nFk F

n ∞ k=−∞

Fk 2F =

∞

Fk 2F .

k=−∞

− k) Fk 2F + F−k 2F n n−1

=

Fk 2F

k=−(n−1)

1 2π

2π

F (ω)2F dω,

∀n ∈ N.

0

(3) Since F, G : R → CM ×N are continuous and 2π-periodic, αF + βG : R → CM ×N is also continuous and 2π-periodic. Furthermore, we have {Tn (αF + βG)} n 2π 1 −(j−k)ωi e ((αF + βG)(ω))dω = 2π 0 j,k=1 n 2π 1 −(j−k)ωi −(j−k)ωi αF (ω) + e βG(ω) dω e = 2π 0 j,k=1

209


=

α 2π

2π

−(j−k)ωi

e 0

β F (ω)dω + 2π

2π

e

+β

1 2π

2π

−(j−k)ωi

e

n

n G(ω)dω

0

n 2π 1 e−(j−k)ωi F (ω)dω = α 2π 0 j,k=1

−(j−k)ωi

j,k=1

G(ω)dω

0

j,k=1

= {αTn (F ) + βTn (G)}. (4) If {Tn (F )} = {Tn (G)} then F and G have the same sequence of Fourier coefficients, i.e., the continuous 2π-periodic functions [F ]r,s and [G]r,s have the same sequence of Fourier coefficients for all 1 ≤ r ≤ M and 1 ≤ s ≤ N . Since from the Fejér theorem (see, e.g., [20, p. 199]) two continuous 2π-periodic functions with the same sequence of Fourier coefficients are equal, [F ]r,s = [G]r,s for all 1 ≤ r ≤ M and 1 ≤ s ≤ N . Consequently, F = G. We now present a result that gives an upper bound for the singular values of the block Toeplitz matrices generated by a continuous matrixvalued function. Theorem 4.3. Let F : R → CM ×N be a matrix-valued function which is continuous and 2π-periodic. Then σ1 (Tn (F )) ≤ σ1 (F ),

∀n ∈ N,

(4.7)

where σ1 (F ) := sup σ1 (F (ω)) < ∞. ω∈[0,2π]

Proof. We begin by proving that σ1 (F ) < ∞. It is well known that if A is an m × n matrix, the m eigenvalues of the Hermitian matrix AA∗ are given by (σ1 (A))2 , . . . , (σm (A))2 when m ≤ n, and by (σ1 (A))2 , . . . , (σn (A))2 , 0, . . . , 0 when n < m. Consequently, ∗

tr(F (ω)(F (ω)) ) =

M k=1

∗

λk (F (ω)(F (ω)) ) =

min{M,N }

k=1

(σk (F (ω)))2

210


for all ω ∈ R. Therefore, (σ1 (F (ω0 )))2 ≤ tr(F (ω0 )(F (ω0 ))∗ ) ≤ max tr(F (ω)(F (ω))∗ ),

∀ω0 ∈ [0, 2π],

ω∈[0,2π]

where the existence of maxω∈[0,2π] tr(F (ω)(F (ω))∗ ) is guaranteed by the continuity of the real function tr(F (ω)(F (ω))∗ ) on the closed interval [0, 2π]. Hence, σ1 (F (ω0 )) ≤

max tr(F (ω)(F (ω))∗ ),

∀ω0 ∈ [0, 2π],

ω∈[0,2π]

and this guarantees that σ1 (F ) < ∞. We now proceed to prove (4.7). Fix n ∈ N, and let Tn (F ) = U ΣV ∗ be a singular value decomposition of Tn (F ). Since U and V are unitary, we obtain n [U ∗ ]1,h [Tn (F )V ]h,1 σ1 (Tn (F )) = [Σ]1,1 = [U Tn (F )V ]1,1 = ∗

h=1

=

=

n

n

h=1

l=1

[U ∗ ]1,h

2π n n 1 [U ]∗h,1 e−(h−l)ωi F (ω)dω[V ]l,1 2π 0 h=1

=

[Tn (F )]h,l [V ]l,1

1 2π

1 = 2π 1 = 2π

l=1

n 2π

n

h=1

l=1

[U ]∗h,1

0

2π

0

2π

n

hωi

e

e(−h+l)ωi F (ω)[V ]l,1 dω

∗

[U ]h,1

F (ω)

h=1 ∗

(S(ω)) 0

(W (ω))∗

n h=1

n l=1

n

lωi

e

l=1 ∗

ehωi [U ]h,1

elωi [V ]l,1 dω

Λ(ω)

[V ]l,1 dω


1 = 2π

2π min{M,N } 0

(W (ω))∗

σj (F (ω))

j=1 n l=1

∗

(S(ω))

h=1

elωi [V ]l,1

n

211

∗ hωi

e

[U ]h,1 1,j

dω j,1

∗ 2π min{M,N n } 1 ∗ hωi σj (F (ω)) (S(ω)) e [U ]h,1 ≤ 2π 0 j=1 h=1 1,j n ∗ lωi dω, e [V ]l,1 (W (ω)) (4.8) l=1 j,1 where F (ω) = S(ω)Λ(ω)(W (ω))∗ is a singular value decomposition of the matrix F (ω) for all ω ∈ [0, 2π]. Applying the Cauchy–Schwarz inequality (see, e.g., [1, p. 14]) we have ∗ min{M,N n n } σj (F (ω)) (S(ω))∗ ehωi [U ]h,1 (W (ω))∗ elωi [V ]l,1 j=1 h=1 l=1 1,j j,1 ∗ min{M,N } n ∗ hωi ≤ σj (F (ω)) (S(ω)) e [U ]h,1 j=1 h=1 1,j n lωi (W (ω))∗ e [V ]l,1 l=1 j,1 ∗ min{M,N } n (S(ω))∗ ≤ σ1 (F ) ehωi [U ]h,1 j=1 h=1 1,j n lωi (W (ω))∗ e [V ]l,1 l=1 j,1 ! ∗ 2 } !min{M,N n ! ∗ ≤ σ1 (F )" ehωi [U ]h,1 (S(ω)) j=1 h=1 1,j ! 2 !min{M,N } n ! ∗ " elωi [V ]l,1 (W (ω)) j=1 l=1 j,1

212


! ∗ 2 ! n !M (S(ω))∗ ehωi [U ]h,1 ≤ σ1 (F )" j=1 h=1 1,j ! 2 ! N n ! (W (ω))∗ , " elωi [V ]l,1 j=1 l=1 j,1

∀ω ∈ [0, 2π].

(4.9)

From (4.8) and (4.9), and using the Cauchy–Schwarz inequality (see, e.g., [20, p. 139]) we obtain ! ∗ ! M n σ1 (F ) ! 2π ∗ " ehωi [U ]h,1 σ1 (Tn (F )) ≤ (S(ω)) 2π 0 j=1

h=1

! ! ! "

2 (W (ω))∗ dω elωi [V ]l,1 j=1 l=1 j,1

N 2π

0

2 dω 1,j

n

! ! 2π n M ! 1 (S(ω))∗ ehωi [U ]h,1 = σ1 (F )" 2π 0 j=1

2 dω j,1

h=1

! ! 2π n N ! 1 (W (ω))∗ " elωi [V ]l,1 2π 0 j=1

2 dω. j,1

l=1

On the other hand, we have 2 2π n N 1 (W (ω))∗ dω elωi [V ]l,1 2π 0 j=1 l=1 j,1 ∗ 2π n N 1 ehωi [V ]h,1 (W (ω))∗ = 2π 0 j=1

(W (ω))∗

n l=1

h=1

elωi [V ]l,1

dω j,1

1,j

(4.10)


1 = 2π =

=

=

=

1 2π 1 2π 1 2π

2π

∗

(W (ω)) 0

0

0

e

[V ]h,1

∗

(W (ω))

elωi [V ]l,1 dω

n

elωi [V ]l,1 dω

l=1

e−hωi [V ]∗h,1

n

elωi [V ]l,1 dω

l=1

n 2π

n

h=1

l=1

[V ]∗h,1

0

n l=1

e−hωi [V ]∗h,1 W (ω)(W (ω))∗

h=1

e(−h+l)ωi [V ]l,1 dω

2π n n n 1 [V ]∗h,1 e(−h+l)ωi dω[V ]l,1 = [V ]∗h,1 [V ]h,1 2π 0 h=1

=

hωi

h=1 n 2π

∗

h=1 n 2π

n

213

l=1

h=1

n

[V ∗ ]1,h [V ]h,1 = [V ∗ V ]1,1 = [InN ]1,1 = 1,

(4.11)

h=1

because 1 2π

#

2π mωi

e

dω =

0

1, if m = 0, 0, if m is a non-zero integer number.

By reasoning as in (4.11) we also obtain 1 2π

0

2 n (S(ω))∗ dω = 1. ehωi [U ]h,1 j=1 h=1 j,1

M 2π

(4.12)

Using (4.10), (4.11) and (4.12) yields (4.7). Observe that if M = N = 1 then σ1 (F ) = max |F (ω)|. ω∈[0,2π]

The next result provides an upper bound and a lower bound for the eigenvalues of the block Toeplitz matrices generated by a continuous matrix-valued function which is Hermitian.

214


Theorem 4.4. Consider a matrix-valued function F : R → CN ×N which is continuous and 2π-periodic. (1) F is Hermitian (that is, F ∗ = F ) if and only if Tn (F ) is Hermitian for all n ∈ N. (2) If F is Hermitian then inf F ≤ λk (Tn (F )) ≤ sup F,

1 ≤ k ≤ nN, n ∈ N,

(4.13)

where inf F :=

inf

1≤k≤N ω ∈ [0, 2π]

λk (F (ω)) > −∞,

and sup F :=

λk (F (ω)) < ∞.

sup 1≤k≤N ω ∈ [0, 2π]

Proof. (1) From property (1) of Lemma 4.2, if F is Hermitian we have (Tn (F ))∗ = Tn (F ∗ ) = Tn (F ),

∀n ∈ N.

The reciprocal is also true because if {Tn (F )} = {(Tn (F ))∗ } = {Tn (F ∗ )} applying property (4) of Lemma 4.2 yields F = F ∗ . (2) We begin by proving that inf F > −∞ and sup F < ∞. Since tr((F (ω))2 ) =

N N (λk (F (ω)))2 = |λk (F (ω))|2 , k=1

∀ω ∈ R,

k=1

we obtain |λk (F (ω0 ))|2 ≤ max tr((F (ω))2 ), ω∈[0,2π]

ω0 ∈ [0, 2π],

1 ≤ k ≤ N,

where the existence of maxω∈[0,2π] tr((F (ω))2 ) is guaranteed by the continuity of the real function tr((F (ω))2 ) on the closed interval [0, 2π]. Hence, if ω0 ∈ [0, 2π] and 1 ≤ k ≤ N then max tr((F (ω))2 ) ≤ λk (F (ω0 )) ≤ max tr((F (ω))2 ), − ω∈[0,2π]

ω∈[0,2π]

and this guarantees that inf F > −∞ and sup F < ∞.

215


We now proceed to prove (4.13). Fix n ∈ N, and let Tn (F ) = U diag(λ1 (Tn (F )), . . . , λnN (Tn (F )))U −1 be an eigenvalue decomposition of the Hermitian matrix Tn (F ), where the eigenvector matrix U is unitary. If 1 ≤ k ≤ nN then ∗

λk (Tn (F )) = [U Tn (F )U ]k,k

n = [U ∗ ]k,h [Tn (F )U ]h,k h=1

=

n

n

h=1

l=1

[U ∗ ]k,h

[Tn (F )]h,l [U ]l,k

2π n n 1 ∗ = [U ]h,k e−(h−l)ωi F (ω)dω[U ]l,k 2π 0 h=1

=

1 2π

1 = 2π =

1 2π

l=1

n 2π

n

h=1

l=1

[U ]∗h,k

0

2π

0

2π

n

e(−h+l)ωi F (ω)[U ]l,k dω

∗

ehωi [U ]h,k

F (ω)

h=1

elωi [U ]l,k dω

l=1

(W (ω))∗

0

n

n

∗ ehωi [U ]h,k

h=1

n diag(λ1 (F (ω)), . . . , λN (F (ω))) (W (ω))∗ elωi [U ]l,k dω 1 = 2π

0

N 2π

∗

λj (F (ω))

j=1

(W (ω))∗

l=1

n l=1

(W (ω))

h=1

elωi [U ]l,k

n

hωi

e

∗

[U ]h,k 1,j

dω j,1

2 2π n N 1 dω, λj (F (ω)) (W (ω))∗ elωi [U ]l,k = 2π 0 j=1 l=1 j,1 where F (ω) = W (ω)diag(λ1 (F (ω)), . . . , λN (F (ω)))(W (ω))−1 is an eigenvalue decomposition of the Hermitian matrix F (ω), and the

216


eigenvector matrix W (ω) is unitary for all ω ∈ [0, 2π]. Consequently, 2 2π n N 1 (W (ω))∗ dω ≤ λk (Tn (F )) inf F elωi [U ]l,k 2π 0 j=1 l=1 j,1 2 2π n N 1 (W (ω))∗ dω elωi [U ]l,k (4.14) ≤ sup F 2π 0 j=1 l=1 j,1 for all 1 ≤ k ≤ nN . By reasoning as in (4.11) we obtain 2 2π n N 1 (W (ω))∗ dω = 1 elωi [U ]l,k 2π 0 j=1 l=1 j,1

(4.15)

for all 1 ≤ k ≤ nN . Using (4.14) and (4.15) yields (4.13). Observe that if N = 1 then inf F = min F (ω), ω∈[0,2π]

and sup F = max F (ω). ω∈[0,2π]

The product of block Toeplitz matrices is not, in general, a block Toeplitz matrix: 1 2 1 0 5 2 = . 0 1 2 1 2 1 To end this subsection, we give a non-trivial example of block Toeplitz matrix that can be obtained as the product of block Toeplitz matrices. Lemma 4.5. Consider a matrix-valued function F : R → CN ×N which is continuous and 2π-periodic. For every n ∈ N, let Hn be the n × (n + m) block Toeplitz matrix with M × N blocks in (4.4). Then {Tn (HF H ∗ )} = {Hn Tn+m (F )Hn∗ }, where (HF H ∗ )(ω) = H(ω)F (ω)(H(ω))∗ and H(ω) = for all ω ∈ R.

0

kωi H k k=−m e


217

Proof. Since F : R → CN ×N and H : R → CM ×N are continuous and 2π-periodic, HF H ∗ : R → CM ×M is also continuous and 2π-periodic. For every n ∈ N we have ∗

[Tn (HF H )]j,k

1 = 2π =

m h=0

=

m h=0

=

m

0

H−h

e−(j−k)ωi H(ω)F (ω)(H(ω))∗ dω

1 2π

=

=

=

2π (−(j−k)−h)ωi

e

F (ω)(H(ω)) dω

H−h

Fj−k+h−l H∗−l

l=0

m m [Hn ]j,j+h [Tn+m (F )]j+h,k+l [Hn ]∗k,k+l l=0

j+m

k+m

ˆ h=j

ˆ l=k

j+m

n+m

ˆ h=j

ˆ l=1

[Hn ]j,hˆ

[Hn ]j,hˆ

∗ [Tn+m (F )]h, ˆˆ l [Hn ]k,ˆ l

∗ [Tn+m (F )]h, ˆˆ l [Hn ]k,ˆ l

j+m

n+m

ˆ h=j

ˆ l=1

[Hn ]j,hˆ

∗ [Tn+m (F )]h, ˆˆ l [Hn ]ˆ l,k

j+m

[Hn ]j,hˆ [Tn+m (F )Hn∗ ]h,k ˆ

ˆ h=j

=

∗

0

l=0

m

h=0

=

2π m 1 (−(j−k)−h+l)ωi H−h e F (ω)dω H∗−l 2π 0

h=0

=

2π

n+m

[Hn ]j,hˆ [Tn+m (F )Hn∗ ]h,k ˆ

ˆ h=1

= [Hn Tn+m (F )Hn∗ ]j,k ,

1 ≤ j, k ≤ n,

where {Fk }k∈Z is the sequence of Fourier coefficients of F .

218


4.3

Examples of Continuous Generating Functions

We now present three interesting examples of continuous symbols. Example 4.3. A simple example of continuous symbols are the trigonometric polynomials. An M × N trigonometric polynomial is a matrix-valued function of a real variable F : R → CM ×N of the form F (ω) =

m

ehωi Ah ,

∀ω ∈ R,

(4.16)

h=−m

where m ∈ N ∪ {0} and Ah ∈ CM ×N with |h| ≤ m. Obviously, the function F in (4.16) is continuous and 2π-periodic. Furthermore, its sequence of Fourier coefficients {Fk }k∈Z is given by 1 Fk = 2π =

1 2π

2π

m

−kωi

e 0

h=−m 2π

0

ehωi Ah dω

m

e(h−k)ωi Ah dω

h=−m

# 2π m 1 Ak , if |k| ≤ m, (h−k)ωi e dωAh = = 0M ×N , if |k| > m. 2π 0 h=−m

Consequently, the block Toeplitz matrix Tn (F ) = (Fj−k )nj,k=1 generated by the trigonometric polynomial F is banded2 with bandwidth max{(m + 1)M − 1, (m + 1)N − 1} for all n > m + 1.

Example 4.4. Let {Fk }k∈Z be the sequence of Fourier coefficients of a matrix-valued function F : R → CM ×N which is continuous and 2πperiodic. We say that F is in the Wiener class if the complex function [F ]r,s is in the Wiener class for all 1 ≤ r ≤ M and 1 ≤ s ≤ N , i.e., if the 2A

= (aj,k )1≤j≤m,1≤k≤n ∈ Cm×n is banded with bandwidth b ∈ N ∪ {0} if aj,k = 0 whenever |j − k| > b. Banded matrices are also called band matrices.

4.3 Examples of Continuous Generating Functions

219

sequence of Fourier coefficients {[Fk ]r,s }k∈Z of [F ]r,s satisfies ∞

|[Fk ]r,s | < ∞

k=−∞

for all 1 ≤ r ≤ M and 1 ≤ s ≤ N . In Appendix B, we present the Wiener class in more detail.

Example 4.5. We prove here that continuous functions of a continuous symbol which is Hermitian are also examples of continuous symbols. Consider a Hermitian matrix-valued function F : R → CN ×N which is continuous and 2π-periodic. If g ∈ C[inf F, sup F ], since F is 2π-periodic, the matrix-valued function g(F ): R → CN ×N with (g(F ))(ω) := g(F (ω)), ω ∈ R, is also 2π-periodic. To finish this example we need to prove that g(F ) is continuous. Let F (ω) = W (ω)diag(λ1 (F (ω)), . . . , λN (F (ω)))(W (ω))−1 be an eigenvalue decomposition of the Hermitian matrix F (ω), where the eigenvector matrix W (ω) is unitary for all ω ∈ R. As g is a continuous complex function on [inf F, sup F ] ⊂ R, from the Stone–Weierstrass theorem there exists a sequence of polynomials {pn (x)} with coefficients in C, that converges uniformly to g on [inf F, sup F ]. Thus, for every ∈ (0, ∞) there exists n0 ∈ N such that |pn (x) − g(x)| < ,

x ∈ [inf F, sup F ], n ≥ n0 .

Applying the Cauchy–Schwarz inequality, we obtain |[pn (F (ω))]j,k − [g(F (ω))]j,k | = |[pn (F (ω)) − g(F (ω))]j,k | = |[(pn − g)(F (ω))]j,k | = |[W (ω)diag((pn − g)(λ1 (F (ω))), . . . , (pn − g)(λN (F (ω)))) (W (ω))∗ ]j,k | N = (pn − g)(λl (F (ω)))[W (ω)]j,l [(W (ω))∗ ]l,k l=1

220


N = (pn − g)(λl (F (ω)))[W (ω)]j,l [W (ω)]k,l l=1

≤

N

|(pn − g)(λl (F (ω)))||[W (ω)]j,l | [W (ω)]k,l

l=1

≤

N

!N !N ! ! |[W (ω)]j,l ||[W (ω)]k,l | ≤ " |[W (ω)]j,l |2 " |[W (ω)]k,l |2

l=1

l=1

l=1

!N !N ! ! " = [W (ω)]j,l [W (ω)]j,l " [W (ω)]k,l [W (ω)]k,l l=1

l=1

!N !N ! ! " ∗ = [W (ω)]j,l [(W (ω)) ]l,j " [W (ω)]k,l [(W (ω))∗ ]l,k $ = = ,

l=1

[W (ω)(W (ω))∗ ]j,j ω ∈ R,

$

l=1

$ $ [W (ω)(W (ω))∗ ]k,k = [IN ]j,j [IN ]k,k

1 ≤ j, k ≤ N,

n ≥ n0 .

(4.17)

Fix 1 ≤ j, k ≤ N . On the one hand, the complex function [pn (F (ω))]j,k is continuous on R for all n ∈ N, because F is continuous and the pn (x) are polynomials. On the other hand, from (4.17) the sequence {[pn (F (ω))]j,k } converges uniformly to the complex function [g(F (ω))]j,k on R. Consequently, [g(F (ω))]j,k is also continuous on R. This finishes the proof of the continuity of g(F ).

5 Block Circulant Matrices

5.1

Definition

This section is devoted to a special type of block Toeplitz matrices: block circulant matrices. This type of matrices will be used in Section 6 to study the asymptotic behaviour of the block Toeplitz matrices generated by a continuous matrix-valued function. We begin by reviewing the concept of a block circulant matrix. Definition 5.1. An n × n block circulant matrix with M × N blocks is an nM × nN matrix of the form   C−1 C−2 · · · C1−n C0 C1−n C0 C−1 · · · C2−n    C2−n C1−n C0 · · · C3−n  (5.1) ,    . . . . . . . . . .  . . . . .  C−1

C−2

C−3

···

C0

where Ck ∈ CM ×N with 1 − n ≤ k ≤ 0. Thus, an nM × nN matrix C = (Cj,k )nj,k=1 is an n × n block circulant matrix with M × N blocks if Cj1 ,k1 = Cj2 ,k2 , whenever (j1 − k1 ) − (j2 − k2 ) ∈ {−n, 0, n}. 221

222

Block Circulant Matrices

The matrix in (5.1) is denoted by circ(C0 , C−1 , . . . , C1−n ). When M = N = 1 this matrix is simply an n × n circulant matrix. The next result, which can be found in [13], characterizes block circulant matrices. Lemma 5.1. Let C = (Cj,k )nj,k=1 ∈ CnM ×nN , then the following statements are equivalent: (1) C is an n × n block circulant matrix with M × N blocks. (2) There exist A1 , . . . , An ∈ CM ×N such that C = (Vn ⊗ IM )diag(A1 , . . . , An )(Vn ⊗ IN )∗ , where ⊗ is the Kronecker product, diag(A1 , . . . , An ) = diag1≤k≤n (Ak ) := (δj,k Aj )nj,k=1 , δj,k is the Kronecker delta, and Vn is the n × n Fourier unitary matrix: 2π(j−1)(k−1) 1 i n , [Vn ]j,k := √ e− n

1 ≤ j, k ≤ n.

Furthermore, if C is a block circulant matrix then     C1,1 A1 C   A2  √   ∗  2,1   ..  = n(Vn ⊗ IM )  ..  .  .   .  An

(5.2)

Cn,1

Proof. (1)⇒(2) Vn ⊗ IM is unitary because ∗ ) (Vn ⊗ IM )(Vn ⊗ IM )∗ = (Vn ⊗ IM )(Vn∗ ⊗ IM

∗ = In ⊗ IM = InM , = Vn Vn∗ ⊗ IM IM

and ∗ )(Vn ⊗ IM ) (Vn ⊗ IM )∗ (Vn ⊗ IM ) = (Vn∗ ⊗ IM

∗ IM = In ⊗ IM = InM . = Vn∗ Vn ⊗ IM

Analogously, it can be proved that Vn ⊗ IN is unitary. Consequently, if D = (Vn ⊗ IM )∗ C(Vn ⊗ IN ) ∈ CnM ×nN , then C = (Vn ⊗ IM )D(Vn ⊗ IN )∗ . We now proceed to prove that D = diag(A1 , . . . , An ) with

5.1 Definition

A1 , . . . , An satisfying (5.2). For every 1 ≤ j, k ≤ n, we obtain ∗ Dj,k = [(Vn ⊗ IM )∗ C(Vn ⊗ IN )]j,k = [(Vn∗ ⊗ IM )C(Vn ⊗ IN )]j,k

= [(Vn∗ ⊗ IM )C(Vn ⊗ IN )]j,k =

n

[Vn∗ ⊗ IM ]j,h [C(Vn ⊗ IN )]h,k

h=1

=

n

[Vn∗ ⊗ IM ]j,h

n

h=1

=

n

[Vn∗ ]j,h IM

h=1

=

Ch,l [Vn ⊗ IN ]l,k

l=1 n

Ch,l ([Vn ]l,k IN ) =

l=1

n n

n

[Vn ]h,j

h=1

n

[Vn ]l,k Ch,l

l=1

[Vn ]h,j [Vn ]l,k Ch,l

h=1 l=1

=

n h

[Vn ]h,j [Vn ]l,k Ch,l +

h=1 l=1

=

h n

n n

[Vn ]h,j [Vn ]l,k Ch−l+1,1 +

h=1 l=1

=

h n

n n

[Vn ]h,j [Vn ]l,k Cn+h−l+1,1

h=1 l=h+1

[Vn ]h,j [Vn ]h−ˆl+1,k Cˆl,1 +

h=1 ˆ l=1

=

[Vn ]h,j [Vn ]l,k Ch,l

h=1 l=h+1

n n

[Vn ]h,j [Vn ]n+h−ˆl+1,k Cˆl,1

h=1 ˆ l=h+1

h n 2π(h−ˆ l)(k−1) 1 2π(h−1)(j−1) i − i n n e Cˆl,1 e n h=1 ˆ l=1

n n 2π(n+h−ˆ l)(k−1) 1 2π(h−1)(j−1) i − i n n e + e Cˆl,1 n h=1 ˆ l=h+1

=

n h 2π(h−ˆ l)(k−1) 1 2π(h−1)(j−1) i − i n n e e Cˆl,1 n

h=1 ˆ l=1

+

n n 2π(h−ˆ l)(k−1) 1 2π(h−1)(j−1) i − i −2π(k−1)i n n e e e Cˆl,1 n

h=1 ˆ l=h+1

=

h n 2π(h−ˆ l)(k−1) 1 2π(h−1)(j−1) i − i n n e Cˆl,1 e n

h=1 ˆ l=1

+

n n 2π(h−ˆ l)(k−1) 1 2π(h−1)(j−1) i − i n n e e Cˆl,1 n h=1 ˆ l=h+1

=

n n 2π(h−ˆ l)(k−1) 1 2π(h−1)(j−1) i − i n n e e Cˆl,1 n h=1 ˆ l=1

223

224


=

n n 2π((h−1)+(1−ˆ l))(k−1) 1 2π(h−1)(j−1) i − i n n e Cˆl,1 e n

h=1 ˆ l=1

=

n n 2π(ˆ l−1)(k−1) 1 2π(h−1)((j−1)−(k−1)) i i n n e Cˆl,1 e n h=1 ˆ l=1

=

n n h=1 ˆ l=1

1 = √ n

1 2π(h−1)(j−k) i n √ e [Vn ]ˆl,k Cˆl,1 n

n

e

2π(h−1)(j−k) n

i

  n   [Vn ]l,k ˆ Cˆ l,1

h=1

1 = √ n

ˆ l=1

  h−1 n n 2π(j−k) i ∗  [Vn ]k,ˆl IM Cˆl,1  e n h=1

nδj,k = √ n 

n

ˆ l=1

n √ [Vn∗ ⊗ IM ]k,ˆl Cˆl,1 = δj,k n [(Vn ⊗ IM )∗ ]k,ˆl Cˆl,1

ˆ l=1



ˆ l=1



C1,1 √  C2,1     = δj,k  n(Vn ⊗ IM )∗  .    ..  Cn,1

,

k,1

where we have used the fact that if z = 1 is an nth root of unity then

n h−1 = 0 (see, e.g., [1, p. 29]). h=1 z (2)⇒(1) For every 1 ≤ j, k ≤ n we have Cj,k = [(Vn ⊗ IM )diag(A1 , . . . , An )(Vn ⊗ IN )∗ ]j,k = [(Vn ⊗ IM )diag(A1 , . . . , An )(Vn∗ ⊗ IN )]j,k =

n n [Vn ⊗ IM ]j,l Al [Vn∗ ⊗ IN ]l,k = [Vn ]j,l IM Al ([Vn∗ ]l,k IN ) l=1

=

=

l=1

n

n

l=1

l=1

[Vn ]j,l [Vn ]k,l Al =

1 n

n l=1

e

2π(k−j)(l−1) i n

Al .

1 − 2π(j−1)(l−1) i 2π(k−1)(l−1) i n n e e Al n (5.3)

5.2 Sequence of Block Circulant Matrices Generated by a Continuous Function

225

Consequently, if (j1 − k1 ) − (j2 − k2 ) = hn with h ∈ {−1, 0, 1}, then we obtain Cj1 ,k1 = =

1 2π(k1 −j1 )(l−1) i 1 2π(k2 −j2 −hn)(l−1) i n n e Al = e Al n n 1 n

n

n

l=1

l=1

n

e

l=1

2π(k2 −j2 )(l−1) i n

1 2π(k2 −j2 )(l−1) i n e Al n n

e2πh(1−l)i Al =

l=1

= Cj2 ,k2 . When M = N = 1, the previous lemma gives an eigenvalue decomposition of a circulant matrix where the eigenvector matrix is unitary. It should be mentioned that circulant matrices and tridiagonal Toeplitz matrices are practically the only types of Toeplitz matrices for which an eigenvalue decomposition is known.1

5.2

Sequence of Block Circulant Matrices Generated by a Continuous Function

Based on Lemma 5.1, we make the following definition. Definition 5.2. Consider a matrix-valued function of a real variable F : R → CM ×N which is continuous and 2π-periodic. For every n ∈ N we define Cn (F ) as the n × n block circulant matrix with M × N blocks given by 2π(n − 1) 2π ,...,F Cn (F ) := (Vn ⊗ IM )diag F (0), F n n ∗ (Vn ⊗ IN ) . The sequence {Cn (F )} is called the sequence of block circulant matrices generated by F . 1 An

eigenvalue decomposition of a tridiagonal Toeplitz matrix can be found, e.g., in [11, 22]. In [10] what can be found is an eigenvalue decomposition of a Hermitian tridiagonal Toeplitz matrix where the eigenvector matrix is unitary.

226


The next lemma provides several properties of sequences of block circulant matrices generated by continuous matrix-valued functions. Lemma 5.2. Let F : R → CM ×N be a matrix-valued function which is continuous and 2π-periodic. Then (1) For every n ∈ N 1 2π(k−1)(l−1) i n = e F n n

[Cn (F )]1,k

l=1

2π(l − 1) , n

1 ≤ k ≤ n.

(2) σ1 (Cn (F )) ≤ σ1 (F ) for all n ∈ N. (3) {Cn (F ∗ )} = {(Cn (F ))∗ }. (4) For every matrix-valued function G: R → CM ×N which is continuous and 2π-periodic {Cn (αF + βG)} = {αCn (F ) + βCn (G)},

∀α, β ∈ C.

(5) For every matrix-valued function G: R → CN ×K which is continuous and 2π-periodic {Cn (F G)} = {Cn (F )Cn (G)}, where (F G)(ω) := F (ω)G(ω), ω ∈ R.

Proof. (1) From (5.3) this property holds. (2) Fix n ∈ N. Since the spectral norm is unitarily invariant we obtain σ1 (Cn (F ))=Cn (F )2 = diag1≤k≤n (F (ωk )) 2 1

∗ x∗ diag1≤k≤n (F (ωk )) diag1≤k≤n (F (ωk )) x 2 = max x∗ x x ∈ CnN ×1 x = 0nN ×1

= max nN ×1 x∈C x = 0nN ×1

x∗ diag1≤k≤n ((F (ωk ))∗ ) diag1≤k≤n (F (ωk )) x x∗ x

12


= max nN ×1 x∈C x = 0nN ×1

= max nN ×1 x∈C x = 0nN ×1

= max nN ×1 x∈C x = 0nN ×1

≤

sup x ∈ CnN ×1 x = 0nN ×1

≤ max nN ×1

x∗ diag1≤k≤n ((F (ωk ))∗ F (ωk )) x x∗ x

n

12 ∗ ∗ k=1 [x ]1,k (F (ωk )) F (ωk )[x]k,1 x∗ x

n

∗ ∗ k=1 [x]k,1 (F (ωk )) F (ωk )[x]k,1 x∗ x

n

2 ∗ k=1 F (ωk )2 [x]k,1 [x]k,1

n

k=1

=

2

n

max σ1 (F (ωj ))

1≤j≤n

1

x∗ x

1≤j≤n

=

2

1 max1≤j≤n F (ωj )22 [x]∗k,1 [x]k,1 2

max F (ωj )2

1

x∗ x

x∈C x = 0nN ×1

=

12

max σ1 (F (ωj ))

1≤j≤n

∗ k=1 [x]k,1 [x]k,1

max nN ×1

x∈C x = 0nN ×1

max nN ×1

x∈C x = 0nN ×1

2

x∗ x

n

∗ 2 k=1 [x ]1,k [x]k,1 ∗ x x

max

x ∈ CnN ×1 x = 0nN ×1

1

x∗ x x∗ x

1

1 2

= max σ1 (F (ωj )) ≤ σ1 (F ), 1≤j≤n

for all 1 ≤ k ≤ n. where ωk = 2π(k−1) n (3) For every n ∈ N we have

∗ (Cn (F ))∗ = (Vn ⊗ IM )diag1≤k≤n (F (ωk )) (Vn ⊗ IN )∗

∗ = (Vn ⊗ IN ) diag1≤k≤n (F (ωk )) (Vn ⊗ IM )∗ = (Vn ⊗ IN )diag1≤k≤n ((F (ωk ))∗ ) (Vn ⊗ IM )∗ = (Vn ⊗ IN )diag1≤k≤n ((F ∗ ) (ωk )) (Vn ⊗ IM )∗ = Cn (F ∗ ),

227

228


where ωk = 2π(k−1) for all 1 ≤ k ≤ n. n (4) For every n ∈ N we obtain αCn (F ) + βCn (G) = α(Vn ⊗ IM )diag1≤k≤n (F (ωk )) (Vn ⊗ IN )∗ + β(Vn ⊗ IM )diag1≤k≤n (G (ωk )) (Vn ⊗ IN )∗ = (Vn ⊗ IM )diag1≤k≤n (αF (ωk )) (Vn ⊗ IN )∗ + (Vn ⊗ IM )diag1≤k≤n (βG (ωk )) (Vn ⊗ IN )∗

= (Vn ⊗ IM ) diag1≤k≤n (αF (ωk )) + diag1≤k≤n (βG (ωk )) (Vn ⊗ IN )∗ = (Vn ⊗ IM )diag1≤k≤n (αF (ωk ) + βG (ωk )) (Vn ⊗ IN )∗ = (Vn ⊗ IM )diag1≤k≤n ((αF + βG) (ωk )) (Vn ⊗ IN )∗ = Cn (αF + βG), for all 1 ≤ k ≤ n. where ωk = 2π(k−1) n (5) Since F : R → CM ×N and G: R → CN ×K are continuous and 2πperiodic, F G: R → CM ×K is also continuous and 2π-periodic. Moreover, for every n ∈ N we have Cn (F )Cn (G) = (Vn ⊗ IM )diag1≤k≤n (F (ωk )) (Vn ⊗ IN )∗ (Vn ⊗ IN ) diag1≤k≤n (G (ωk )) (Vn ⊗ IK )∗ = (Vn ⊗ IM )diag1≤k≤n (F (ωk )) diag1≤k≤n (G (ωk )) (Vn ⊗ IK )∗ = (Vn ⊗ IM )diag1≤k≤n (F (ωk ) G (ωk )) (Vn ⊗ IK )∗ = (Vn ⊗ IM )diag1≤k≤n ((F G) (ωk )) (Vn ⊗ IK )∗ = Cn (F G), where ωk =

2π(k−1) n

for all 1 ≤ k ≤ n.

Property (1) of Lemma 5.2 shows that our definition of sequence of block circulant matrices generated by a continuous matrix-valued function (i.e., Definition 5.2) extends the definition of sequence of circulant


229

matrices generated by a function in the Wiener class given by Gray in [8, Equation (4.32)]. The following lemma deals with sequences of block circulant matrices generated by a continuous matrix-valued function which is Hermitian. Lemma 5.3. Let F : R → CN ×N , and suppose that it is continuous, 2π-periodic and Hermitian. Then (1) Cn (F ) is Hermitian for all n ∈ N. (2) For every n ∈ N inf F ≤ λk (Cn (F )) ≤ sup F,

1 ≤ k ≤ nN.

(5.4)

(3) {Cn (g(F ))} = {g(Cn (F ))} for all g ∈ C[inf F, sup F ].

Proof. (1) From property (3) of Lemma 5.2 we have (Cn (F ))∗ = Cn (F ∗ ) = Cn (F ),

∀n ∈ N.

(2) Fix n ∈ N and let F (ω) = W (ω)diag1≤j≤N (λj (F (ω))) (W (ω))−1 be an eigenvalue decomposition of the Hermitian matrix F (ω) for all ω ∈ R. Then Cn (F ) = (Vn ⊗ IN )diag1≤k≤n (F (ωk ))(Vn ⊗ IN )∗ = (Vn ⊗ IN )diag1≤k≤n W (ωk )diag1≤j≤N (λj (F (ωk )))(W (ωk ))−1 (Vn ⊗ IN )−1

= (Vn ⊗ IN )diag1≤k≤n (W (ωk ))diag1≤k≤n diag1≤j≤N (λj (F (ωk ))) diag1≤k≤n (W (ωk ))−1 (Vn ⊗ IN )−1 = (Vn ⊗ IN )diag1≤k≤n (W (ωk ))diag1≤k≤n diag1≤j≤N (λj (F (ωk ))) −1

diag1≤k≤n (W (ωk )) (Vn ⊗ IN )−1 ,

230


where ωk = 2π(k−1) for all 1 ≤ k ≤ n. Consequently, an eigenvalue n decomposition of Cn (F ) is Cn (F ) = (Vn ⊗ IN )diag1≤k≤n (W (ωk ))diag1≤k≤n diag1≤j≤N (λj (F (ωk )))

−1 . (Vn ⊗ IN )diag1≤k≤n (W (ωk ))

(5.5)

Hence, (5.4) holds. (3) From (5.5), we obtain g(Cn (F ))

= (Vn ⊗ IN )diag1≤k≤n (W (ωk ))diag1≤k≤n diag1≤j≤N (g(λj (F (ωk )))) −1 (Vn ⊗ IN )diag1≤k≤n (W (ωk ))

= (Vn ⊗ IN )diag1≤k≤n (W (ωk ))diag1≤k≤n diag1≤j≤N (g(λj (F (ωk )))) diag1≤k≤n (W (ωk ))−1 (Vn ⊗ IN )−1 = (Vn ⊗ IN )diag1≤k≤n W (ωk )diag1≤j≤N (g(λj (F (ωk )))) (W (ωk ))−1 (Vn ⊗ IN )∗ = (Vn ⊗ IN )diag1≤k≤n (g(F (ωk )))(Vn ⊗ IN )∗ = (Vn ⊗ IN )diag1≤k≤n ((g(F ))(ωk ))(Vn ⊗ IN )∗ = Cn (g(F )) for all n ∈ N and g ∈ C[inf F, sup F ]. We finish this section with a result on sequences of block circulant matrices generated by trigonometric polynomials. Lemma 5.4. Consider the M × N trigonometric polynomial F (ω) =

m

ehωi Fh ,

∀ω ∈ R,

h=−m

where m ∈ N ∪ {0}. Then Cn (F ) = circ(F0 , F−1 , . . . , F−m , 0M ×N , . . . , 0M ×N , Fm , . . . , F1 ) for all n > 2m.


231

Proof. Fix n > 2m and 1 ≤ k ≤ n. From property (1) of Lemma 5.2 we have n 1 2π(k−1)(l−1) i 2π(l − 1) n e F [Cn (F )]1,k = n n l=1

=

m n 1 2π(k−1)(l−1) i 2πh(l−1) i n e e n Fh n l=1

=

1 n

n

h=−m

m

e

2π(k−1)(l−1) i n

e

2πh(l−1) i n

Fh

l=1 h=−m

m 1 = n

n

h=−m

e

2π(k−1)(l−1) i n

e

2πh(l−1) i n

Fh

l=1

n m 1 2π(h+k−1)(l−1) i n = e Fh n h=−m l=1 n m 1 2π(h+k−1) i l−1 n = e Fh n h=−m l=1 = Fh −m ≤ h ≤ m h + k − 1 ∈ {0, n}

 F1−k , = Fn+1−k ,  0M ×N ,  F1−k , = Fn+1−k ,  0M ×N ,

if −m ≤ 1 − k ≤ 0, if 1 ≤ n + 1 − k ≤ m, otherwise, if 1 ≤ k ≤ m + 1, if n − m + 1 ≤ k ≤ n, otherwise,

where we have used the fact that if z = 1 is an nth root of unity then

n l−1 = 0. l=1 z

6 Asymptotic Behaviour of Block Toeplitz Matrices

6.1

Products and Functions of Large Block Toeplitz Matrices

In this section, we use the results of the previous sections to study the asymptotic behaviour of the block Toeplitz matrices generated by a continuous matrix-valued function. The first result of the present section was given in [13] and it states that the sequence of block Toeplitz matrices and the sequence of block circulant matrices generated by the same continuous matrix-valued function are asymptotically equivalent. Lemma 6.1. Let F : R → CM ×N be a matrix-valued function which is continuous and 2π-periodic. Then {Tn (F )} ∼ {Cn (F )}. Proof. From Theorem 4.3 and property (2) of Lemma 5.2, we obtain Tn (F )2 , Cn (F )2 ≤ σ1 (F ) < ∞, To finish the proof we need to show that lim

n→∞

Tn (F ) − Cn (F )F √ = 0. n 232

∀n ∈ N.

6.1 Products and Functions of Large Block Toeplitz Matrices

233

Let {Fm }m∈N∪{0} be the sequence of partial sums of the Fourier series of F (i.e., {[Fm ]r,s }m∈N∪{0} is the sequence of partial sums of the Fourier series of [F ]r,s for all 1 ≤ r ≤ M and 1 ≤ s ≤ N ): m

Fm (ω) =

ekωi Fk ,

m ∈ N ∪ {0}, ω ∈ R,

k=−m

where {Fk }k∈Z is the sequence of Fourier coefficients of F . Applying the Parseval theorem yields 2π 1 lim (Fm − F )(ω)2F dω m→∞ 2π 0 2π 1 Fm (ω) − F (ω)2F dω = lim m→∞ 2π 0 2π M N 1 |[Fm (ω) − F (ω)]r,s |2 dω = lim m→∞ 2π 0 r=1 s=1 2π N M 1 |[Fm (ω)]r,s − [F (ω)]r,s |2 dω = lim m→∞ 2π 0 r=1 s=1

=

N M r=1 s=1

=

M N

1 lim m→∞ 2π

2π 0

|[Fm ]r,s (ω) − [F ]r,s (ω)|2 dω

0 = 0.

r=1 s=1

Hence, given ∈ (0, ∞), there exists m0 ∈ N such that 2π 1 (Fm0 − F )(ω)2F dω 0≤ 2π 0 2π 1 2 (Fm0 − F )(ω)F dω < . = 2π 0 Thus,

1 2π

2π 0

(Fm0 −

F )(ω)2F dω

12
0,

∀n ∈ N.

k=1

As det(E(xn:1 x∗n:1 )) = 0 and the random vector xn:1 is proper and Gaussian with zero mean, from [18, Theorem 2] the differential entropy h(xn:1 ) of xn:1 is given by

h(xn:1 ) = log2 (πe)nN det (E(xn:1 x∗n:1 ))

= log2 (πe)nN + log2 det (E(xn:1 x∗n:1 )) = nN log2 (πe) + log2

nN )

λk (E(xn:1 x∗n:1 ))

k=1

= nN log2 (πe) +

nN

log2 λk (E(xn:1 x∗n:1 )),

∀n ∈ N.

k=1

Therefore, since the differential entropy rate of {xn : n ∈ Z} is defined as limn→∞ n1 h(xn:1 ), from Theorem 6.6 we obtain 1 1 h(xn:1 ) = lim N log2 (πe) + log2 λk (E(xn:1 x∗n:1 )) n→∞ n n→∞ n nN

lim

k=1

1 log2 λk (E(xn:1 x∗n:1 )) n→∞ n nN

= N log2 (πe) + lim

k=1

1 = N log2 (πe) + 2π = N log2 (πe) + = N log2 (πe) +

1 2π 1 2π

N 2π

0

log2 λk (X(ω))dω

k=1 2π

log2 0

N )

λk (X(ω))dω

k=1 2π

log2 det(X(ω))dω. 0

A Asymptotically 2-Equivalent Sequences of Matrices

In Definition 3.1, we extended the Gray concept of asymptotically equivalent sequences of matrices to sequences of non-square matrices. We present here another definition that is also an extension to sequences of non-square matrices of that concept of Gray. Definition A.1. Consider two strictly increasing sequences of natural (1) (2) (1) (2) numbers {dn } and {dn }. Let An and Bn be dn × dn matrices for all n ∈ N. We say that the sequences {An } and {Bn } are asymptotically 2-equivalent, and write {An } ∼2 {Bn }, if they satisfy (3.1) and lim

n→∞

An − Bn F = 0. (1) (2) max dn , dn

Definition A.1 was introduced in [12, Appendix] for the case in which (1) (2) {dn } = {dn }. The following lemma relates Definition A.1 with Definition 3.1.

249

250

Asymptotically 2-Equivalent Sequences of Matrices

Lemma A.1. Consider two strictly increasing sequences of natural (1) (2) (1) (2) numbers {dn } and {dn }. Let An and Bn be dn × dn matrices for all n ∈ N. If {An } ∼ {Bn } then {An } ∼2 {Bn }. (1)

(2)

Proof. Since {dn } and {dn } are strictly increasing sequences of nat(1) (2) ural numbers, they satisfy dn , dn ≥ n for all n ∈ N. Consequently, 0≤

An − Bn F An − Bn F √ → 0. ≤ n (1) (2) max dn , dn

As we will now prove Lemma 3.2 is not true when ∼ is replaced by ∼2 , and this is the reason why we have used ∼ instead of ∼2 in the present monograph. Observe that if {An } = {(In |0n×(n2 −n) )}, {Bn } = {0n×n2 }, {Cn } = {A n }, and {Dn } = {Bn }, then = {An 2 } = {1} {Cn 2 } = A n 2

and

{Dn 2 } = Bn = {Bn 2 } = {0}. 2

Moreover,

An − Bn (An − Bn ) Cn − Dn F F F √ √ √ = lim = lim lim 2 n→∞ n→∞ n→∞ n2 n2 n

In |0n×(n2 −n) An − Bn F F √ √ = lim = lim n→∞ n→∞ n2 n2 √ n 1 = lim √ = lim √ = 0 n→∞ n2 n→∞ n and An Cn − Bn Dn F In − 0n×n F √ √ = lim lim n→∞ n→∞ n n √ In F n = lim √ = 1 = 0. = lim √ n→∞ n→∞ n n

Therefore, {An } ∼2 {Bn } and {Cn } ∼2 {Dn }, but the two sequences of matrices {An Cn } and {Bn Dn } are not asymptotically 2-equivalent.

B The Wiener Class

We present here the Wiener class in detail. Definition B.1. A function in the Wiener class is a function of the form ∞ m kωi kωi e Ak := lim e [Ak ]r,s , (B.1) F (ω) = m→∞

k=−∞

k=−m

1≤r≤M,1≤s≤N

where ω ∈ R, Ak ∈ CM ×N with k ∈ Z, and ∞

|[Ak ]r,s | < ∞

k=−∞

for all 1 ≤ r ≤ M and 1 ≤ s ≤ N .

If 1 ≤ r ≤ M and 1 ≤ s ≤ N , { m k=−m |[Ak ]r,s |} is a Cauchy sequence because it is convergent. Therefore, for every ∈ (0, ∞) there exists m0 ∈ N such that m n |[Ak ]r,s | − |[Ak ]r,s | < k=−m

k=−n

251

252

The Wiener Class

for all m, n ≥ m0 . Since m n kωi kωi e [A ] − e [A ] k r,s k r,s k=−m k=−n − min{m,n} max{m,n} kωi kωi e [Ak ]r,s + e [Ak ]r,s = k=− max{m,n} k=min{m,n} − min{m,n}

≤

kωi e [Ak ]r,s +

k=− max{m,n}

kωi e [Ak ]r,s

k=min{m,n}

− min{m,n}

=

max{m,n}

max{m,n}

|[Ak ]r,s | +

k=− max{m,n}

|[Ak ]r,s |

k=min{m,n}

− min{m,n} max{m,n} |[Ak ]r,s | + |[Ak ]r,s | = k=− max{m,n} k=min{m,n} m n |[Ak ]r,s | − |[Ak ]r,s | < , = k=−m

k=−n

for all m, n ≥ m0 and ω ∈ R, from the Cauchy criterion for uniform convergence (see, e.g., [20, p. 147]) we obtain that the sequence

kωi [A ] } converges uniformly on R. Hence, of functions { m k r,s k=−m e

m kωi { k=−m e [Ak ]r,s } converges pointwise on R to [F ]r,s . Thus, F is well defined and F (ω) ∈ CM ×N for all ω ∈ R. The following two lemmas provide several fundamental properties of the functions in the Wiener class. Lemma B.1. Let F : R → CM ×N be the function given by (B.1). Then, (1) F is continuous. (2) F is 2π-periodic. (3) {Ak }k∈Z is the sequence of Fourier coefficients of F .

253 Proof. (1) If 1 ≤ r ≤ M and 1 ≤ s ≤ N , [F ]r,s is continuous on R

kωi [A ] } conbecause the sequence of continuous functions { m k r,s k=−m e verges uniformly on R to [F ]r,s . (2) If 1 ≤ r ≤ M and 1 ≤ s ≤ N then [F ]r,s is 2π-periodic since m

[F ]r,s (ω + 2π) = lim

m→∞

= lim

m→∞

k=−m m

ek(ω+2π)i [Ak ]r,s ekωi [Ak ]r,s = [F ]r,s (ω),

∀ω ∈ R.

k=−m

kωi [A ] } converges (3) Fix 1 ≤ r ≤ M and 1 ≤ s ≤ N . Since { m k r,s k=−m e uniformly on R to [F ]r,s , for every ∈ (0, ∞) there exists m0 ∈ N such that m ekωi [Ak ]r,s − [F ]r,s (ω) < k=−m

for all m ≥ m0 and ω ∈ R. Consequently, m −hωi kωi −hωi e [Ak ]r,s − e [F ]r,s (ω) e k=−m m ekωi [Ak ]r,s − [F ]r,s (ω) = e−hωi k=−m m ekωi [Ak ]r,s − [F ]r,s (ω) < , = k=−m

kωi [A ] } confor all m ≥ m0 , ω ∈ R and h ∈ Z. Thus, {e−hωi m k r,s k=−m e verges uniformly on R to e−hωi [F ]r,s (ω) for all h ∈ Z. Hence, applying a well known result on uniform convergence and integration (see, e.g., [20, Theorem 7.16]) yields 2π 2π m 1 1 lim e−hωi [F ]r,s (ω)dω = e−hωi ekωi [Ak ]r,s dω 2π 0 2π m→∞ 0 k=−m 2π m 1 e(k−h)ωi [Ak ]r,s dω = lim m→∞ 2π 0 k=−m

254

The Wiener Class m

= lim

m→∞

= lim

m→∞

[Ak ]r,s

k=−m m

1 2π

2π

e(k−h)ωi dω 0

[Ak ]r,s δk,h = [Ah ]r,s

∀h ∈ Z,

k=−m

where δk,h is the Kronecker delta. Lemma B.2. Let {Fk }k∈Z be the sequence of Fourier coefficients of a matrix-valued function F : R → CM ×N which is continuous and 2πperiodic. F is in the Wiener class if and only if ∞

|[Fk ]r,s | < ∞,

1 ≤ r ≤ M, 1 ≤ s ≤ N.

(B.2)

k=−∞

Proof. From Lemma B.1, if F is in the Wiener class then (B.2) holds. We now proceed to show that the reciprocal is also true. We consider the function in the Wiener class given by G(ω) =

∞

ekωi Fk ,

∀ω ∈ R.

k=−∞

From Lemma B.1 G: R → CM ×N is continuous, 2π-periodic, and {Fk }k∈Z is its sequence of Fourier coefficients. Consequently, F = G because F, G: R → CM ×N are two continuous 2π-periodic functions with the same sequence of Fourier coefficients.

Acknowledgments

The authors thank Dr. Adam Podhorski and the two anonymous reviewers for their helpful comments. This work was supported in part by the Basque Government through the INFOLIMITS project (PI2012-10), and by the Spanish Ministry of Science and Innovation through the projects COSIMA (TEC2010-19545-C04-02) and COMONSENS (CSD2008-00010).

255

References

[1] T. M. Apostol, Mathematical Analysis. Addison-Wesley, 1974. [2] D. S. Bernstein, Matrix Mathematics: Theory, Facts, and Formulas. Princeton University Press, 2009. [3] A. B¨ ottcher, J. Gutiérrez-Gutiérrez, and P. M. Crespo, “Mass concentration in quasicommutators of Toeplitz matrices,” Journal of Computational and Applied Mathematics, vol. 205, pp. 129–148, 2007. [4] W. E. Boyce and R. C. DiPrima, Elementary Differential Equations and Boundary Value Problems. Wiley, 2005. [5] P. M. Crespo and J. Gutiérrez-Gutiérrez, “On the elementwise convergence of continuous functions of Hermitian banded Toeplitz matrices,” IEEE Transactions on Information Theory, vol. 53, no. 3, pp. 1168–1176, March 2007. [6] H. Gazzah, P. A. Regalia, and J. P. Delmas, “Asymptotic eigenvalue distribution of block Toeplitz matrices and application to blind SIMO channel identification,” IEEE Transactions on Information Theory, vol. 47, no. 3, pp. 1243–1251, March 2001. [7] R. M. Gray, “On the asymptotic eigenvalue distribution of Toeplitz matrices,” IEEE Transactions on Information Theory, vol. IT-18, no. 6, pp. 725–730, November 1972. [8] R. M. Gray, “Toeplitz and circulant matrices: A review,” Foundations and Trends in Communications and Information Theory, vol. 2, no. 3, pp. 155–239, 2006. [9] U. Grenander and G. Szeg¨ o, Toeplitz Forms and their Applications. AMS Chelsea Publishing, 1984. [10] J. Gutiérrez-Gutiérrez, “Positive integer powers of certain tridiagonal matrices,” Applied Mathematics and Computation, vol. 202, pp. 133–140, 2008. 256

References

257

[11] J. Gutiérrez-Gutiérrez, “Powers of tridiagonal matrices with constant diagonals,” Applied Mathematics and Computation, vol. 206, pp. 885–891, 2008. [12] J. Gutiérrez-Gutiérrez and P. M. Crespo, “Asymptotically equivalent sequences of matrices and Hermitian block Toeplitz matrices with continuous symbols: Applications to MIMO systems,” IEEE Transactions on Information Theory, vol. 54, no. 12, pp. 5671–5680, December 2008. [13] J. Gutiérrez-Gutiérrez and P. M. Crespo, “Asymptotically equivalent sequences of matrices and multivariate ARMA processes,” IEEE Transactions on Information Theory, vol. 57, no. 8, pp. 5444–5454, August 2011. [14] J. Gutiérrez-Gutiérrez, P. M. Crespo, and A. B¨ ottcher, “Functions of banded Hermitian block Toeplitz matrices in signal processing,” Linear Algebra and its Applications, vol. 422, pp. 788–807, 2007. [15] J. Gutiérrez-Gutiérrez, I. Iglesias, and A. Podhorski, “Geometric MMSE for one-sided and two-sided vector linear predictors: From the finite-length case to the infinite-length case,” Signal Processing, vol. 91, pp. 2237–2245, 2011. [16] R. A. Horn and C. R. Johnson, Topics in Matrix Analysis. Cambridge University Press, 1994. [17] T. Kailath, Linear Systems. Prentice-Hall, 1980. [18] F. D. Neeser and J. L. Massey, “Proper complex random processes with applications to information theory,” IEEE Transactions on Information Theory, vol. 39, no. 4, pp. 1293–1302, July 1993. [19] D. Ram´ırez, J. V´ıa, I. Santamar´ıa, and L. L. Scharf, “Detection of spatially correlated Gaussian time series,” IEEE Transactions on Signal Processing, vol. 58, no. 10, pp. 5006–5015, October 2010. [20] W. Rudin, Principles of Mathematical Analysis. McGraw-Hill, 1976. [21] P. Tilli, “A note on the spectral distribution of Toeplitz matrices,” Linear and Multilinear Algebra, vol. 45, pp. 147–159, 1998. [22] W. F. Trench, “On the eigenvalue problem for Toeplitz band matrices,” Linear Algebra and its Applications, vol. 64, pp. 199–214, 1985. [23] S. Verd´ u, “The capacity region of the symbol-asynchronous Gaussian multipleaccess channel,” IEEE Transactions on Information Theory, vol. 35, no. 4, pp. 733–751, July 1989.