Commutative orthogonal block structure and error orthogonal models

Electronic Journal of Linear Algebra Volume 25 ELA Volume 25 (2012)

Article 11

2012

Commutative orthogonal block structure and error orthogonal models Francisco Carvalho [email protected]

Joao T. Mexia Carla Santos

Follow this and additional works at: http://repository.uwyo.edu/ela Recommended Citation Carvalho, Francisco; Mexia, Joao T.; and Santos, Carla. (2012), "Commutative orthogonal block structure and error orthogonal models", Electronic Journal of Linear Algebra, Volume 25. DOI: http://dx.doi.org/10.13001/1081-3810.1601

This Article is brought to you for free and open access by Wyoming Scholars Repository. It has been accepted for inclusion in Electronic Journal of Linear Algebra by an authorized administrator of Wyoming Scholars Repository. For more information, please contact [email protected].

Electronic Journal of Linear Algebra ISSN 1081-3810 A publication of the International Linear Algebra Society Volume 25, pp. 119-128, September 2013

ELA

http://math.technion.ac.il/iic/ela

COMMUTATIVE ORTHOGONAL BLOCK STRUCTURE AND ERROR ORTHOGONAL MODELS∗ ˜ T. MEXIA‡ , AND CARLA SANTOS§ FRANCISCO CARVALHO† , JOAO

Abstract. A model has orthogonal block structure, OBS, if it has variance-covariance matrix that is a linear combination of known pairwise orthogonal orthogonal projection matrices that sum to the identity matrix. These models were introduced by Nelder is 1965, and continue to play an important part in randomized block designs. Two important types of OBS are related, and necessary and sufficient conditions for model of one type belonging to the other are determined. The first type, is that of models with commutative orthogonal block structure in which T, the orthogonal projection matrix on the space spanned by the mean vector, commutes with the orthogonal projection matrices in the expression of the variance-covariance matrix. The second type, is that of error orthogonal models. These results open the possibility of deepening the study of the important class of models with OBS.

Key words. COBS, Error orthogonal models, Commutative Jordan algebras

AMS subject classifications. 15A15, 15F10.

1. Introduction. Models with orthogonal block structure, OBS, have the family

V

 ◦ m X 

j=1

γj◦ Q◦j

 

= {V(γ ◦ )}



∗ Received

by the editors on November 30, 2011. Accepted for publication on March 6, 2013 Handling Editor: Simo Puntanen. † Unidade Departamental de Matem´ atica e F´ısica, Instituto Polit´ ecnico de Tomar, 2300-313 Tomar, Portugal ([email protected]) CMA - Centro de Matem´ atica e Aplica¸co ˜es, Universidade Nova de Lisboa, 2829-516 Caparica, Portugal. ‡ Departamento de Matem´ atica, Faculdade de Ciˆ encias e Tecnologia, Universidade Nova de Lisboa, 2829-516 Caparica, Portugal. CMA - Centro de Matem´ atica e Aplica¸co ˜es, Universidade Nova de Lisboa, 2829-516 Caparica, Portugal. § Departmento de Matem´ atica e Ciˆ encias F´ısicas, Instituto Polit´ ecnico de Beja, Rua Pedro Soares, 7800-295 Beja, Portugal. 119


ELA


120

F. Carvalho, J.T. Mexia, and C. Santos

of variance-covariance matrices V(γ ◦ ) =

◦ m P

j=1

γj◦ Q◦j . In these linear combinations,

◦ ◦ ◦ γ1◦ , . . . , γm ◦ , are nonnegative and the matrices Q1 , . . . , Qm◦ are known pairwise orthogonal orthogonal projection matrices, POOPM, such that ◦

m X

Q◦j = In .

j=1

These models were introduced by [10, 11] and continue to play an important part in the theory of randomized block designs, see e.g. [1, 2]. Our work will be centered on two classes of models: 1. Models with Commutative Orthogonal Block Structure, COBS. These are OBS models where T, the orthogonal projection matrix, OPM, on the range space Ω, spanned by the mean vectors, commutes with POOPM Q◦1 , . . . , Q◦m◦ . These models were introduced in [7] and further studied in [3]. As we shall see, the least square estimators, LSE, will be uniformly best linear unbiased estimators, UBLUE, i.e., they will be best linear unbiased estimators, BLUE, whatever the variance models; 2. Error orthogonal models, EO. These are models whose LSE are UBLUE, and with Tc = In − T, the submodel y′ = Tc y is OBS with variance-covariance matrix ′

′

V (γ) =

m X

γj′ Q′j

j=1

Q′1 , . . . , Q′m′

where the matrices are known POOPM and moreover, the pa′ rameter space, Γ′ , of the variance components γ ′ = (γ1′ , . . . , γm ′ ), contains a non-empty open set. These models were introduced in [17, 18]. To distinguish this reinforced version of OBS from the usual one, we will call it robust orthogonal block structure, ROBS. In our study, we will use commutative Jordan algebras of symmetric matrices, CJAS, which we will consider in the next section. We then have a section on COBS and one in which we relate both classes of models, giving necessary and sufficient conditions for a EO to be COBS and for a COBS to be EO. 2. Commutative Jordan Algebras. For our purposes, CJAS will be linear spaces of symmetric matrices that commute and are closed under taking squares. These structures were introduced by [8] in a reformulation of Quantum Mechanics. Later on Seely, see [13, 14], rediscovered these structures and used them to carry out linear statistical inference. This rediscovery initiated a very productive line of work, see for instance [15, 16], [4], [17, 18], [5, 6, 7], [3], [9], etc.


ELA


Commutative orthogonal block structure and error orthogonal models

121

Now, the symmetric matrices M1 , . . . , Mv commute if and only if they are jointly diagonalized by the same orthogonal matrix P, see [12]. Thus the set M = {M1 , . . . , Mv } will be contained in V (P), the family of matrices diagonalized by P. Since V (P) is itself a CJAS, we see that a family of symmetric matrices is contained in a CJAS if and only if its matrices commute. Since the intersection of CJAS is a CJAS, if the matrices in M commute, there will be a minimum CJAS A (M ) containing M , the CJAS generated by M . Let us now establish the following proposition: Proposition 2.1. A symmetric matrix U commutes with the matrices of a family M = {M1 , . . . , Mv } of symmetric matrices that commute, if and only if it commutes with all the matrices of the CJAS generated by that family, A (M ). Proof. Since M ⊂ A (M ), if U commutes with the matrices of A (M ), it will commute with M1 , . . . , Mv . Conversely, if the matrices in U = {U, M1 , . . . , Mv } commute we will have a CJAS generated by U , A (U ), that, since M ⊂ U , contains A (M ) as well as U, so U will commute with all the matrices in A (M ). Of importance when considering CJAS is the fact, shown in [15], that any CJAS, A , has an unique basis constituted by POOPM. This will be the principal basis of that CJAS, pb(A ). From the existence of this principal basis it follows that any CJAS contains the products of its matrices and not only their squares. Moreover, if Q is an OPM belonging to a CJAS A with principal basis pb = {Q1 , . . . , Qm }, we will have Q=

m X

aj Qj

j=1

and, since Q is idempotent and the Q1 , . . . , Qm are idempotent and pairwise orthogonal, we must have aj = 0 or aj = 1, j = 1, . . . , m. Thus the OPM’s belonging to a CJAS are in fact sums of all or part of the matrices in the principal basis of the CJAS. m P Qj = In , the CJAS with principal basis pb(A ) = {Q1 , . . . , Qm } will When j=1

be complete. It is now clear that any family of POOPM will be the principal basis of the CJAS constituted by the linear combinations of the matrices in the family. If we consider models with OBS, the Q◦1 , . . . , Q◦m◦ will constitute the principal basis of a complete CJAS, A ◦ . It’s also important to point out that for a CJAS to contain invertible matrices, that CJAS must be complete.


ELA


122


If we consider an n × n matrix G belonging to a complete CJAS A , it’s clear that In and Gc = In − G, the complement of G, will also belong to A as well as GU and Gc U, whatever the matrix U of A . We may now establish the following proposition. Proposition 2.2. Let Q be an orthogonal projection matrix that commutes with the matrices of a complete CJAS A ◦ . If pb(A ◦ ) = {Q◦1 , . . . , Q◦m◦ }, the principal basis of the CJAS generated by Q, the matrices of A ◦ is constituted by the non-null products QQ◦j and Qc Q◦j , j = 1, . . . , m◦ . Proof. Let Q1 , . . . , Qm be the non-null products QQ◦j and Qc Q◦j , j = 1, . . . , m◦ . It is easy to see that Q1 , . . . , Qm are POOPM and therefore constitute the principal basis of a CJAS A that contains Q and the matrices of A ◦ , since it contains the Q◦1 , . . . , Q◦m◦ . To complete the proof we have only to point out that any CJAS containing both Q and the matrices of A ◦ contains the Q1 , . . . , Qm , so it will contain A. 3. Models with commutative orthogonal block structure. As stated in the introduction, this section is focussed on COBS and related models. We will consider a mixed model with n observations written in its usual form, y=

w X

Xi β i ,

i=0

where β 0 is fixed and the β1 , . . . , β w are random, independent, with null mean vectors 2 Icw , having mean vector and varianceand variance-covariance matrices σ12 Ic1 , . . . , σw covariance matrix  µ = X0 β 0    0 w P    V= σi2 Mi , i=1

2 2 where Mi = Xi X⊤ i , i = 1, . . . , w. The σ1 , . . . , σw will be the variance components, while the OPM on Ω will be given by

† † X⊤ X T = X0 X⊤ 0 0 = X0 X0 , 0 where A† denotes the Moore-Penrose inverse of matrix A. The range space of U is denoted by R(U) R

Proposition 3.1. If the matrices M1 , . . . , Mw commute and = n , the model is OBS. X1 · · · Xw

R


ELA



123

Proof. Let Q ◦ = {Q◦1 , . . . , Q◦m◦ } be the principal basis of the CJAS, A ◦ , gener◦ m P ◦ ◦ bi,j Qj , i = 1, . . . , w, and so the varianceated by the M1 , . . . , Mw , then Mi = j=1

covariance matrix can be expressed as the linear combination   ◦ m◦ m w X X X ◦ ◦ 2 γj◦ Q◦j , bi,j Qj = σi V= with γj◦ =

w P

i=1

b◦i,j σi2 , j = 1, . . . , m◦ . Moreover, since

R

w X

Mi

i=1

w P

j=1

j=1

i=1

!

=R

···

X1

Xw

=

Rn,

Mi is an invertible matrix belonging to A ◦ which will be complete.

i=1

R

We are now able to establish the following proposition. Proposition 3.2. If the matrices T, M1 , . . . , Mw commute and = n , the model is COBS. X1 · · · Xw

R

Proof. According to Proposition 3.1, the model is OBS. Moreover, from Proposition 2.1 it follows that T will commute with the Q◦1 , . . . , Q◦m◦ , so the model will be COBS. The principal basis of the CJAS A generated by T and the matrices Q◦1 , . . . , Q◦m◦ will be constituted by the {Q1 , . . . , Qm }, these are the non-null products TQ◦j and Tc Q◦j , j = 1, . . . , m◦ . Moreover, T will be the sum of matrices in pb(A ) which can be reordered in such a way that T=

z X

Qj .

j=1

We also will have Mi =

m X

j = 1, . . . , w,

bi,j Qj

j=1

so the variance-covariance matrix is written as V=

m X

γj Qj ,

j=1

with γj =

w P

i=1

bi,j σi2 , j = 1, . . . , m. These will be the canonical variance components.


ELA


124


According to the last expressions let us take the vectors     γz+1 γ1   .   .  γ(1) =  ..  ; γ(2) =  ..  ; σ 2 =  γm γz 

 σ12 ..  .  2 σw

and the matrix B = [bi,j ] that can be partitioned as B=

B(1) B(2)

,

where the sub-matrix B(1) has z columns and the sub-matrix B(2) has m−z columns, and we can express the canonical variance components vectors as γ(ℓ) = B(ℓ)⊤ σ 2 ,

ℓ ∈ {1, 2} .

We now point out that, due to R(Qj ) ⊂ Ω, j = 1, . . . , z, only the γz+1 , . . . , γm are directly estimable. But if the row vectors of B(2), which are the columns vectors of B(2)⊤ , are linearly independent, we will have  2 ⊤ +  γ(2)  σ = B(2)   γ(1) = B(1)⊤ B(2)⊤ + γ(2),

so all variance components, either usual or canonic, will be estimable. This condition is not a very restrictive condition in most situations. In this case the relevant parameters for the random effects part determine each other. Thus that part segregates as a sub-model and we say there is segregation. Another interesting case is when B(1) is a sub-matrix of B(2). Thus γ(1) is a sub-vector of γ(2) and its components are directly estimable. This case is called matching since it is based on the matching of columns of B(1) and B(2). Now, in COBS, matrices T and V commute whatever the variance components, which, see Theorem 2 in [19], is a necessary and sufficient condition for LSE to be UBLUE. A similar result can also be found in [20]. 4. Relating the models. To lighten the writing, we name as UBLUE the models whose LSE are UBLUE. Since EO and COBS are UBLUE, we start by considering the wider class of models. We put Θ = σ 2 ; σ 2 ≥ 0 and V = V σ 2 ; σ 2 ∈ Θ , with w X σi2 Mi . V σ2 = i=1


ELA


125

Commutative orthogonal block structure and error orthogonal models 2

With δ i the vectors whose components are zero but the i-th, which is 1, if σ ◦ ∈ Θ, 2 we have σ ◦ + δ i ∈ Θ, i = 1, . . . , w. Now, if the models is UBLUE, we have, see again Theorem 2 in [19], TV = TV σ 2 ; σ 2 ∈ Θ = V σ 2 T; σ 2 ∈ Θ = V T, 2 2 so, since Mi = V σ ◦ + δ i − V σ ◦ , i = 1, . . . , w, we have 2 2 2 T = Mi T TMi = T V σ ◦ + δ i − V σ◦2 = V σ ◦ + δ i − V σ ◦

for i = 1, . . . , w.

Moreover, with Tc = In − T, we also have Tc Mi = Mi Tc ,

i = 1, . . . , w,

so, with M◦i = TMi T and M′i = Tc Mi Tc , i = 1, . . . , w, it is easy to see that Mi = M◦i + M′i ,

i = 1, . . . , w,

as well as Mi Mℓ = M◦i M◦ℓ + M′i M′ℓ ,

i = 1, . . . , w, ℓ = 1, . . . , w,

since M◦i M′ℓ = M′ℓ M◦i = 0n×n , i = 1, . . . , w, ℓ = 1, . . . , w. Besides, these matrices M1 , . . . , Mw commute if and only if the matrices ′ M◦1 , . . . , M◦w M1 , . . . , M′w

commute. Thus, M = {M1 , . . . , Mw } will be commutative when and only when M ◦ = {M◦1 , . . . , M◦w } and M ′ = M′1 , . . . , M′w are commutative. As we saw, when M is commutative, it generates a CJAS, and likewise, when M ◦ [M ′ ] are commutative, they generate a CJAS D ◦ [D ′ ], with principal basis Q ◦ = {Q◦1 , . . . , Q◦m◦ } ′ ′ Q = Q1 , . . . , Q′m′ . Now let the model be EO and put V ′ = V′ σ 2 ; σ 2 ∈ Θ , with w X σi2 M′i . V′ σ 2 = i=1

As we saw, the variance-covariance matrices V′ σ 2 belong, when the model is EO, to a CJAS, A ′ , with principal basis Q′1 , . . . , Q′m′ . Thus, with σ 2 ∈ Θ, we have, 2 2 M′i = V′ σ ◦ + δ i − V′ σ ◦ ∈ A ′ , i = 1, . . . , w,


ELA


126


and so ′

M′i

=

m X

b′i,j Q′j ,

i = 1, . . . , w.

j=1

Thus m′ 2 X ◦ γj′ Q′j , = V σ ′

j=1

with γj′ = B′⊤ σ 2 , where B′ = b′i,j . Now Γ′ = γ ′ : σ 2 ∈ Θ contains non-empty open sets if and only ′⊤ ′ if there are no linear restrictions on the γ1′ , . . . , γm and, ′ , which is equivalent to B ′ consequently, B being invertible. Thus a necessary and sufficient condition for the model y′ = Tc y enjoying ROBS is that matrix B′ is invertible. We now have the following proposition. Proposition 4.1. A UBLUE is EO if and only if its family M ′ is commutative and its matrix B′ is invertible. Proof. As we saw, if a model is EO, it is UMVUE and has a commutative M ′ matrix family and the corresponding matrix B′ is invertible. Besides this, if the model is UMVUE and has a commutative matrix family M ′ , we will have ′

M′i

=

m X

b′i,j Q′j ,

i=1

with

Q′1 , . . . , Q′m′

, the principal basis of A ′ = A (M ′ ), so that we will have ′

′

V (γ ) =

m X

γj′ Q′j ,

j=1

with γ ′j = B′⊤ σ 2 . To complete the proof, we have only to point out that if B′ is invertible, there will be no linear restrictions on the components of γ ′ and so the corresponding parameter space will contain nonempty open sets. Corollary 4.2. A COBS is EO if and only if its matrix B(2) is invertible. Proof. A COBS is EO and its family M is commutative, so its family M ′ will be commutative. The rest of the proof follows from Proposition 4.1, since we now have B′ = B(2).


ELA



127

We now establish the following proposition, Proposition 4.3. An EO is COBS, if and only if its family M ◦ is commutative. Proof. Since an EO is UBLUE with commutative family M ′ , it will be UBLUE with commutative family M if and only if its family M ◦ is commutative. The rest of the proof is straightforward.

Acknowledgment. We deeply thank the referee for insightful comments, namely those on the distinction between the usual OBS concept and the one used by VanLeeuwen et al., which led to significant enhancement of this work. This work was partially supported by CMA/FCT/UNL, under the project PEstOE/MAT/UI0297/2011. REFERENCES

[1] Cali´ nski, T. and Kageyama, S. Block Designs. A Randomization Approach. Volume I: Analysis. Lecture Notes in Statistics. Springer, 2000. [2] Cali´ nski, T. and Kageyama, S. Block Designs. A Randomization Approach. Volume II: Design. Lecture Notes in Statistics. Springer, 2003. [3] Carvalho, F.; Mexia, J. T. and Oliveira, M. Estimation in models with commutative orthogonal block structure. Journal of Statistical Theory and Practice, 3 2, 523–533, 2009. [4] Drygas, H. and Zmy´slony, R. Jordan algebras and bayesian quadratic estimation of variance components. Linear Algebra and its Applications, 259–275, 1992. [5] Fonseca M., Mexia J.T. and Zmy´slony R. Estimators and Tests for Variance Components in Cross Nested Orthogonal Models. Discussiones Mathematicae–Probability and Statistics 23, 173–201, 2003. [6] Fonseca, M.; Mexia, J. T.; and Zmy´slony, R. Binary Operation on Jordan algebras and orthogonal normal models. Linear Algebra And Its Applications 417, 75–86, 2006. [7] Fonseca, M.; Mexia, J. T. and Zmy´slony, R. Inference in normal models with commutative orthogonal block structure. Acta et Commentationes Universitatis Tartunesis de Mathematica, 3–16, 2008. [8] Jordan, P.; von Neumann, J. and Wigner, E. On the algebraic generalization of the quantum mechanical formalism. Ann. of Math. 36, 26–64, 1934. [9] Mexia, J.T.; Vaquinhas, R.; Fonseca, M. and Zmy´slony, R. COBS: segregation, matching, crossing and nesting. Latest Trends and Applied Mathematics, simulation, Modelling, 4-th International Conference on Applied Mathematics, Simulation, Modelling (ASM’10), 249– 255, 2010. [10] Nelder, J.A. The Analysis of Randomized Experiments with Orthogonal Block Structure. I - Block Structure and the Null Analysis of Variance. Proceedings of the Royal Society of London. Series A, Mathematical and Physical Sciences, 283, 1393, 147–162, 1965. [11] Nelder, J.A. The Analysis of Randomized Experiments with Orthogonal Block Structure. II Treatment, Structure and the General Analysis of Variance. Proceedings of the Royal Society of London. Series A, Mathematical and Physical Sciences, 283, 1393, 163–178, 1965. [12] Schott, James R. Matrix Analysis for Statistics. Wiley Series in Probability and Statistics, 1997.


ELA


128


[13] Seely, J. Linear spaces and unbiased estimators. The Annals of Mathematical Statistics 41, 1735–1745, 1970. [14] Seely, J. Linear spaces and unbiased estimators. Application to a mixed linear model. The Annals of Mathematical Statistics 41, 1735–1745, 1970. [15] Seely, J. Quadratic subspaces and completeness. The Annals of Mathematical Statistics 42, 710–721, 1971. [16] Seely, J. Zyskind Linear spaces and minimum variance estimators. The Annals of Mathematical Statistics 42, 691–703, 1971. [17] VanLeeuwen, Dawn M.; Birks, David S.; Seely, Justus F.; J. Mills, J.; Greenwood, J. A. and Jones, C. W. Sufficient conditions for orthogonal designs in mixed linear models. Journal of Statistical Planning and Inference 73, 373–389, 1998. [18] VanLeeuwen, Dawn M.; Birks, David S. and Seely, Justus F. Balance and Orthogonality in Designs for Mixed Classification Models. The Annals of Statistics 27, 1927–1947, 1999. [19] Zyskind, George. On canonical forms, non-negative covariance matrices and best and simple least squares linear estimators in linear models. The Annals of Mathematical Statistics 38 (4), 1092–1109, 1967. [20] Zmy´slony, R. A characterization of Best Linear Unbiased Estimators in the general linear model. Lecture Notes in Statistics 2, 365–373, 1978.