Information Geometric Similarity Measurement for

0 downloads 0 Views 430KB Size Report
Keywords: multi-symbol sequences, texture comparison, search, random, informa- tion geometry, gamma models, stochastic process. 1 Introduction to gamma ...
Information Geometric Similarity Measurement for Stochastic Symbol Sequences C.T.J. Dodson Department of Mathematics University of Manchester Institute of Science and Technology Manchester M60 1QD, UK

J. Scharcanski Departamento de Informatica Aplicada Instituto de Informatica Universidade Federal do Rio Grande do Sul Porto Alegre, RS, Brasil Preprint date: April 29, 2002

Abstract We outline the information-theoretic differential geometry of gamma distributions, which contain exponential distributions as a special case, and log-gamma distributions. Our arguments support the opinion that these distributions have a natural role in representing departures from randomness, uniformity and Gaussian behaviour in stochastic processes. We show also how the information geometry provides a surprisingly tractable Riemannian manifold and product spaces thereof, on which may be represented the evolution of a stochastic process or the comparison of different processes, by means of well-founded maximum likelihood parameter estimation. Our model incorporates possible correlations among parameters. We discuss applications and provide some data from amino acid self-clustering in protein sequences; we provide also some results from simulations for multi-symbol sequences.

Keywords: multi-symbol sequences, texture comparison, search, random, information geometry, gamma models, stochastic process

1

Introduction to gamma models and their geometry

Elsewhere we have discussed the differential geometry of manifolds of gamma distributions and their application to clustering problems and security testing [2, 7, 3, 4, 9]. The family of gamma distributions with event space Ω = R+ , parameters τ, ν ∈ R+ has probability density functions given by  ν ν tν−1 f (t; τ, ν) = e−tν/τ (1.1) τ Γ(ν) Then < t >= τ √is the mean and V ar(t) = τ 2 /ν is the variance, so the coefficient of variation p V ar(t)/τ = 1/ ν is independent of the mean. In fact, this latter property actually characterizes gamma distributions as shown recently by Hwang and Hu [13]. They proved, for n ≥ 3 independent positive random variables x1 , x2 , . . . , xn with a common continuous probability density function h, that having independence of the sample mean x ¯ and sample coefficient of variation cv = S/¯ x is equivalent to h being a gamma distribution. The special case ν = 1 corresponds to the situation of the random or Poisson process with mean inter-event interval τ. In fact, for integer ν = 1, 2, . . . , (1.1) models a process that is Poisson but with intermediate events removed to leave only every ν th . Formally, the gamma distribution is the ν-fold convolution of the exponential distribution, called also the Pearson Type III distribution. 1

2

Information Geometric Similarity Measurement for Near-Random Stochastic Processes

1.6

ν = 0.5 (Clustered) f (t; 1, ν) 1.4 1.2 1 ν = 1 (Random) 0.8

ν = 2 (Smoothed) ν = 5 (Smoothed)

0.6 0.4 0.2 0.25

0.5

0.75

1

1.25

1.5

1.75

2

Inter-event interval t Figure 1: Probability density functions, f (t; τ, ν), for gamma distributions of inter-event intervals t with unit mean τ = 1, and ν = 0.5, 1, 2, 5. The case ν = 1 corresponds to an exponential distribution from an underlying Poisson process; ν 6= 1 represents some organization—clustering or smoothing. Thus, gamma distributions can model a range of stochastic processes corresponding to nonindependent clustered events, for ν < 1, and smoothed events, for ν > 1, as well as the random case. Figure 1 shows sample gamma distributions, all of unit mean, with ν = 0.5, 1, 2, 5. Shannon’s information theoretic entropy or ‘uncertainty’ is given, up to a factor, by the negative of the expectation of the logarithm of the probability density function (1.1), that is Z ∞ τ Γ(ν) Γ0 (ν) + log (1.2) Sf (τ, ν) = − log(f (t; τ, ν)) f (t; τ, ν) dt = ν + (1 − ν) Γ(ν) ν 0 In particular, at unit mean, the maximum entropy (or maximum uncertainty) occurs at ν = 1, which is the random case, and then Sf (τ, 1) = 1 + log τ. Figure 2 shows a plot of Sf (τ, ν), for the case of unit mean τ = 1. So, a Poisson process of points on a line are as disorderly as possible and among all homogeneous point processes with a given density, the Poisson process has maximum entropy. The maximum likelihood estimates τˆ, νˆ of τ, ν can be expressed in terms of the mean and mean logarithm of a set of independent observations X = {X1 , X2 , . . . , Xn }. These estimates are obtained in terms of the properties of X by maximizing the log-likelihood function ! n Y log likX (τ, ν) = log p(Xi ; τ, ν) i=1

with the following result n

τˆ = < X >= log νˆ −

Γ0 (ˆ ν) Γ(ˆ ν)

1X Xi n i=1

= < log X > − log < X >

(1.3) (1.4)

Pn where < log X >= n1 i=1 log Xi . We note here that the usual method to test the fit of sample data to an arbitrary distribution is that of Kolmogorov-Smirnov. It uses as deviation statistic the maximum absolute difference between the sample distribution function and the test distribution (cf. [14], pp 476-487).

3

C.T.J. Dodson and J. Scharcanski

Entropy 1

Sf (τ, ν)

0.5 2

4

6

8

10

12

14

ν

-0.5 -1 -1.5 -2 -2.5

Figure 2: Information entropy Sf (τ, ν), for gamma distributions of inter-event intervals t with unit mean τ = 1. The Riemannian information metric on the parameter space G = {(τ, ν) ∈ R+ × R+ } for gamma distributions is given by   1 ν dν 2 for τ, ν ∈ R+ , (1.5) ds2G = 2 dτ 2 + ψ 0 (ν) − τ ν 0

(ν) is the logarithmic derivative of the gamma function. The 1-dimensional where ψ(ν) = ΓΓ(ν) subspace parametrized by ν = 1 corresponds to all possible ‘random’ (Poisson) processes, or equivalently, exponential distributions. We have proved elsewhere [6] that there is a Riemannian manifold L consisting of log-gamma distributions and isometric with G. This log-gamma manifold L has several useful properties in security testing of smartcards [9] and in modeling of galactic cluster evolution [5]:

Proposition 1.1 ([6]) The log-gamma probability density functions for random variable N ∈ [0, 1] ν

g(N, τ, ν) =

1 1− τ N

( τν )ν (log Γ(ν)

1 ν−1 N)

for τ, ν ∈ R+ × R+

(1.6)

determine a metric space L of distributions with the following properties • L contains the uniform distribution as the limit: limτ →1 g(N, τ, 1) = g(N, 1, 1) = 1 • L contains approximations to truncated Gaussian distributions • L ≡ G is an isometry of the Riemannian manifold of gamma distributions with informationtheoretic metric. In fact, the log-gamma family (1.6) arises from the gamma family (1.1) for the non-negative random variable t = log N1 , or equivalently, N = e−t . So, the gamma and log-gamma families of distributions have a common differential geometry through the information metric and the exponential distributions in G map onto the uniform distribution in L.

1.1

Correlation

A well known but apparently difficult problem is to find a closed analytic representation of a correlated process for two variables with different marginal gamma distributions. Kibble’s bivariate gamma distribution has been used in a variety of applications [15], but from our viewpoint it

4

Information Geometric Similarity Measurement for Near-Random Stochastic Processes

ν=5

2.5

g(N ; τ, ν)

2

ν=2

1.5

ν=1

1

ν = 0.5 0.5

0.2

0.4

0.6

0.8

1

N ¯ = 0.5, and Figure 3: Log-gamma probability density functions g(N ; τ, ν), with central mean N ν = 0.5, 1, 2, 5. The case ν = 1 is the normal distribution, ν < 1 corresponds to clustering in the underlying spatial process; conversely, ν > 1 corresponds to smoothing. suffers from the disadvantage that its two marginal gamma distributions have a common dispersion parameter ν. Kibble’s density function takes the following form for correlation coefficient 0 ≤ ρ < 1 and random variable (t, d) ∈ R+ × R+ ! p  2 ν  (ν−1)/2 −ν(t/τ +d/δ)/(1−ρ) 2ν ρtd/τ δ ν tdτ δ e (1.7) Iν−1 g(t, d) = 1−ρ τδ ν2ρ (1 − ρ) Γ(ν) where Iν−1 is the modified Bessel function of the first kind of order (ν − 1), which appears here in the form ! p  ν−1+2k ∞ X 2ν ρtd/τ δ (ρtd/τ δ)k+(ν−1)/2 ν = (1.8) Iν−1 1−ρ k! Γ(ν + k) 1−ρ k=0

Special cases have been studied for various applications. Downton [10] used a bivariate exponential distribution and compared it in application to reliability theory with Marshall and Olkin’s distribution [17]. Ong [19] has provided some expansions for the corresponding cumulative distribution function. Shi and Lai [20] provided the Fisher information matrix components for the bivariate exponential distribution as a special case of Kibble’s bivariate gamma distribution (1.7) but still had to resort to a numerical evaluation of one of the integrals. So it would be difficult to apply the Fisher matrix to develop the Riemannian geometry of the parametric models even in the correlated exponential case.

2

Curves, length and energy in G

A path through the parameter space G of gamma models determines a curve, parametrized by t in some interval a ≤ t ≤ b, given by c : [a, b] → G : t 7→ (c1 (t), c2 (t)) and its tangent vector c(t) ˙ = (c˙1 (t), c˙2 (t)) has norm ||c|| ˙ given via (1.5) by   1 c2 (t) 2 2 0 ||c(t)|| ˙ = c˙1 (t) + ψ (c2 (t)) − c˙2 (t)2 c1 (t)2 c2 (t)

(2.9)

(2.10)

5

C.T.J. Dodson and J. Scharcanski

ν Random Processes: ν = 1

1

G

0.8

0.6

Clustering For ν < 1

0.4

0.2

0.4

0.6

τ

0.8

1.2

Figure 4: Geodesics radiating from (τ, ν) = (1, 0.5) in the gamma manifold G; these are curves of minimum information length. A curve through the 1-d subspace ν = 1 of random processes from (a, 1) to (b, 1) is c : [a, b] → G : t 7→ (t, 1); it has information length log ab and energy b−a ab . and the information length of the curve is Z

b

||c(t)|| ˙ dt

Lc (a, b) =

for a ≤ b.

(2.11)

a

Since arc length is often difficult to evaluate analytically, we sometimes use the ‘energy’ of the curve instead of length for comparison of information cost differences between nearby curves. Energy is given by integrating the square of the norm of c˙ Z b 2 Ec (a, b) = ||c(t)|| ˙ dt. (2.12) a

The curve c(t) = (t, 1), Figure 4, which passes through random processes with t = τ and ν = 1 = constant, has information length log ab and energy b−a ab . It is easily shown that a curve of constant τ has c(t) = (constant, t) where t = ν and c(t) ˙ = (0, 1); this has energy log ab + ψ 0 (b) − ψ 0 (a). Locally, minimal paths in G are given by the autoparallel curves or geodesics [8] defined by (1.5). An example of a spray of geodesics in the vicinity of the point (τ, ν) = (1, 0.5) is shown in Figure 4. These geodesics were computed using Gray’s Mathematica code for surfaces [11].

2.1

Local approximations and a geodesic search mesh

In the manifold G of gamma models for the distribution of intervals between events, we use the Riemannian metric to measure information distances between pairs of points. In a neighbourhood of a given point we can obtain a locally bilinear approximation to this distance. From (1.5) for small variations ∆τ, ∆ν, near (τ0 , ν0 ) ∈ G; it is approximated by s   ν0 1 2 + ψ 0 (ν ) − ∆τ ∆ν 2 . (2.13) ∆sG ≈ 0 τ02 ν0 2

As ν0 increases from 1, the factor (ψ 0 (ν0 ) − ν10 ) decreases monotonically from π6 − 1. So, in the information metric, the difference ∆τ has increasing prominence over ∆ν as the standard deviation reduces with increasing ν0 —corresponding to increased temporal smoothing of event scheduling.

6

Information Geometric Similarity Measurement for Near-Random Stochastic Processes

In particular, near the exponential distribution, where (τ0 , ν0 ) = (1, 1), (2.13) is approximated by s  2  π 2 ∆sG ≈ ∆τ + − 1 ∆ν 2 . (2.14) 6 For a practical implementation we need to obtain rapid estimates of distances in larger regions than can be represented by quadratics in incremental coordinates. This can be achieved using the result of Dodson and Matsuzoe [7] that established geodesic foliations of the gamma manifold. Now, a geodesic curve is locally minimal and so a network of two non-parallel sets of geodesics provides a mesh of upper bounds on distances by using the triangle inequality about any point. Such a geodesic mesh is shown in Figure 5 using the geodesic curves τ = ν and ν = constant, which foliate G, as described in [7]. Explicitly, the arc length along the geodesic curves τ = ν from (τ0 , ν0 ) to (τ = ν, ν) is |

d2 log Γ d2 log Γ (ν) − (ν0 )| dν 2 dν 2

and the distance along curves of constant ν = ν0 from (τ0 , ν0 ) to (τ, ν0 ) is τ0 | τ

|ν0 log

In Figure 5 we use the base point (τ0 , ν0 ) = (20, 1) ∈ G and combine the above two arc lengths of the geodesics to obtain an upper bound on distances from (τ0 , ν0 ) as Distance[(τ0 , ν0 ), (τ, ν)] ≤ |

3

d2 log Γ d2 log Γ τ0 (ν) − (ν0 )| + |ν0 log |. 2 dν dν 2 τ

(2.15)

Product geometries

In a practical application of the above differential geometry we can measure departures from randomness in the gamma manifold G. Equivalently, in the log-gamma manifold L we can measure differences between approximations to truncated Normal distributions or departures from a uniform distribution. In fact all of these types of comparison between such distributions—or empirical sampling of them— arise in the cost function for approximating a given stochastic texture. In general, however, we have n distributed parameter sets to optimize. First we consider the case where our n search parameters all come from the joint families of gamma and log-gamma distributions and are independent of one another. Then we have a product manifold P of dimension 2n with n pairs of coordinates {(τi , νi )|i = 1, 2, . . . , n}. So, P consists of a product of γ copies of G and λ copies of L, where γ, λ ≥ 0 and γ + λ = n. Hence, P = G γ × Lλ (3.16) with the n-fold direct product metric of (1.5) ds2P

γ  X νi

  1 2 = + ψ (νi ) − dνi νi i=1     n X νi 2 1 0 2 dνi + dτ + ψ (νi ) − τi2 i νi i=γ+1 dτ 2 τi2 i



0

(3.17)

for τi , νi ∈ R+ but νi ≥ 1 for i > γ. We note that each component space in such products contributes two dimensions.

3.1

Warped products and correlation

More intricate products arise in applications of geometry to physics. A warped product of two Riemannian manifolds (X, g) with coordinates (xi ) and (Y, h) with coordinates (yi ) is a manifold

7

C.T.J. Dodson and J. Scharcanski

(X × Y, g ×f h) with coordinates (zi ) = ((xi ), (yi )) where the metric g ×f h has the form g ×f hij dz i dz j = gij dxi dxj + f (xi )hij dy i dy j for some positive warping function, f defined on X. It is possible that correlation may be represented to some extent by a suitable choice of warping function in a warped product; this is under investigation. Meanwhile, it seems that some empiricism may be needed to introduce correlation between variables in the manifold P. One way would be to modify the direct product metric by introducing symmetrically off-diagonal terms gij , i 6= j, while preserving positive definiteness. These offdiagonal terms could be bounded by ±, say, and ranked in absolute size by the relative strengths of the corresponding correlations.

3.2

Example of 2-fold products

Let us take for illustration the submatrix of the metric gij for i, j = 1, 2, . . . , 4; so it applies to one of the spaces G 2 , G × L, L × G, L2 , as part of the n-fold product P. Then this part of the metric tensor matrix gij will have the form   ν1 /τ12 0 ρ13 ρ14  0  ρ23 ρ24 ψ 0 (ν1 ) − ν11  [M12 ] =  (3.18) 2  ρ13  ρ23 ν2 /τ2 0 ρ14 ρ24 0 ψ 0 (ν2 ) − ν12 and the information distance arc length element from this component of the metric tensor will be given by  T ds2 = dτ1 dν1 dτ2 dν2 [M12 ] dτ1 dν1 dτ2 dν2 (3.19) Here the off-diagonal terms ρij are symmetric and consist of the product of the correlation coefficient ρij = ρji between the two parameter spaces and the scale control . The scaling value  must be chosen such that det[M12 ] > 0, to ensure that positive definiteness is preserved. The maximum likelihood estimates should be used for the parameter values (τi , νi ) obtained from measured data histograms. The simplest case is perhaps that of a relationship between the two mean values, τ1 , τ2 . For this suppose that all of the ρij are zero except ρ13 = ρ, say. Then we have to control the size of ρ in order to have det[M12 ] > 0, namely ν1 /τ12 0 ρ 0 1 0 0 0 0 ψ (ν1 ) − ν1 > 0. (3.20) det[M12 ] = 2 0 ν2 /τ2 0 ρ 0 0 0 ψ 0 (ν2 ) − ν12 But we know that the product of diagonal terms is positive because this is the determinant for the trivial product space, ie with ρ = 0. Hence the constraint reduces to ρ2 < √ −

ν1 ν2 τ1 τ 2

1. In the case of the nonuniform abundances, we observe, as expected that the mean interval τ between occurrences of a symbol decreases with increasing abundance, essentially one is a reciprocal of the other. The other parameter, ν, increases also with abundance, in a systematic way. The extraction of such features from long multi-symbol sequences might be of value in monitoring and managing information flow through large networks. In such cases, dynamic management might benefit from knowledge of qualitative and partially quantitative properties of datastream flow simply by exploiting stable stochastic features.

4.3

Product space geometry for multi-feature image characterization

In the previous paragraph we considered the case of multi-symbol sequences having symbol abundances uniformly or exponentially distributed. This handles the case when the only ordering of importance for the symbols is by relative abundance: either the abundances are the same for each symbol or they can be ranked. In the case of an image—we envisage the case when it has a high degree of stochasticity—there may be many features of interest, each having an observable distribution of values. Then one very simple way in which we may utilize our product space P is by reducing each feature distribution to a 3-block histogram. Then we can be sure of being able to fit one of the gamma or log-gamma distributions. Such a procedure makes a reduction in the total data but preserves some qualitative aspects of the shape of the distribution for each feature: uniform, decreasing, increasing, central maximum or central minimum. These shapes can be modelled reasonably by gamma or log-gamma distributions and hence maximum likelihood parameter estimations may be made and distance estimates between points in P may be used to compare one image with another. That the gamma and log-gamma families have an adequate range of shapes may be believed by looking at the samples plotted in Figure 3 and Figure 7. Figure 7 shows some skew log-gamma ¯ from 1 to about 3 , which contrast with the symmetric distributions for ν > 1, having mean N 2 4 cases in Figure 3. In some situations we expect that the data reduction is worthwhile, for example when stable stochastic features have good classifying properties.

11

C.T.J. Dodson and J. Scharcanski

3 h(N ; τ, ν)

3

2 1 0

2.5 2 ν 0.2 0.4 N

1.5

0.6 0.8 1

Figure 7: Part of the surface representing the log-gamma family of probability densities h(N ; τ, ν), for 31 < τ < 1, showing some cases having increasing density function—here the mean increases ¯ = 1 to about 3 . The limiting distribution at ν = 1 is the uniform distribution. These skew from N 2 4 distributions contrast with the symmetric ones in Figure 3.

4.4

Autocorrelated symbol sequences from discretized time series

Consider a continuous real time series ˜ : [0, T ] → R : t 7→ X(t). ˜ X

(4.24)

˜ over subintervals of Discretizing to give a sequence of m values consisting of local averages of X length ∆T = T /m, we obtain 1 X : {1, 2, . . . , m} → R : k 7→ Xk = ∆T

Z

k∆T

˜ dt. X(t)

(4.25)

(k−1)∆T

Next we discretize the ordinate values Xk , using n appropriately scaled integers, which without loss of generality we may denote by {1, 2, . . . , n}. Now we have a sequence of length m consisting of n symbols drawn from {1, 2, . . . , n}. In general, for time series arising from continuous processes, we may expect that such sequences of symbols exhibit autocorrelation: Xk tends to have a similar value to Xk±1 .

5

Concluding Remarks

We have offered arguments to support the opinion that the gamma and log-gamma distributions have a natural role in representing departures from randomness, uniformity and Gaussian behaviour in stochastic processes. We show also how the information geometry provides a surprisingly tractable Riemannian manifold and product spaces thereof, on which may be represented the evolution of a stochastic process or the comparison of different processes, by means of well-founded maximum likelihood parameter estimation.

12

Information Geometric Similarity Measurement for Near-Random Stochastic Processes

References [1] Y. Cai, C.T.J. Dodson, A.J. Doig and O. Wolkenhauer. Information theoretic analysis of protein sequences shows that amino acids self cluster. To appear J. Theor. Biology. http://www.ma.umist.ac.uk/kd/PREPRINTS/amino.pdf [2] C.T.J. Dodson. Gamma manifolds and stochastic geometry. In: Proceedings of the Workshop on Recent Topics in Differential Geometry, Santiago de Compostela 16-19 July 1997. Public. Depto. Geometr´ia y Topolog´ia 89 (1998) 85-92. [3] C.T.J. Dodson. Information geodesics for communication clustering. J. Statistical Computation and Simulation 65, (2000) 133-146. [4] C.T.J. Dodson. Evolution of the void probability function. Presented at Workshop on Statistics of Cosmological Data Sets, 8-13 August 1999, Isaac Newton Institute, Cambridge. http://www.ma.umist.ac.uk/kd/PREPRINTS/vpf.ps. [5] C.T.J. Dodson. Spatial statistics and information geometry for parametric statistical models of galaxy clustering. Int. J. Theor. Phys., 38, 10, (1999) 2585-2597. [6] C.T.J. Dodson. Geometry for stochastically inhomogeneous spacetimes. Nonlinear Analysis, 47 (2001) 2951-2958. http://www.ma.umist.ac.uk/kd/PREPRINTS/gsis.ps. [7] C.T.J. Dodson and Hiroshi Matsuzoe. An affine embedding of the gamma manifold. InterStat January 2002, 1-6. http://www.ma.umist.ac.uk/kd/PREPRINTS/affimm.pdf. [8] C.T.J. Dodson and T. Poston. Tensor Geometry Graduate Texts in Mathematics 130, Second edition, Springer-Verlag, New York, 1991. [9] C.T.J. Dodson and S.M. Thompson. A metric space of test distributions for DPA and SZK proofs. Poster Session, Eurocrypt 2000, Bruges, 14-19 May 2000. http://www.ma.umist.ac.uk/kd/PREPRINTS/mstd.pdf. Cf. also: C.T.J. Dodson. Information-geometry of departures from randomness and from uniformity. Preprint 2001, http://www.ma.umist.ac.uk/kd/PREPRINTS/igssc.pdf. [10] F. Downton. Bivariate exponential distributions in reliability theory. J. Royal Statist. Soc. Series B 32 (1970) 408-417. [11] A. Gray Modern Differential Geometry of Curves and Surfaces 2nd Edition, CRC Press, Boca Raton 1998. [12] R.C. Griffiths. The canonical correlation coefficients of bivariate gamma distributions. Annals Math. Statist. 40, 4 (1969) 1401-1408. [13] T-Y. Hwang and C-Y. Hu. On a characterization of the gamma distribution: The independence of the sample mean and the sample coefficient of variation. Annals Inst. Statist. Math. 51, 4 (1999) 749-753. [14] M. Kendall and A. Stuart. The Advanced Theory of Statistics, Volume 2 Inference and Relationship 4th Edition. Charles Griffin, London, 1979. [15] W.F. Kibble. A two variate gamma-type distribution. Sankhy¯ a 5 (1941) 137-150. [16] C.D. Lai. Constructions of bivariate distributions by a generalized trivariate reduction technique. Statistics and Probability Letters 25, 3 (1995) 265-270. [17] A.W. Marshall and I. Olkin. A generalized bivariate exponential distribution. J. Appl. Prob. 4 (1967) 291-302. [18] G.K. Miller and U.N. Bhat. Estimation for renewal processes with unobservable gamma or Erlang interarrival times. J. Statistical Planning and Inference 61, 2 (1997) 355-372.

C.T.J. Dodson and J. Scharcanski

13

[19] S.H. Ong. Computation of bivariate-gamma and inverted-beta distribution functions. J. Statistal Computation and Simulation 51, 2-4 (1995) 153-163. [20] D. Shi and C.D. Lai. Fisher information for Downton’s bivariate exponential distribution. J. Statistical Computation and Simulation 60, 2 (1998) 123-127. [21] M. Tribus. Thermostatics and Thermodynamics D. Van Nostrand and Co., Princeton N.J., 1961.) [22] M. Tribus, R. Evans and G. Crellin. The use of entropy in hypothesis testing. In Proc. Tenth National Symposium on Reliability and Quality Control 7-9 January 1964.