Optimal design of digital IIR filters by model-fitting ... - IEEE Xplore

702

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS-11:

ANALOG AND DIGITAL SIGNAL PROCESSING. VOL. 42. NO. 11. NOVEMBER 1995

Optimal Design of Digital IIR Filters by Model-Fitting Frequency Response Data Arnab K. Shaw, Member, IEEE

Abstract-A new frequency-domain (FD) approach is presented for optimal estimation of rational transfer functions coefficients. The proposed method seeks to match any arbitrarily-shaped FD specifications in the least-squares sense. The desired specifications may be arbitrarily spaced in frequency. The design is performed directly in the digital domain, and no analog to digital transformation is necessary. The proposed method makes use of the inherent mathematical structure in this rational modeling problem to theoretically decouple the numerator and denominator estimation problems into two lower dimensional problems. The denominator criterion is nonlinear but possesses a weighted-quadratic structure, which is convenient for iterative optimization. The optimal numerator is found linearly by solving a set of simultaneous equations. The decoupled criteria retain the global optimality properties. The performance of the algorithm is demonstrated with some simulation examples.

space, which is independent of the numerator coefficients. The optimal numerator estimation problem turns out to be a simple linear LS problem. It is demonstrated in this paper that the optimal rational identification problem in FD belongs to a special class of mixed-nonlinear optimization framework, where the linear and nonlinear variables separate [ 131. It is further shown that the true nonlinear criterion can be decoupled into the following: 1) a purely linear problem for obtaining the optimal numerator coefficients, and 2) a nonlinear problem of reduced dimensionality for determining the optimal denominator coefficients.

These important underlying theoretical and algorithmic aspects of designing digital filters in FD appear to have remained mostly unutilized. After decoupling, the denominator I. INTRODUCTION criterion possesses a convenient weighted-matrix structure, RADITIONALLY, digital filters are designed by which is then utilized to develop an iterative minimization performing impulse invariance or bilinear transformation algorithm. Once the denominator is estimated, the optimal on available analog designs. Classical analog designs utilize numerator is found only once with linear LS. The decoupolynomial approximations to match standard filter shapes pled criteria retain the global optima of the original mixed such as low-pass, high-pass, etc. [9], [IO]. An obvious criterion. The proposed approach is closely related to some drawback of classical analog design techniques is that filters time-domain results developed recently by the present auwith arbitrary or nonclassical specifications, as in case of thor [SI, [16], [17], [35]. The design methodology described a notch filter, can not be obtained. In this paper, a direct here will be based on matching desired discrete-time-Fouriermethod for frequency-domain design of digital IIR filters is transform (DTFT) values, which may be arbitrarily spaced in proposed. The method seeks to match a desired frequency frequency. However, the algorithm can be easily modified if response with any arbitrary shape by minimizing the optimal the desired specifications are available in the form of DFT least-squares (LS) fitting error criterion. The LS criterion values. The paper is arranged as follows. In Section 11, the rational for this problem involves multidimensional nonlinear search, and several linearized or modified approaches have been transfer model is defined and the FD identification problem developed [2], [31, [21], [31]. There have been some ad hoc is stated. In Section 111, some existing methods addressing attempts to design digital filters with special shapes [9], [ 121. this problem are briefly outlined. The details of the proposed A frequency-domain version of Prony's algorithm has also decoupled solution are presented in Section IV. Finally, some been presented recently [ 141, [ 1.51, [ 191, [2.5]. But it appears simulation examples are given in Section V to demonstrate the that the underlying mathematical structure inherent in this performance of the proposed approach. rational modeling problem has not been fully exploited. In this paper, the FD Ls problem is by identifying TRANSFERFUNCTION MODELAND THE 11. THERATIONAL the orthogonal projection space, which is shown to be FREQUENCY-DOMAIN DESIGNPROBLEM formed entirely by the denominator parameters. The optimal as An ARMA ( P . q ) digita1 lilter can be denominator is estimated by minimizing the exact projection

T

Manuscript received July 13, 1993; revised June 15, 1994. This paper was supported in part by AFOSR under Grant AFOSR-F49620-90-C-09076 and Grant AFOSR-F49620-93-1-0014.This paper was recommended by Associate Editor V. K. Madisetti. The author is with the Department of Electrical Engineering, Wright State University, Dayton, OH 45435 USA. IEEE Log Number 9414442.

H(z) = Ch(i)z-' i=O -

1057-7130/95$04.00 0 1995 IEEE

+ U ( 1)z-1 + ' . + u(q)z-9 a-N ( z ) + b(1)z-1 + . . . + b(p)z-P = D(z)'

u(0)

1

'

(1)

703

SHAW: OPTIMAL DESIGN OF DIGITAL IIR FILTERS

Let

B. Sanathanan-Koemer 's Prejltering Method (SKM)

h

6 [h(O) -

h(1) . . .

h ( N - I)]T,

(2a)

be the vector with the first N significant samples of H ( z ) and

a - [a(O) a(1) ... a(q)IT and b A [I b ( 1 ) . . . b(p)lT -

(2b) (2c)

be the numerator and denominator coefficient vectors, respectively. Let H d ( z ) represent the desired IIR filter, which needs to be modeled as H ( z ) in (1). Using the notations of (I), let Hd(wlc), N ( w L ) ,and D ( w k ) be defined as the frequency response values of H d ( z ) , N ( z ) , and D ( z ) , respectively, at z = e J w k .The FD identification problem can be stated as follows: Given H d ( w k ) , at k = O , 1 . 2 , . . . , N - 1, the desired frequency response values (possibly arbitrarily spaced), estimate the parameters in N ( w k ) and D ( w k )by optimizing the following LS error criterion:

111. SOMEEXISTINGFREQUENCY-DOMAIN DIRECTDESIGNMETHODS The problem stated in (3) is a nonlinear optimization problem, and standard nonlinear optimization schemes can be used [7], [ l l ] . But these generic algorithms are known to be sensitive to initial choice of estimates, and they do not specifically make use of the unique mathematical structures inherent in this problem. Some linearized methods that specifically address the design problem stated in (3) have also been proposed [2], [3]. More recently, a decoupled algorithm that utilizes divided-differences and Newton-Raphson, has been reported in [141, [28]. In order to motivate the proposed algorithmic framework, brief outlines of some of the direct FD design methods are given next. A. Levy's Method (LM) The following criterion was proposed by Levy [2] as a FD counterpart of Kalman's original work in the time-domain [ 11:

Note that the original error criterion in (3) is modified in Levy's case. Apart from the obvious advantage of singlestep linear solution, this algorithm does not possess any other

the first term of the error criterion was minimized. The AR parameter estimation work is further related to Prony's method [ 191 and Pad6 Approximation [20]. Similar error criterions for the ARMA problem have been later rediscovered [21] and

The earliest work that most closely approximates the true LS fitting-error criterion appears to be due to Sanathanan and Koerner [3]. Their goal was to improve upon Levy's work, which did not really attempt to optimize the true criterion in (3). In this case, an initial estimate of the denominator coefficients D(O)(wo)is first obtained by minimizing Levy's criterion in (4), and then the following mod$ed fitting error criterion is optimized at the kth iteration [3]:

a

K-1

minllesK112 A min a.b a,b

D(wz ) H d ( W , )

1=0

-

N(wt) D("-')(w,)

D("-')(w,)

12. (5)

where D("-')(w,) denotes the denominator estimate at the previous iteration, which is used as a prefilter for obtaining the estimates at the following iteration step. Note that ( 5 ) closely approximates (3), and both are identical if D ( w l ) = D ( k - l ) ( w z ) .But using (3,the unknown parameters in a and b can be estimated simultaneously by solving a set of linear equations. A time-domain counterpart of SKM was later discovered independently by Steiglitz and McBride in [4], though the later work is definitely more recognized in signal processing and system identification literature [9], [lo], [23], ~41.

C. Kumaresan 's Decoupled Method-Generalized (KM-G) The FD error criterion in (3) has been recently decoupled by Kumaresan's group [14], [15], 1251, [28], where divideddifference matrices [26], [27] have been utilized. Similar to a time-domain decoupled algorithm due to Evans and Fischl [6], this approach was originally proposed for strictly-proper cases, ie, when p = q 1. In the brief outline given below, appropriate modifications have been introduced in order to generalize KM for any arbitrary numerator and denominator orders. For qth order numerator and pth order denominator, the decoupled criterion for estimating the optimal denominator is

+

min hi CH(CCH ) - l Ch$ b

(6)

where

h$ A

[Hd(Wo)

...

Hd(w1)

Hd(WIv-i)IT

(7a)

denotes the vector containing the N samples of the prescribed frequency response data and C is defined in (7b)-(7e) at the bottom of the next page. Defining f - UDh;;' E ~ ( A ~ ' + P - q - l ) x s , the error critenon can be written in the following weighted-quadratic form: min b bTFH(CCH)-lFb.

F A

f(P) f ( P + 1)

f ( P - 1)

f(P)

(8)

... ...

f(0)

'..

f ( r- q - 2 ) -

f(1)

=

-f(S + p -q

-

2)

f ( r+ p

-

q - 3)

704

ANALOG AND DIGITAL SIGNAL PROCESSING, VOL. 42, NO. 11. NOVEMBER 1995


The optimal denominator coefficients are obtained using an iterative algorithm. Once the optimal denominator is available, the numerator is estimated as

further step of Newton-Raphson had been invoked in order to achieve the desired optimum [14]. Unlike KM-G, the optimally decoupled method developed in this work reaches the desired optimum criterion more directly and without using Newton-Raphson. It may also be noted that the Signal Processing Toolbox of the widely popular MATLAB software package provides a direct FD design macro called yulewalk,which essentially implements a modified Yule-Walker method developed by Friedlander and Porat [31]. This method does not attempt to minimize the true criterion in (3). Instead, it attempts to fit the deterministic correlation values to obtain the rational model parameters by minimizing an equation error. The simulation section includes some comparison of the performance of the proposed method with this approach.

IV. PROPOSED METHOD(OM-DTFT) For time-domain rational model identification problems, a new framework has been recently presented for decoupling the denominator and numerator problems into two separate It can be easily verified that for the special case of p = q 1, but lower-dimensional optimization problems [8], [ 161, [ 171, the general criteria given here will be exactly same as the one [35]. In this paper it is shown that the nonlinear FD criterion given in [141 and [28]. It may be emphasized here that the LS of (3) can also be decoupled in a similar fashion. Let Hb(2) be the inverse filter corresponding to D ( z ) , i.e., algorithms in (4) and ( 5 ) are

+

1) approximations or modifications of the original criterion in (3), 2) ( p q)-dimensional nonlinear optimization problems for estimating a and b simultaneously.

D(z)Hb(2) = 1.

+

In contrast, the decoupled method (KM-G) estimate a and b separately. According to [14] and [28], the desired minimum of the criterion in (8) may not be achieved with only an Evans-Fischl type LS minimization of (8). Instead, a

Clearly, this is a convolution operation in the time-domain and it can be expressed using matrix notation as

cn B,TUD, rb(p) . . . b(i)

with,

1

(13 4

... 01

DHb = I N ,

(13b)


705

where IN denotes an N x N identity matrix; D E E t N x N (corresponding to Hb(z)). Hence it can be shown that the and Ha E E t N x Nare defined on the next page in appropriate model impulse response vector h can be expressed as partitioned forms which will be useful in the algorithm ... 1 0 0 0 ... 0 0 b(1) ” ’ 0 0 0 ... 0 0 Using this in (16), . . b(q) ... 1 0 0 ... 0 0 --

--

:q+1)

b(p)

-

Hb A -

--

--

1

0

.

......

” ’

b(1)

... b(p) . . . . . .

0

and

--

. . . b(1) .

hb(0) hb(1)

hb(q+l)

” ’

0

..’

0

..

:

. . . hb(1)

..

hb(N-1)

: ......

-- - -

...

0

.

.

-0

1

‘..

0

” ’

b(1)

1

1 I .“ I ’. I ... I ’.. I t..

” ’

With these definitions, the FD filter design problem in (3) can be restated as

Equation (20) represents the original criterion in (3), albeit in vector-matrix form. This form of the criterion explicitly 0 demonstrates the linear relationship between the fitting error e and a, and also the nonlinear relationship between e and : 0 A [ H I I H , ] . b through the matrix Hl. From this equation it is also apparent that this problem belongs to a certain class of mired optimization problems, where the linear and nonlinear hb(0) variables appear separately [13]. In order to decouple the numerator and denominator estimation problems, consider the following. If Hl (i.e., b) is known, then the minimization of (20) will produce the following linear LS estimate of a: 0

-

N-1

H(w;)

-

H(z)IZ=e3Wx h(n)e-jwzn. n=O

(15)

where (WHl)# A ((WHl)T(WHl))-l(W H I )is~ known as the pseudoinverse. In practice, though, b needs to be estimated also. Plugging a back into (20), the optimization criterion for b is found as

Stacking the model frequency response values at all the N specified frequencies, W O , w1, W N - ~ , the model frequency-domain vector can be expressed as ..e,

where P W HA ~ WH,((WHI)~(WHI))-~(WH~)~ denotes the projection matrix of (WHl). In a more general setting encompassing a larger class of mired nonlinear optimization problems, Theorem 2.1 in [13] proves that if the linear (i.e., a) and the nonlinear (i.e., b) parameters appear separately in any nonlinear optimization criterion, then the L optimization procedure can be decoupled while maintaining A Wh. the identical optimal points. Specifically, following [ 131 it can be shown that if b is estimated by minimizing the criterion By definition, in (22) (or (30), as explained in the following subsection) and then that estimate is utilized for computing a using (21), then the resulting estimates are the unique and global minimizers using (13a), of the criterion in (20). Note that the numerator and denominator estimation probwhere the right-hand side represents convolution of the lems are now in decoupled forms in (21) and (22), respectively. numerator coefficients with the inverse sequence, hb( n ) But in (22), the parameters in b are related to the error

706


ANALOG AND DIGITAL SIGNAL PROCESSING. VOL. 42, NO. I I . NOVEMBER 1995

criterion in a somewhat complicated manner through P w H ~ .It can be easily shown that Interestingly, the operator (IN - P w H ~on ) hy in (22) is the BTz A Zb. (294 projection component in h;t:that is orthogonal to the subspace spanned by the columns of WHl. Next it is shown that this orthogonal space can be explicitly defined by the denominator where the matrix Z is constructed with the elements of z as coefficients. A. Reparameterization

Let WI denote the inverse of the DTFT matrix W, i.e, WIW = 1 ~ 7 This . inverse exists as long as the frequencies Wk's

are distinct. In combination with (13b),

Invoking the partitioned forms of (14) into (23) leads to (24), as shown at the bottom of the page. The bottom-left corner element in (24) shows that the Nx(N-q-1)matrix W T B a n d t h e N x ( q + l ) m a t r i x WHl are orthogonal, i.e., (BTWr)(WHi)= O(,Y-n-l)x(4+1). By construction,

Hence, using a property of projection matrices,

Using this result in (22), i.e., replacing ( I N

-

P w H ~ by )

PWTB*

Using (29) in (28), the optimization criterion can be rewritten as min bTZT(BT WIWFB)-'Zb. b

(30)

Note that this alternate form has a weighted-quadratic structure which is convenient for minimization. Equations (21) and (30) represent the final decoupled estimators to be utilized in the algorithm described below. Reduction in computational load is the primary advantage of estimating the linear and nonlinear parameters independently. This is because the iterations are only with respect to the p coefficients in b. Once the optimal b is found, estimation of the optimal a is a simple linear LS problem. But more importantly, a needs to be computed only once.

B. Algorithm The nonlinear optimization criterion in (30) possesses a convenient matrix structure. Specifically, the expression appears to be a weighted quadratic criterion in the unknown vector b. The matrices Z and WI are known. But the weight matrix (BTWIWTB)-litself is dependent on the unknowns in B. The computational algorithm utilizes this weighted quadratic structure of the criterion to formulate the iterations. Specifically, the algorithm minimizes the following quadratic error criterion at the kth iteration step:

Note that this reparameterized criterion is directly related to b, as desired. In order to further simplify this expression, define a vector z of length N as

A niiri bTRlb

=

b

where B("-') is formed by using the estimate of b obtained at the previous iteration and R1 A [ZT(BT("') such that the criterion in (26) becomes b min

z T ~ ( ~ T ~ I ~ T ~ ) - l ~ T(28)z .

WIWTB("'))-~Z] is the weight matrix. An initial estimate of b is necessary to start the iterative process. b(O) A [l 0 . . . 0IT can be used or the initial estimates could also befound by setting the middle matrix (BTWrWTB)-'


707

estimate the numerator coefficient vector a. Equivalently, a can be also found using (10). The necessary and sufficient condition for convergence of the iterations in (A.8) and (A.13) for the two phases, respectively, is that the iterative transformations be contraction mapping. According to the mean value inequality, an iterative transformation is a contraction mapping if the norm of the gradient of the transformation is less than unity [32]. This is difficult to verify theoretically in a general sense for any desired frequency response. However, the same type of iterative algorithm had been used for estimating complex sinusoidal frequencies [33], where it was numerically demonstrated for a large number of independent realizations that the gradient of the transformation was indeed bounded by unity. Some theoretical discussion on convergence may also be found in [34], where a similar iterative algorithm had been utilized for spatial-domain design of 2-D IIR filters. Analogous iterative algorithms have been used for system identification from impulse response [6], [8], [ 161 and input-output data [ 171, for spatial-domain design of 2-D IIR filters [34], [35], and also for 1- and 2-D frequency-wavenumber estimation [33]. In all these cases, the convergence of iterations has been found to be quite rapid in simulations. It may be further noted here that the initial solution for the algorithm is self-generating in the sense that the structure of the optimal denominator criterion itself is utilized to obtain the initial estimate in (31b), as discussed above. However, the algorithm showed surprising robustness to any arbitrary initial choice. For example, when the initial estimates were chosen randomly (keeping b(0) = l), the iterations converged to very close to the same optima, after taking one to three extra iterations. On the other hand, Newton-type algorithms were found to be highly sensitive to initial choice. as shown in the next section.

Lowpass Filter Design

0

.I

.- -20 g m

0

-30-

0 E

-40 -

-501 11

11

-600

'I

0.1

0.2

frequency

0.3

0.4

0.5

(a)

I

I

(b)

V. SIMULATION RESULTS

Fig. I . The desired lowpass response is shown as the solid line. (a) The estimated responses using the proposed method and the Yule-Walker method are shown in dashed and dot-dash lines, respectively. The filter order is six. (b) Results using Newton-Raphson method are shown for two cases. The dot-dashed and the dashed lines show the result with b = (10 0 . . 0lT and (3Ib) as initial estimates, respectively.

Two examples are included to demonstrate the effectiveness of the proposed algorithm. The first example considers a lowpass filter design problem, whereas in the second one a notch filter is designed. In all plots the frequency response values are displayed up to half the sampling frequency. For the proposed method, only the Phase 1 results are given. The times given for all the simulation runs correspond to CPU time reported by the MATLAB code.

to identity, i.e., by optimizing,

A min bTRzb min bTZTZb b

-

b

(3lb)

where the weight matrix R2 A ZTZ. In order to ensure nontrivial solutions, the first termof the denominator, b( 0), is set to unity. The computational algorithm is similar in nature to the time-domain counterparts developed recently [8], [ 161, [17]. As outlined in the Appendix, the algorithm has two phases. In Phase 1, the criterion in (31a) is minimized by neglecting the variation w.r.f., the weight matrix. Simulation experience shows that Phase 1 alone brings the error quite close to the minimum. But, if necessary, the variation of the weight matrix may also be included by invoking Phase 2, where the gradient of the entire criterion is set to zero. Once the iterations converge, the estimated b is used in (21) to linearly

A. Simulation 1 : Lowpass Filter Design Magnitude response values at 56 frequency points around the unit circle were taken for the matching purpose. In Fig. ](a) the estimated response with p = 6 and q = 5 for the proposed method are shown by the dashed curve, and the solid line represents the desired response. For the sake of comparing with a widely used direct method, the modified Yule-Walker (MYW) method [30], [31] available in the MATLAB software package was used to design a sixth-order filter. The magnitude response fit using MYW is shown as the dot-dash line in Fig. l(a). MYW being a noniterative method produced the results in 0.68 s, whereas the proposed algorithm converged in six iterations, taking about 4.8 s. In Fig. l(b) the results are

708


ANALOG AND DIGITAL SIGNAL PROCESSING, VOL. 42. NO. 11, NOVEMBER 1995

shown for the Newton-Raphson method. The dot-dashed line corresponds to the result with b = [l 0 0 . . . OIT as the initial estimate. This case converged after seven iterations and took about 50 min. The dashed line in Fig. l(b) shows the result when (3 lb) was used for initiating the Newton-Raphson iterations. The plot shows the result after 21 iterations, taking about 3 h when the iterations were stopped.

No(&

.

I

Filter Design

.

.

I

B. Simulation 2: Notch Filter A notch filter design problem was considered in this case. The magnitude response values at 50 frequency points around the unit circle were taken. The estimated response with tenth order denominator and ninth order numerator as produced by the proposed method as well as the desired response are shown in dB scale in Fig. 2(a) in dashed and solid lines, respectively. The dash-dot line again shows the fit when the MYW method [30], [31] was used to design the 10th order filter. MYW needed 0.66 s to produce the estimate. The proposed algorithm converged in 7 iterations taking 3.76 s. Fig. 2(b) shows the results with Newton-Raphson method in this case. The dot-dashed line corresponds to the result with b = [l 0 0 . . . OIT as the initial estimate. The dashed line in Fig. 2b shows result when (31b) was used for initiating the Newton-Raphson iterations. Both cases did not converge after 21 iterations each and were stopped after about 5.5 h.

Ndch Filter DeBipn

C. Discussion The first example has been adopted from [ 141 and [28]. The results presented above for the proposed method did not have to make use of any generic nonlinear optimization technique, such as Newton-Raphson, to reach the final optimum. Also, during the minimization process, all the coefficients were enforced to be real and hence the filter is readily realizable. It may also be stated here that the final designs were stabilized using the macro called polystab available in MATLAB [291, 1301, where the unstable roots are reflected inside the unit circle. The simulations clearly demonstrate that the proposed method can closely match arbitrarily shaped frequency response data. Furthermore, it appears to perform better than a widely used method for direct design and much faster than the Newton-Raphson based general iterative algorithm. The simulation experiments also indicate that the proposed method is less sensitive to initial conditions than the Newton-Raphson approach.

-201

0

0.05

0.1

0.15

V

0.2

0.25

0.3

0.35

0.4

0.45

0.5

WeqUeW

(b) Fig. 2. The desired notch filter is shown as the solid line. (a) The estimated responses using the proposed method and the Yule-Walker metod are shown in dashed and dot-dash lines, respectively. The filter order is ten. (b) Results using Newton-Raphson method are shown for two cases. The dot-dashed and the dashed lines show the result with b = [ l o 0 . . . 0IT and (31b) as initial estimates, respectively.

Phase I

The final form of the error vector in (31a) is rewritten as

APPENDIX COMPUTATIONAL ALGORITHM The criterion in (31a) is nonlinear in b, and hence it can not be minimized directly. But instead of using any generic nonlinear optimization technique, the inherent mathematical structure of the criterion will be utilized to develop an iterative computational algorithm. The algorithm consists of two phases. In Phase 1, the variations in the middle matrix (BTWIW:B) in (31a) is not taken into account in the derivative calculations, whereas in Phase 2 the gradient of the error norm in (31a) is set to zero.

= Vg

+ VGb

(A.4)

where

va W:B(B~W~W?B)-~ and

(A.5)


709

If the matrix V is treated as independent of b, an expression for b can be easily obtained by minimizing lleb11* w.r.f. b as follows:

&

and has the same form as the B matrix defined in (10) but is filled with all zeros except at the locations where b ( k ) appears. For example,

b = -(VG)#Vg = - (GTVTVG)-lG’VTVg.

(A.7)

But since V does depend on the elements in b, (A.7) can only be computed iteratively. At the ( i 1)th step of iteration, V(i) is formed using the estimate of b found in the ith iteration step. This leads to the following iterative algorithm for computing bZ+l.

+

(A. 12) Once b(Z+’) is found using (A.10), b(i+l)can be formed as,

(A. 13a) where

F(2) - GTVT(i)V(i),

(A.9)

=

[

1

]

................. . - [S(i)G]-l[S(i)]g A

(A.13b)

The iterations are continued until Ilbi+l - bill2 < 6,where 6 is an arbitrarily small number. It must be noted here This minimization phase continues until bi+’ N_ bz is reached that the iterations in (A.8) may not always converge to the and this optimum b vector corresponds to a minimum of the absolute minimum of the error criterion in (31a), and hence error surface of Ileb II;. the estimated b may not be the optimum one. This is because in (A.8) the variability of V w.r.t. b had been ignored while ACKNOWLEDGMENT minimizing lleb11*. To achieve the optimum, the gradient of the complete expression of lleb11* must be set to zero. If desired, The author thanks S . Pokala and S. Kundu for their assisthis can be done in Phase 2 of the algorithm, which is outlined tance in running the simulations. next. It may be noted here that the simulation studies indicate REFERENCES that the Phase 1 iterations using (A.8) perform an excellent job of bringing the estimate very close to the optimum. Once R. E. Kalman, “Design of a self optimizing control system,” Trans. the estimates of b converge, a is computed using (21) or (10). ASME, vol. 80, pp. 468478, 1958. Phase 2 In this phase, the derivative of the matrix Vw.r.t.b is taken into consideration while minimizing the fitting error norm. By setting the derivative of the squared norm in (31a) to zero, the updated b(z+l)at the ( i 1)th iteration is written as

+

b(z+1) = - [S(i)G]-1[S(i)Ig

(A.lO)

where (suppressing the superscript (i))

S A LTV+GTVTV: -

(A.l la)

E. C. Levy, “Complex curve fitting,” IRE Trans. Automatic Control, vol. AC-4, pp. 3 7 4 4 , May, 1959. C. K. Sanathanan and J. Koemer, “Transfer function synthesis as a ratio of two complex polynomials,” IEEE Trans. Automat. Conrr., vol. AC-8, pp. 56-58, Jan. 1963. K. Steiglitz and L.E. McBride, “A technique for identification of linear systems,” IEEE Trans. Automat. Contr., vol. AC-IO, pp. 461464, 1965. C. S. B u m s and T. W. Parks, “Time domain design of recursive digital filters,” IEEE Trans. Audio Electro-Acousr., vol. AU-18, pp. 137-141, June 1970. A. G. Evans and R. Fischl, “Optimal least squares time-domain synthesis of recursive digital filters,” IEEE Trans. Audio Electro-Acoust., vol. AU-21, pp. 61-65, 1973. J. A. Cadzow, “Recursive digital filter synthesis via gradient based algorithms,” IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP24, pp. 349-355, 1976. A. K. Shaw, “Optimal identification of discrete-time systems from impulse response data,” IEEE Trans. Signal Process., vol. 42, pp. 113-120, Jan. 1994. L. B. Jackson, Digital Filters and Signal Processing. Boston: Kluwer, 1986. T. W.Parks and C. S. B u m s , Digital Filters. Englewood Cliffs, NJ: Prentice-Hall, 1987. R. Fletcher and M. J. D. Powell, “A rapidly convergent descent method for minimization,” Compur. J., vol. 6, pp. 163-168, 1963. A. Nehorai, “A minimal parameter adaptive notch filter with constrained poles and zeros,” IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-33, pp. 983-996, Aug. 1985. G. H. Golub and V. Pereyra, “The differentiation of pseudoinverses and nonlinear problems whose variables separate,’’ SIAM J. Numer. Anal., vol. 10, no. 2, pp. 413432. Apr. 1973. R. Kumaresan and C. S. B u m s , “Fitting a pole-zero filter model to arbitrary frequency response samples,” in Proc. ASILOMAR-91, pp. 1649-1652. 1991.


ANALOG AND DIGITAL SIGNAL PROCESSING, VOL. 42, NO. 1 1 . NOVEMBER 1995

R. Kumaresan, “On the frequency-domain analog of Prony’s method,” IEEE Trans. Acoust., Speech, Signal Process., vol. 38, pp. 168-170, Jan. 1990. A. K. Shaw, “An optimal method for identification of pole-zero transfer functions,” in Proc. In?.Symp. Circuits Syst., San Diego, CA, May 1992, pp. 2409-2412. -, “A decoupled approach for optimal estimation of transfer function parameters from input-output data,” IEEE Trans. Signal Process., vol. 42, pp. 1275-1278, May 1994. N. Levinson, “The Wiener RMS error criterion in filter design and prediction,” J. Marh. Phys., vol. 25, pp. 261-278, Jan. 1947. R. Prony, “Essai experimental et analytique etc.,” L’Polytechnique, Paris, 1 Cahier 2, pp. 24-76, 1795. L. Weiss and R. N. McDonough, “Prony’s method, Z-transforms, and Pad6 approximations,” SIAM Rev., vol. 5 , no. 2, Apr. 1963. J. N. Brittingham, E. K. Miller, and J. L. Willows, “Pole extraction from real frequency information,” Proc. IEEE, vol. 68, pp. 263-273, Feb. 1980. C. T. Mullis and R. A. Roberts, “The use of second-order information in the approximation of discrete-time linear systems,’’ IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-24, pp. 226-238, June 1976. L. L. Scharf, Statistical Signal Processing: Detection, Estimation and 7ime Series Analysis. Reading, MA: Addison-Wesley, 1990. C. W. Therrien, Discrete Random Signals and Sratistical Signal Processing. Englewood Cliffs, NJ: Prentice-Hall, 1992. R. Kumaresan, “On identifying a rational transfer function from the frequency response samples,” IEEE Trans. Aerospace Electron. Syst., pp. 925, Nov. 1990. F. B. Hildebrand, Introduction to Numerical Analysis, 2nd ed. New York: Dover, 1987. R. Kumaresan, “Parameter estimation of signals corrupted by noise using a matrix of divided differences,” Invited Paper presented at the NATO Advanced Studies Institute, Canada, July 1988. J. Romero, Fitting a Pole-Zero Model to Arbitrary Frequency Response Samples, M.S. thesis, Univ. Rhode Island, Kingston, 1989.

[29] MATLAB 4.0, ref. guide, The Mathworks, Inc., Natick, MA, 1993. [30] Signal Processing Toolbox for use with MATLAB, User’s Guide, The Mathworks, Inc., Natick, MA, 1992. [31] B. Friedlander and B. Porat, “The modified Yule-Walker method of ARh4A spectral estimation,” IEEE Trans. Aerospace Electron. Syst., pp. 158-173, Mar. 1984. [32] D. G. Luenberger, Optimizarion by Vector Space Method. New York: Wiley, 1969. [33] A. K. Shaw, “Structured matrix problems in signal processing, “ Ph.D. dissertation, Univ. Rhode Island, 1987. [34] T. Hinamoto, “Design of 2-D separable-denominator recursive digital filters,” IEEE Trans. Circuirs Sysf., vol. CAS-31, pp. 925-933, Nov. 1984. [35] A. K. Shaw, “Design of denominator separable 2-D IIR filters,” to appear, Signal Process., vol. 42, pp. 191-206, Feb. 1995.

Arnab K. Shaw (S’78-M’87) was bom in Calcutta, India, on April 2, 1955. He received the B.S.E.E. degree from Jadavpur University, India, in 1979, the M.S.E.E. degree from Villanova University, Villanova, PA, in 1983, and the Ph.D. degree from the University of Rhode Island in 1987, all in electrical engineering. Since 1987, he has been with Wright State University, Dayton, OH, where he is now an Associate Professor with the Electrical Engineering Department. His primary research interests include linear and nonlinear system identification, 1- and 2-D digital filter design, spectrum estimation, speech processing, and neural network applications.