Numerical Optimization-Based Extremum Seeking ... - IEEE Xplore

454

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 52, NO. 3, MARCH 2007

Numerical Optimization-Based Extremum Seeking Control With Application to ABS Design Chunlei Zhang, Member, IEEE, and Raúl Ordóñez, Member, IEEE

Abstract—Extremum seeking control (ESC) schemes based on numerical optimization are proposed in this paper. The extremum seeking problem is treated as an optimization with dynamic system constraints. The numerical optimization-based extremum seeking control scheme is first applied to lineat time-invariant (LTI) systems, then it is extended to a class of feedback linearizable systems. The convergence of the ESC scheme is guaranteed by the numerical optimization algorithm and state regulation. The robustness of line search methods and trust region methods is studied, which provides further flexibility for the design of robust extremum seeking controller. Simulation study of antilock braking systems (ABS) design via extremum seeking control is addressed. Index Terms—Antilock braking systems (ABS), extremum seeking, numerical optimization, state regulation.

I. INTRODUCTION RADITIONAL automatic control deals with the problem of stabilization of a known reference trajectory or set point, that is, so called “tracking” and “regulation” problem. The reference is often easily determined. However, in some occasions it can be very difficult to find a suitable reference value. For instance, the friction force coefficient of the wheel in a car depends on the slip. In order to maximize the friction force, it is necessary to apply a suitable braking torque to maintain the optimal slip, which changes as different road conditions and weather. Tracking a varying maximum or minimum (extremum) of a performance (output, cost) function is called extremum seeking control [1], which has two layers of meaning: first, we need to seek an extremum of the performance function; second, we need to be able to control (stabilize) the system and drive the performance output to that extremum. The goal of the extremum seeking control is to operate at a set point that optimizes the performance function. Extremum seeking control is related to optimization; many of the ideas used in extremum seeking control have been transferred from numerical optimization. Developments in computers and optimization have led to a renewed interest in extremum seeking control. Many investigations of extremum seeking control systems assume that the system is static, which can be justified if the time between the changes in the optimal

T

Manuscript received November 7, 2005; revised June 23, 2006. Recommended by Associate Editor A. Hansson. This work was supported by the Dayton Area Graduate Studies Institute (DAGSI). C. Zhang is with the Etch Engineering Technology, Applied Materials, Sunnyvale, CA 94085 USA (e-mail: [email protected]). R. Ordóñez is with the Department of Electrical and Computer Engineering, University of Dayton, Dayton, OH 45469 USA (e-mail: [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TAC.2007.892389

reference is sufficiently long. Then, the extremum seeking is reduced to an optimization problem in some sense. A mathewhere matical interpretation of the process is is a static output function, is a vector of parameters that may change with time. The goal to optimize the output function is essentially a numerical optimization problem given only the knowledge of the output measurements and its derivatives if available. The parameters can be the parameters of the controller or the controller itself. Thus the extremum seeking control of static systems is generally solved in two ways, one by tuning the parameters of the feedback controller, the other by tuning the control input directly [2]. A few references (for instance, see [3] and [4]) have approached problems where the plant is a cascade of a nonlinear static map and a linear dynamic system (the so-called Hammerstein and Wiener model). The first rigorous proof [5] of local stability of perturbationbased extremum seeking control scheme uses averaging analysis and singular perturbation, where a high-pass filter and slow perturbation signal are employed to derive the gradient information. Reference [6] presents a systematic description of the perturbation-based extremum seeking control and its applications. New progress in semiglobal stability appears in [7]. Gradient estimation-based extremum seeking control is studied in [8], nonlinear programming-based methods can be found in [9]. Extremum seeking control based on sliding mode are studied in [10]–[12], where time delay, excessive oscillation and performance improvement issues are addressed. Extremum seeking via continuous time nonderivative optimizers is proposed in [4]. An extremum seeking control problem is proposed and solved in [13] for a class of nonlinear systems with unknown parameters, where an explicit structure information for the performance function to be maximized is required. Moreover, many applications of extremum seeking have been studied recently, such as in optimizing bioreactor [14], combustion instability [15], antilock braking systems (ABS) design [16]–[18], electromechanical valve actuator [19], thermoacoustic cooler [20], and human exercise machine [21]. In this paper, we combine numerical optimization and state regulator to form an extremum seeking control scheme, where a numerical optimization algorithm provides search candidates of the unknown extremum and a state regulator is designed to regulate the state to where the search sequence leads to. A general problem statement is given in Section II. Two main types of numerical optimization algorithms, line search methods and trust region methods, are reviewed in Section III. In Section IV, a numerical optimization-based extremum seeking control scheme is proposed first to the LTI system, then it is extended to a class of feedback linearizable systems, where the convergence of proposed schemes is analyzed. Then, the robustness issues

0018-9286/$25.00 © 2007 IEEE

ZHANG AND ORDÓÑEZ: NUMERICAL OPTIMIZATION-BASED EXTREMUM SEEKING CONTROL WITH APPLICATION TO ABS DESIGN

455

whose constraint is the differential equation (1) as compared to the traditional algebraic constraints, and the manipulation of has to be done indirectly through the control input . The extremum seeking control problem then can be stated as: subject to Now the state is feasible if it is a solution of the dynamic system. In the case when (1) is controllable, there always exists in a finite time. an input that transfers to any where in Although controllable dynamic system constraints do allow to be anywhere in the state space where the numerical optimizer wants, the way in which reaches the particular place is determined by the dynamic system and the state regulator to be designed.

Fig. 1. Numerical optimization-based extremum seeking control.

are studied in Section V. Application to ABS design is considered in Section VI and finally some conclusions can be found in Section VII. II. PROBLEM STATEMENT In general, we consider the nonlinear model

III. UNCONSTRAINED OPTIMIZATION Two main methods for unconstrained optimization problem are reviewed in this section. A. Line Search Methods

(1) (2) where is the state, is the input, is the and are performance output, and sufficiently smooth functions on . Without loss of generality, we consider the minimization of the performance function (2). in this paper. The explicit For simplicity, we assume is unknown and hence not available for the design. form of The goal of extremum seeking control is to design a controller based on the output measurements and state measurements to , and thereregulate the state to an unknown minimum of fore minimize the performance output. Unlike optimal control, we do not suppose the knowledge of the performance function (2), therefore we cannot explicitly find the solution of the optimal condition and design an open-loop controller to operate the system at the optimal set point. A block diagram of numerical optimization-based extremum seeking control can be found in Fig. 1, where the nonlinear system is modeled as (1) and the performance function is (2). , Based on the measurements of the state, function values , the extremum seeking loop is expected to or gradient , and regulate the state as guided by the search sequence eventually minimizes the performance output. The following assumption is generally needed for the fulfillment of the extremum seeking purpose: Assumption 2.1 (Existence of the Minimum): The perforis continuous on the compact level sets mance function for all in . Assumption 2.1 guarantees the existence of the minimum, and any numerical optimization algorithms with first-order global convergence property will produce a sequence converging to a minimizer (strictly speaking, first-order stationary point) of the performance function. Moreover, from the perspective of optimization, extremum seeking control can be considered as a kind of constrained optimization problem,

Each iteration of a line search method computes a search direction and then decides how far to move along that direction. The iteration is given by (3) where the positive scalar is called the step length and requires to be a descent direction (one for which ), because this property guarantees that the function can be reduced along this direction. The steepest descent direction is the most obvious choice for search direction. Given a descent direction , we face a tradeoff in choosing that gives a substantial reduction of and not step length spending too much time making the choice. It can be found by approximately solving the following one-dimensional minimization problem: (4) An exact line search to find the optimal step length needs , which is expensive and sometimes the minimization of unnecessary. More practical strategies perform an inexact line search to identify a step length that achieves adequate reduction of with a small computational cost. In particular, the Armijo condition (5) prevents steps that are too long via a sufficient decrease criterion, while the Wolfe condition (6) prevents steps that are too short via a curvature criterion, with . The restriction ensures that acceptable points exist. Moreover, in order to avoid a poor choice of descent directions, an angle condition is set up to enforce

456

a uniform lower bound on the angle between vector , that is


and

(7) is independent of . The following is the first-order where global convergence1 result for line search methods [22], [23]. be continuously differentiable Theorem 3.1: Let and be bounded below. Suppose that is Lipschitz on continuous with constant ; that is, for all . If the sequence satisfies conditions (5), (6) and (7), then

The following lemma will be used in the robustness analysis for line search methods. be continuously differentiable Lemma 3.2: Let is Lipschitz continuous with constant on . Suppose that . Let be the step length and descent direction, then

where for exact line search, and for inexact line search satisfying conditions (5) and (6), represents and . the angle between vector Proof: See Appendix I. Since is required to ensure the feasibility . of inexact line search, we will have This observation is consistent for the upper bound results in the previous lemma. That is, we always expect that the exact line search achieves more decrease along the search direction than the inexact line search.

B. Trust Region Methods At each iteration of a trust region method, we consider the minimization of a model function instead of the objective function at current iterate . Because the model function may not be a good approximation of when is far away from , to a local we have to restrict the search for a minimizer of region involving . Such a region is called a trust region. A trust region method is defined as

model, the trial point is accepted as the new iterate and the trust region is centered at the new point and possibly enlarged. On the other hand, if the achieved reduction is poor compared with the predicted one, the current iterate is typically unchanged and trust region is reduced. This process is then repeated until convergence occurs. Define the ratio (9) The following algorithm [22] describes the process. Trust Region Algorithm , initialize the trust region Step 0 Given , and , set . Step 1 Approximately solve the trust region problem (8) to obtain . from (9). Step 2 Evaluate Step 3 If ; if and ; else . , else . Set Step 4 If . Go to step 1. Quadratic approximations of are often used for constructing . In this case, in (8) can be formed as (10) is either the gradient or an approxiThe vector is either the Hessian matrix mation of it, and the matrix or an approximation of it. Thus, such construction of still requires gradient information. However, the trust region framework provides large flexibility in designing derivative free optimization methods. This compares very favorable with most line search methods which do require gradient measurements of the objective function. Derivative free trust region algorithms proposed in [24]–[26] use multivariate interpolation , where only an interpolato construct the model function tion set containing the interpolating nodes and their objective function values are needed. Overall, trust-region methods retain the quadratic convergence rate while being globally convergent. The following is a global convergence result for trust region methods [22]. be Lipschitz continuously difTheorem 3.3: Let . ferentiable and bounded below on level set in the trust region algorithm. Suppose that Further, let for some constant , and that all approximate solutions of (8) satisfy the inequality

(8) be the minimizer obtained. The current iterate is then Let , if the achieved objective function reducupdated to be tion is sufficiently compared with the reduction predicted by the 1In the optimization community, first-order convergence of an optimization algorithm means that one (or some, or all) of the limit points of the iterate sequence is a stationary point of J (x). Global convergence is used to mean first-order convergence from an arbitrary starting point. In contrast, local convergence is used to mean convergence when the initial point is close enough to a stationary point.

for some constant . Then stant

, and

for some con-


IV. EXTREMUM SEEKING CONTROL A. ESC of LTI Systems Here, we consider a single-input–single-output (SISO) linear time-invariant (LTI) system (11) is with the performance function defined in (2), where is the input. The matrices are given as a the state, model of a real system. However, the explicit form of the perand its minimum are not known, where formance function we can only measure the function value or its derivatives. We need the following assumption to ensure the feasibility of extremum seeking control for the LTI system (11). Assumption 4.1: The LTI system (11) is controllable. Now, we can combine an optimization algorithm and a state regulator originated from the controllability proof in [27] to form an extremum seeking control scheme. ESC Scheme for LTI Systems Step 0 Given , set and . Step 1 Use an optimization algorithm with first-order based on current global convergence to produce state , the measurement of or . Denote

Step 2 Choose a regulation time , let and design the control input during

, to be

(12) where (13) Step 3 If

, then stop. Otherwise, set . Go to Step 1. Remark 4.2: The above extremum seeking control scheme can be derivative free if the optimization algorithm used in Step 1 does not require gradient information. For example, we can use derivative free trust region methods [24], [26] or direct search [23]. Some modifications of the above scheme are required in order to use the trust region methods due to the need to obtain the ratio in (9), where we may need additional regulation time to drive the state back to if . Please refer to [28] for the details of trust region-based extremum seeking control. Remark 4.3: If steepest descent method is used in Step 1, we will have Even though it requires gradient measurement by using steepest descent method, such measurement is only

457

time, therefore, we can estimate the gradient needed every by collecting enough measurements of during the time. is Remark 4.4: The stopping criterion used only for simplicity, where is a predefined small positive constant. In case gradient information is not available, there are other stopping criteria only based on the difference of function values [22]. Now, we present the convergence result of the extremum seeking control scheme that follows. Theorem 4.5: Consider the LTI system (11) with performance output (2), suppose the LTI system (11) satisfies 4.1 and the performance function (2) satisfies Assumption 2.1; moreover, if the extremum seeking control scheme above is applied, where the optimization algorithm used is of first-order global will globally asymptotically convergence, then the state converge to the first-order stationary point of the performance function (2). Proof: See Appendix II. Remark 4.6: The convergence result for the extremum seeking control scheme is global since the numerical optimization algorithm used is of first-order global convergence. Remark 4.7: Additional assumptions about the performance function may be required to guarantee an arbitrary optimization algorithm with first-order global convergence indeed converges to the stationary point of (2). For example, according to Theorem 3.1, we need assume is continuous differentiable and is Lipschitz continuous for line search methods; also we will assume is Lipschitz continuously differentiable and bounded below on level set when using trust region methods. Remark 4.8: The design of controller (12) is not limited to single input system and just one way to fulfill the state regulation task. It is an open loop controller during the time without , which is fed back from the opticonsidering the change of mization algorithm. Such design has the advantage of achieving regulation in a finite time, but it relies on the precise knowledge of and matrices, and is very difficult to robustify. Later on, we can relax the state regulation design criterion from perfect , regulation to regulation within certain neighborhood of which provides further flexibility of using other designs of state regulator to deal with input disturbance or unmodeled plant dynamics. Remark 4.9: The only requirement for the feasibility of state of regulator (12) is that the LTI system is controllable ( (13) is nonsingular), which means the extremum seeking control scheme works for both stable and unstable systems. However, we would like to first stabilize the unstable system by pole placement, then design a state regulator on the stabilized LTI system since the unstable LTI system will amplify the regulation error resulting from an input disturbance, for example. Remark 4.10: The performance of the extremum seeking control design largely depends on the performance function to be optimized, the optimization algorithm used, and a robust and efficient state regulator. Remark 4.11: If the performance function (2) is differentiable and convex, then the convergence to stationary point becomes convergence to the global minimum [22].

458


, and The banana function (15) has its minimizer at . The explicit form of the function and its minimum are both unknown to the designer. Extremum seeking control scheme based on steepest descent algorithm with inexact line search is applied, the simulation results with the first 15 steps . Hence, we are shown in Fig. 2, where every time to only require the gradient measurement at continue the optimization algorithm. The performance output and the [Fig. 2(a)] approaches its minimum at . state [Fig. 2(b)] accordingly converges to the minimizer as The steepest descent algorithm produces a sequence a guideline for the controller. The trajectory between and is shaped by the dynamical system (11) and the regulator (12). This can be viewed clearly in Fig. 2(d),2 where the blue and the red dashed line represents the circle represents the state trajectory. The choice of is rather heuristic in this example. However, in practice this is an important design factor. We can see that a smaller yields a larger control force to satshould be chosen such isfy the short regulation time. Thus, that the control force does not exceed the practical limits. There is always a tradeoff between the extremum seeking time and the control gain. B. ESC of State Feedback Linearizable Systems Now, we consider a SISO nonlinear affine system (16) is with the performance function defined in (2), where is the input, are smooth functhe state, tions on . We have the following assumption for the nonlinear affine system. Assumption 4.12: The nonlinear affine system is state feedback linearizable on the domain . Upon this assumption, we can always put the system in the controllable canonical form [29]. That is, there exists a diffeosuch that contains the morphism transforms the origin and the change of variables system (16) into the form (17)

Fig. 2. Extremum seeking control for a LTI system. (a) Performance output. (b) State. (c) Control input. (d) Phase portrait, steepest descent sequence fx g over the contour of the performance function.

Now, consider a second-order stable LTI system in its controllable canonical form. Let ,

with controllable and nonsingular for all . Then, we can easily extend the results on LTI systems to outline the extremum seeking control scheme for the state feedback linfor simearizable systems (16), where we still assume plicity. ESC Scheme for State Feedback Linearizable Systems and . Step 0 Given , set . Step 1 . Step 2 Step 3 Choose a regulation time , let , to be and design the control input during (18)

(14) (15)

2Since it will take thousands of steps for the steepest descent algorithm to converge to the minimizer of the banana function, we only simulate the first 15 steps for illustration purpose.


where

(19) , then stop. Otherwise, set . Go to step 1. Theorem 4.13: Consider the nonlinear affine system (16) with performance output (2), suppose the system (16) satisfies Assumption 4.12 and the performance function (2) satisfies Assumption 2.1; moreover, if the extremum seeking control scheme above is applied, where the optimization algorithm used is of first-order global convergence, then the state will globally asymptotically converge to the first-order stationary point of the performance function (2). The proof mainly follows the proof of Theorem 4.5. The feasibility of the controller defined in (18) is guaranteed by Assumption 4.12. Similar remarks like 4.2–4.11 also apply here. Step 4 If

C. ESC of Input-Output Feedback Linearizable Systems We can extend the previous results on state feedback linearizable systems to input-output feedback linearizable systems given some minor modifications. We will have the following assumption. Assumption 4.14: The nonlinear affine system (16) is input–output feedback linearizable from input to output on . We define (20) is sufficiently smooth in the domain . The where motivation of defining a new output is to retain the claim that in the extremum seeking control we do not have the knowledge of the performance function, while we do need the knowledge to perform input–output linof some suitable output earization. Let be the relative degree of the nonlinear system , a neighborhood of and a (20), then for every exists such that the change of diffeomorphism variables , transforms the system (20) into the form (21) (22) (23) is controllable and is where nonsingular in . Since is uncontrollable, therefore, in order to fulfill the extremum seeking of the performance function, two more assumptions are proposed. Assumption 4.15: The performance function is not dependent on the state of the internal dynamics. Assumption 4.16: The state of the internal dynamics (21) will be bounded given bounded and any initial state . The Assumption 4.15 puts the performance function as

459

It is a reasonable assumption since we have to be able to control the variables of the performance function to achieve the extremum seeking. Moreover, Assumption 4.16 simply means that the internal dynamics are well behaved such that the linearizing controller will not grow unbounded due to the existence of the uncontrollable state . Note that assuming input-to-state stability of the internal dynamics is stronger than Assumption 4.16. are However, simply assuming the zero dynamics asymptotically stable is also not enough since the state may grow unbounded given bounded input . Thus, the same analysis for the state feedback linearizable systems holds here given the extremum seeking scheme in Section IV-B with minor modifications, where we replace with . The following theorem is a straightforward extension of Theorem 4.13. Theorem 4.17: Consider the nonlinear affine system (16) with performance output (2), suppose the system (16) satisfies Assumptions 4.14 and 4.16, and the performance function (2) satisfies Assumptions 2.1 and 4.15; moreover, if the extremum seeking control scheme of Section IV-B is applied with replaced by , then the state will globally asymptotically converge to the first-order stationary point of the performance function (2). The proof mainly follows the proof of Theorem 4.5. The feasibility of the controller defined in (18) is guaranteed by Assumptions 4.14 and 4.16. Similar remarks like 4.2–4.11 also apply here. V. ROBUSTNESS ISSUES The main restriction of Theorems 4.5, 4.13, and 4.17 is the requirement of perfect state regulation to guarantee the convergence. That is, at each iteration, the controller needs to regulate , which is offered the state precisely to the desired set point by the iterative optimization algorithm based on the current state . In practical applications, noisy output or state measurements, input disturbance, saturation and time delay, unmodeled plant dynamics and computational error will be detrimental to the theoretical result. In fact, we are only able to regulate the . state to the neighborhood of the set point For example, let us consider an LTI system with input dis, where is given as turbance. Let , the state and in (12), then at time , then at time , we will have

where . However, the error will not accumulate since the optimization algorithm will generate the next destination based on current state . That is, by including the numerical optimizer in the extremum seeking loop, it offers a feedback mechanism to robustify the extremum seeking scheme. Denote , then by induction,

460


we will have the controller interpolates precisely between the , where sequence

given (24)

will be bounded if is bounded and the input disturbance is bounded. Moreover, for stable systems, we will have matrix with negative eigenvalues and therefore the transition matrix has will asympexponentials with negative real parts. Then, approaches infinity, totically converge to some constant as that is, even if is unbounded, we will still have a bounded given bounded input disturbance. Moreover, the more negwill be. On the conative the eigenvalues are, the smaller trary, for unstable systems, the transition matrix will amplify , and will grow as even a small input disturbance increases. Therefore, we wish to have a stable LTI system and a short regulation time. The latter amounts to a need of high gain controller to deal with disturbance. Consider extremum seeking control of an unstable but still controllable LTI system, we will perform pole placement to transform the unstable LTI system to a desired stable LTI system, and then design the state regulator on the stabilized LTI system. Similarly, for state feedback linearizable systems given the which are in the controllable canonical form, knowledge of and , the controller will be implementing functions , and , that is approximations of

where comes

is defined in (19). Then the closed-loop systems be-

where . Then, the imperfect modeling is equivalent to having input disturbance in the resulting linear system, therefore we need to deal with imperfect regulation as well. Overall, we will hope that a well designed optimization algorithm will convey its robustness to the extremum seeking scheme. That is, if the new sequence is able to converge to the minimum or its small neighborhood given the error is bounded, then the extremum seeking control schemes in Section IV will converge to the minimum or its small neighborhood as well. In the following, we will present the robustness analysis of two types of optimization algorithms. A. Robustness of Line Search Methods Theorem 5.1: Let be continuously differentiable and be bounded below. Suppose that is Lipschitz on continuous with constant . A line search method starting from is used but with bounded error at each iteration, i.e., and . The new sequence is a descent sequence, that is

where for exact line search and for inexact line search satisfying conditions (5) and (6). Proof: See Appendix III. Although the bound (24) is very conservative, it can give us some insights into the robustness of line search methods. First, the exact line search allows a larger error bound than the inexact line search. Second, we can see that the bound is an increasing . That is, when is far away from the function of minimizer of the performance function, we will expect the gradient to be large and thus the error the algorithm can tolerate is also large. This observation implies that line search methods will be very robust until the gradient converges to some invariant set, which is illustrated in the following corollary. be continuously differCorollary 5.2: Let and be bounded below. Suppose is Lipsentiable on chitz continuous with constant . A steepest descent algorithm is used with bounded error at each iteration. If assuming for some constant , then we will have the gradient of the converges to the invariant set satisfying sequence (25) where for exact line search and for inexact line search satisfying conditions (5) and (6). Proof: See Appendix IV. Observed from inequality (25), a diminishing step is preferred later on to decrease the bound of length the invariant set. As , the bound converges to . This is again coincident with theory of numerical optimization, where generally a diminishing step length is required for the algorithms to converge and , we to a minimum. And if there is no error between will see that the gradient converges to zero. Moreover, exact line search can achieve a smaller bound than the inexact line search. Now, we continue the simulation in Section IV-A on the LTI system (14) with performance function (15), a random disturbance uniformly distributed with amplitude 0.8 is introduced to the input. The simulation results are shown in Fig. 3 with . The controller (12) is not able to regulate the state to the desired set point precisely. For example, at the first step, the controller cannot transfer the state to the desired , instead it arrives at destination due to the input disturbance. Then, , that is, the line search method tries to amend the deviated path towards the minimum. , thereAgain, the state only arrives at fore, eventually we will still have a descent sequence as long as the error satisfies the bound (24). The comparison of and can be seen in Fig. 3(d), where the blue circle , magenta square denotes the and the represents the red dashed line is the state trajectory. Interestingly, we find that the disturbance actually helps the algorithm to achieve more reduction in function values in the first few steps by comparing with the ideal case [see Figs. 2(d) and 3(d)]. However, as shown


461

we can do given the current design of state regulator (12) with input disturbance. B. Robustness of Trust Region Method For trust region methods, the ratio in (9) is required at each . Let us call th step a successful step if step to determine , and unsuccessful step if . However, in order to obtain , we need the measurement of . Then, in the implementation of extremum seeking control based on trust region method, this means we need to regulate the state to to obtain measurement of even if it is not the next iterate . For example, at th step, the controller to to obtain the ratio needs to regulate the state . If , then it is a successful step and we are done. Oth, then we need to regulate the state back to erwise, if since it is an unsuccessful step. However, for practical applicacannot be obtained due to the imperfect tions, the ideal ratio regulation. The following theorem provides one possible quantitative analysis about the convergence of trust region methods under certain bounded error. be Lipschitz continuously difTheorem 5.3: Let . ferentiable and bounded below on level set is Lipschitz continuous with constant . A Suppose that trust region method starting from is used but with bounded error at each iteration, that is, for a successful step or for an unsuccessful step. If, for every successful step (26) and for every unsuccessful step (27) where

and , then there converges to the first-order staexists a subsequence of . tionary point of Proof: See Appendix V. C. Design of Robust Extremum Seeking Control Scheme

Fig. 3. Extremum seeking control for a LTI system with input disturbance. (a) Performance output. (b) State. (c) Control input. (d) Phase portrait, steepest descent sequence fx g over the contour of the performance function.

in Fig. 3(a), the performance output becomes diverging eventually since the state regulator design (12) with input disturbance cannot regulate the state even into the desired neighborhood (24) as is very small. Therefore, in such cases, we of will prefer using (25) as the stopping criterion, which is the best

The importance of Theorems 5.1 and 5.3 is that they relax the design criterion for the state regulator, where now no peris needed. Robust design of state fect state regulation to regulator to deal with disturbance, unmodeled plant dynamics via adaptive control [30] or sliding mode control can be made , which easier by regulating state to a neighborhood of will be presented in our future work. For extremum seeking control based on line search methods, as long as we can design a regulator satisfying (24), then the extremum seeking will continue to decrease the performance output. For extremum seeking control based on trust region methods, , we do not know whether it will be a successful at time step or not in advance since we do not have the knowledge of .

462


Fig. 4. Design criterion for the robust regulator to satisfy (26) and (27) of the trust region-based extremum seeking control.

Then, one way to guarantee the convergence under input disturbance is the following. The state regulator for set point needs to be designed to satisfy (26). If it is a successful step, then the design will guarantee the step still to be a successful (see (34) in Appendix V). If it is not a sucstep since , then there is no guarantee that cessful step, that is will be greater than . So if , then we are luck in the sense that we are on the right way due to the disturbance. If not, then we need to regulate the state back to , that means we need to to satisfy (27), which means we can design the regulator for return to the right path as inferred from previous steps. Both inequalities (26) and (27) define similar criteria for the design of is state regulators. A pictorial illustration of such criteria in in (26) and shown in Fig. 4, where we let the set point, in (27), to be and its gradient to be , the Lipschitz constant . The blue star is the desired destination, and the dotted area depicts the acceptable region satisfying inequalities (26) and (27). The size of the region is proportional to the gradient of the set point, and the reciprocal of the Lipschitz constant. VI. ABS DESIGN VIA EXTREMUM SEEKING CONTROL In this section, we apply the extremum seeking control to solve the ABS design problem [6]. It is well known that the ABS is used to deal with braking on slippery surfaces. The friction force has a maximum for a low wheel slip and decreases as the slip increases. Our goal is to design a control algorithm to achieve maximal friction force without prior knowledge of the

optimal slip. Consider only one wheel model, where the tire dynamics are described as in [16] (28) (29) is the linear velocity and is the angular velocity of where the mass, the weight of the wheel, the wheel, the radius of the wheel, the moment of inertia of the wheel, the braking friction torque, the braking torque, is the friction force coefficient and the wheel slip is defined as (30) for . There exists a maximum for the friction force at the optimal slip , but will change coefficient as the road conditions changes. Now, the purpose of the ABS design is to generate a control input such that the friction force coefficient is maximized. is not controllable from By observing (28), we find that given fixed. Fortunately, the friction force coefficient is only dependent on , which is a function of and , and it may be controllable by . Therefore, we try to linearize the system from the input to output . We define the change that transforms the of variables system (28), (29) into the form (31)

(32)


463

Fig. 5. ABS design via numerical optimization-based extremum seeking control.

Since is the linear velocity, it will be bounded all the time due to physical restrictions. Moreover, even though the knowledge is not available, we are able to obtain the measurement of of from (28) given the linear acceleration is measurable , let the breaking via an accelerometer. Then, given torque be (33) Then (32) becomes

where

is the regulator designed according to (19)

coefficient is only regulated to the neighborhood of the maximum since no robust controllers are used to guarantee the convergence. In both cases, the wheel is stopped within the same time. Similarly, the delay introduced by hydraulic or pneumatic line dynamics may result in regulation error as well and, therefore, a robust term is needed in the state regulator to address the delay problem, which is much easier than robustifying perturbation-based or sliding mode-based extremum seeking design. Moreover, oscillation is successfully avoided compared with those methods because no perturbation signal or sliding mode function is introduced. However, ABS design via perturbation-based or sliding mode-based extremum seeking control are much easier to tune. Extremum seeking control based on line search methods achieves comparable performance.

VII. CONCLUSION and is offered by the optimization algorithm used in the extremum seeking loop. A block diagram of extremum seeking scheme for the wheel model can be found in Fig. 5. That is, by designing the control torque as in (33), we are able to adjust the slip to maximize the friction force coefficient. For simulation purpose, we postulate a simple function which qualitatively as in [6] matches

where are chosen as: conditions are makes

and

. The parameters of the wheel m. Initial m/s, , which . The parameters of extremum seeking loop: . The simulation results of extremum seeking control based on trust region method are shown in Figs. 6 and 7, the latter where the input is disturbed by a uniformly distributed noise with amplitude 2. Without disturbance, the extremum seeking scheme maximizes the friction force coefficient. Due to the existence of the disturbance, the friction force kg,

In this paper, we successfully incorporate numerical optimization algorithms into the set up of an extremum seeking control scheme. The convergence of the proposed extremum seeking control scheme is guaranteed if the optimization algorithm is globally convergent and with appropriate state regulation. We also analyze the robustness of line search methods and trust region methods, which relaxes the design requirement for the state regulator and provides further flexibility in designing the robust extremum seeking control scheme. Numerical examples are given to illustrate the analysis results. Furthermore, an application of ABS design via extremum seeking control is used to illustrate the feasibility of the proposed scheme. The current setting of the proposed extremum seeking control scheme is one attractive way to use the numerical optimization for the purpose of real time optimization, as it retains the global convergence property of the numerical optimization algorithm. It allows us to utilize the research results from the optimiza, it becomes a tion community. For example, when constrained extremum seeking control problem, and generally we need to first resort to constrained optimization algorithms

464

Fig. 6. ABS design via trust region-based extremum seeking control. (a) Friction force coefficient. (b) Slip. (c) Linear velocity and angular velocity. (d) Braking torque.

and the state regulator design will be more challenging in ensuring the state will not violate the constraints during the transient. In the future, the exploration of more robust numerical optimization algorithms and the design of robust state regulators will be two ways to enhance the robustness of the extremum


Fig. 7. ABS design via trust region-based extremum seeking control with input disturbance. (a) Friction force coefficient. (b) Slip. (c) Linear velocity and angular velocity. (d) Braking torque.

seeking control schemes. Moreover, the design of the regulation time needs to be further studied to deal with various requirements originated from practical applications. Current design also wastes a lot of information by setting the numerical optimizer to work discretely. Similar to the perturbation-based [6] or sliding mode-based [12] extremum seeking control, other analog optimization algorithms are in need of attention.


APPENDIX I PROOF OF LEMMA 3.2 We first present the following result. Lemma 1.1 (Descent Lemma [31]): Let continuously differentiable on . Suppose that continuous with constant . Then, for

By induction, at then we obtain , we will have

465

, we suppose the state . At time

,

be is Lipschitz

We now proceed to establish Lemma 3.2. First, for exact line is the solution of (4). From descent Lemma 1.1, we search, have valid for all . Letting , then

where is defined in (12). Thus, the controller (12) interpoprecisely within finite time . Therelates between the fore, the state of the system will globally asymptotically converge to the first-order stationary point of performance function (2).

APPENDIX III PROOF OF THEOREM 5.1 Now, for line search method at step

, we have

Second, for inexact line search, satisfies conditions (5) and (6). From the Lipschitz condition, we have . Then, from (6), we have . . Finally,

That is, from (5)

where

where the second row is obtained via Descent Lemmas 1.1, and the third row is achieved based on Lemma 3.2. Then, if we have

.

APPENDIX II PROOF OF THEOREM 4.5 First, given the performance function (2) satisfies Assumption (2.1), an optimization algorithm with first-order global conthat globally vergence will produce a search sequence asymptotically converges to the first-order stationary point of the performance function (2). By assuming the LTI system (11) is controllable, the conis nonsingular. First, at troller (12) is feasible since , the state , we have . Then, at time , we will have

where the control input during

is

we can obtain

.

APPENDIX IV PROOF OF COROLLARY 5.2 for steepest descent algorithm, and from We have inequality (24), as long as

we will always have . So we can find a congiven the error bound . servative bound on For steepest descent method, we have . Then, from Lipschitz condition

466


Since

is always positive, we need

Now, the bound can be found via Now, define , that is

to be the angle between

and

. Then

Thus, we will have the gradient of the sequence to the invariant set satisfying

converges

Case II, for an unsuccessful step, , then we do not . need to care about the regulation error to the set point , then it regulates the state to a place If the resulting with lower performance function value, which actually helps , then we need to regulate the optimization process. If the state back to . Here, we redefine the regulation error as an error from this second state regulation, i.e., . Therefore, we wish to be a successful step compared with the previous one , i.e.,

APPENDIX V PROOF OF THEOREM 5.3 , we obtain approximately solving At iteration and the trust region subproblem (8). And ideally we would have the to regulator such as (12) to drive the state from to obtain an ideal ratio (9). However, due to the input disturbance or model uncertainty, such controller can only regand end up with a practical ulate the state to ratio (34) Let such that up to

be a subsequence of represents the index set of successful steps , that is and

Now, define is,

to be the angle between

and , that . Then

(35) which means is a successful step if started from . Two cases need to be analyzed to guarantee the global convergence . of the sequence Case I, for a successful step, , then we want to , that is we guarantee a successful step in the presence of wish

Thus, if (26) and (27) are satisfied, we will have a descent subof globally converges to the first-order sequence . stationary point of ACKNOWLEDGMENT The authors would like Prof. R. Liu and Prof. M. Maggiore for their valuable suggestions. REFERENCES [1] K. J. Åström and B. Wittenmark, Adaptive Control. Boston, MA: Wiley, 1994. [2] Y. Zhang, “Stability and performance tradeoff with discrete time triangular search minimum seeking,” in Proc. Amer. Control Conf., 2000, vol. 1, pp. 423–427.


[3] J.-Y. Choi, M. Krstic´ , K. B. Ariyur, and J. S. Lee, “Extremum seeking control for discrete-time systems,” IEEE Trans. Autom. Control, vol. 47, no. 2, pp. 318–323, Feb. 2002. ˙ [4] Nusawardhana and S. H. Zak, “Extremum seeking using analog nonderivative optimizers,” in Proc. Amer. Control Conf., 2003, vol. 4, pp. 3242–3247. [5] M. Krstic´ and H.-H. Wang, “Design and stability analysis of extremum seeking feedback for general nonlinear systems,” Automatica, vol. 36, no. 2, pp. 595–601, 2000. [6] K. B. Ariyur and M. Krstic´ , Real-Time Optimization by ExtremumSeeking Control. Hoboken, NJ: Wiley-Interscience, 2003. [7] Y. Tan, D. Nesic, and I. Mareels, “On non-local stability properties of extremum seeking control,” in Proc. 16th IFAC World Congr., 2005, pp. 1–6. [8] R. N. Banavar, D. F. Chichka, and J. L. Speyer, “Convergence and synthesis issues in extremum seeking control,” in Proc. Amer. Control Conf., 2000, vol. 1, pp. 438–443. [9] A. R. Teel and D. Popovic´ , “Solving smooth and nonsmooth multivariable extremum seeking problems by the methods of nonlinear programming,” in Proc. Amer. Control Conf., 2001, vol. 3, pp. 2394–2399. [10] H. Yu and Ü. Özgüner, “Extremum-seeking control via sliding mode with periodic search signals,” in Proc. 41st Conf. Decision Control, 2002, vol. 1, pp. 323–328. [11] H. Yu and Ü. Özgüner, “Smooth extremum-seeking control via second order sliding mode,” in Proc. Amer. Control Conf., 2003, vol. 4, pp. 3248–3253. [12] Y. Pan, Ü. Özgüner, and T. Acarman, “Stability and performance improvement of extremum seeking control with sliding mode,” Int. J. Control, vol. 76, no. 9/10, pp. 968–985, 2003. [13] M. Guay and T. Zhang, “Adaptive extremum seeking control of nonlinear dynamic systems with parametric uncertainties,” Automatica, vol. 39, no. 7, pp. 1283–1293, 2003. [14] M. Guay, D. Dochain, and M. Perrier, “Adaptive extremum seeking control of continuous stirred tank bioreactors with unknown growth kinetics,” Automatica, vol. 40, no. 5, pp. 881–888, 2004. [15] A. Banaszuk, K. B. Ariyur, M. Krstic´ , and C. A. Jacobson, “An adaptive algorithm for control of combustion instabilityand application to compressor instability control,” Automatica, vol. 40, no. 11, pp. 1965–1972, 2004. [16] S. Drakunov, Ü. Özgüner, P. Dix, and B. Ashrafi, “Abs control using optimum search via sliding modes,” IEEE Trans. Control Syst. Technol., vol. 3, no. 1, pp. 79–85, Jan. 1995. [17] H. Yu and Ü. Özgüner, “Extremum-seeking control strategy for ABS system with time delay,” in Proc. Amer. Control Conf., 2002, vol. 5, pp. 3753–3758. [18] ˙I. Tunay, “Antiskid control for aircraft via extremum-seeking,” in Proc. Amer. Control Conf., 2001, vol. 2, pp. 665–670. [19] K. S. Peterson and A. G. Stefanopoulou, “Extremum seeking control for soft landing of an electromechanical valve actuator,” Automatica, vol. 40, no. 6, pp. 1063–1069, 2004. [20] Y. Li, M. A. Rotea, G. T.-C. Chiu, L. G. Mongeau, and I.-S. Paek, “Exetremum seeking control of tunable thermoacoustic cooler,” in Proc. Amer. Control Conf., 2004, pp. 2033–2038. [21] X. T. Zhang, D. M. Dawson, W. E. Dixon, and B. Xian, “Extremum seeking nonlinear controllers for a human exercise machine,” in Proc. 43rd Conf. Decision Control, 2004, pp. 3950–3955. [22] J. Nocedal and S. Wright, Numerical Optimization. New York: Springer-Verlag, 1999. [23] T. G. Kolda, R. M. Lewis, and V. Torczon, “Optimization by direct search: New perspectives on some classical and modern methods,” SIAM Rev., vol. 45, no. 3, pp. 385–482, 2003.

467

[24] A. R. Conn, K. Scheinberg, and P. L. Toint, “Recent progress in unconstrained nonlinear optimization without derivatives,” Math. Program., vol. 79, pp. 397–414, 1997. [25] A. R. Conn, K. Scheinberg, and P. L. Toint, “On the convergence of derivative-free methods for unconstrained optimization,” Approx. Theory Optim.: Tributes to M. J. D. Powell, pp. 83–108, 1997. [26] M. J. D. Powell, Uobyqa: Unconstrained Optimization by Quadratic Approximation Univ. Cambridge, Cambridge, U.K., Tech. Rep. NA2000/14, DAMTP, 2000. [27] C.-T. Chen, Linear System Theory and Design. New York: Oxford Univ. Press, 1999. [28] C. Zhang and R. Ordóñez, “Non-gradient extremum seeking control of feedback linearizable systems with application to abs design,” in Proc. Conf. Decision Control, 2006, pp. 6666–6671. [29] H. K. Khalil, Nonlinear Systems. Upper Saddle River, NJ: PrenticeHall, 2001. [30] J. T. Spooner, M. Maggiore, R. Ordóñez, and K. M. Passino, Stable Adaptive Control and Estimation for Nonlinear Systems, Neural and Fuzzy Approximator Techniques. New York: Wiley, 2002. [31] D. P. Bertsekas, Nonlinear Programming. Belmont, MA: Athena Scientific, 1995.

Chunlei Zhang (M’01) received the B.S. degree from the Nanjing University of Science and Technology, Nanjing, China, in 1999, the M.S. degree in electrical engineering and the M.S. degree in applied mathematics from the University of Dayton, Dayton, OH, in 2003 and 2005, respectively, and the Ph.D. degree in electrical engineering from the University of Dayton, in 2006. He joined the Etch Control Engineering Group at Applied Materials, Sunnyvale, CA, in 2006, as a System Design Engineer. His research interests include extremum seeking control and its applications, cooperative control of multiple autonomous agents and control systems design, and implementation in semiconductor industry. Dr. Zhang is a member of the IEEE Control Systems Society (CSS).

Raúl Ordonez (S’96–M’99) received the M.S. and Ph.D. degrees in electrical engineering from the Ohio State University, Columbus, in 1996 and 1999, respectively, specializing in controls research, and minoring in mathematics and signal processing. Prior to joining the University of Dayton, Dayton, OH, where he is currently an Associate Professor of Electrical and Computer Engineering, he served for two years as a Faculty Member at Rowan University, Glassboro, NJ. He is a coauthor (with J. T. Spooner, M. Maggiore, K. Passino) of the book Stable Adaptive Control and Estimation for Nonlinear Systems: Neural and Fuzzy Approximator Techniques (New York: Wiley, 2002). Dr. Ordonez is a member of the IEEE Control Systems Society (CSS), where he serves as Associate Editor for Conference Activities. He was Publicity Chair for the 2001 IEEE International Symposium on Intelligent Control, and a member of the Program Committee for the 2001 IEEE Conference on Decision and Control. He is currently serving as Associate Editor for the journal Automatica.