CDC 2003

1

Communication Delay and Instability in Rate-Controlled Networks Priya Ranjan, Eyad H. Abed, and Richard J. La Abstract— We analyze the optimization framework for rate allocation problem proposed by Kelly and characterize the oscillations in terms of underlying map’s period two orbit in the instability regime with arbitrary communication delay. We show the oscillations can be fairly large by computing the bounds explicitly for large return trip times.

I. I NTRODUCTION With the unprecedented growth and popularity of the Internet the problem of rate/congestion control is emerging as a more crucial problem. Poor management of congestion can render one part of a network inaccessible to the rest and significantly degrade the performance of networking applications. Kelly has proposed an optimization framework for rate allocation in the Internet [7]. Using the proposed framework he has shown that the system optimum is achieved at the equilibrium between the end users and resources. Based on this observation researchers have proposed various rate-based algorithms that solve the system optimization problem or its relaxation [7], [11], [12]. However, the convergence of these algorithms has been established only in the absence of feedback delay, and the impact of feedback delay has been left open as well as any trade-off that may exist between stability and selected utility and cost functions. In particular the issue of stability has not been systematically investigated with large communication delays. In our earlier [15] we have established a global stability criterion for system optimization problem in the presence of arbitrary delay for simple one resource problem with multiple homogeneous flows. This stability condition was derived using the invariance-based global stability results for nonlinear delaydifferential equations [13], [5], [4], [3]. This kind of global stability results are different from that based on Lyapunov or Razumikhin theorems in the sense that in our approach also provides us with insight on the structure of emerging periodic orbits, e.g., their periodicity and amplitude, in the case of loss of stability. This loss of stability and characterization of ensuing periodic orbit are main topics of this paper. Generally speaking, our results tell us that if the user and resource curves have a stable market equilibrium, then corresponding dynamical equation for flow-optimization will converge to the optimal solution in the presence of arbitrary delay. This result essentially shows that stability is related to utility and price curves in a fundamental way. In particular, for a given price curve, it possible to design stable user utility functions such that the corresponding dynamical system converges to the optimal flow irrespective of communication delay. Conversely, if the underlying market equilibrium is unstable then it is possible to find a large enough delay for which the optimal The authors are with the Department of Electrical and Computer Engineering and the Institute for Systems Research, University of Maryland, College Park, MD 20742 USA. Email: [email protected]

point loses its stability and gives way to oscillations. This paper studies the oscillatory orbits by explicitly giving the bounds on their amplitude. It is also shown that these bounds are derived from an underlying discrete time map which goes through a period doubling bifurcation with the loss of stability. These results provide an interesting perspective for designing end user algorithms and active queue management (AQM) mechanisms. It is also worth noting that in general characterizing the exact conditions for stability with a delay is difficult. Hence, our result provides a simple and yet robust way of dealing with the problem of widely varying feedback delay in communication networks through a clever choice of the user utility function and price functions. This paper is organized as follows. Section II describes the optimization problem for rate control. Relevant previous work on the stability criterion of a system given by a delayed differential equation is given in Section III. Our main results on instability are presented in Section IV, which is followed by numerical examples in Section VI. We conclude the paper in Section VII. II. BACKGROUND In this section we briefly describe the rate control problem in the proposed optimization framework. Consider a flow traversing a single resource. The rate control problem can be formulated as the following net utility optimization problem from the end user’s point of view [7]: max x

U (x) x

s. t.

x p(x)

(1)

C

where x is the rate, U (x) is the utility of the user when it receives a rate of x, p(x) is the price per unit flow the user has to pay when the rate is x, and C is the capacity of the resource. The proposed end user algorithm in the absence of delay is given by the following differential equation [9]. d dt

x(t) = k (w(t)

x(t)(t))

(2)

where w(t) is the price per unit time user is willing to pay, (t) = p(x(t)), and k; k > 0, is a gain parameter. The case where w(t) is a fixed constant, i.e., U (x) = log (x), is studied in [8]. In this paper we assume that w(t) = x(t) 0 U (x(t)) with any monotonically decreasing concave family of utility functions [9] with their decrease rate smaller than x1 . Now, suppose that congestion signal generated at the shared resource, i.e., p(x(t)), is returned to the user after a fixed and common round trip time T . In the presence of delay the interaction is given by the following delayed differential equation d dt

x(t) = k (w(t)

x(t

T )(t

T ))

(3)

CDC 2003

2

=k

0

x(t)U (x(t))

x(t

T )p(x(t

After normalizing time by T and replacing t becomes: d T ds d ds

T)

= s

(4)

T , eq. 4

0

x(s) = k x(s)U (x(s)) 0

x(s) = x(s)U (x(s))

x(s

x(s

1)p(x(s

1)p(x(s

1))

1))

(5) (6)

where = T1k . It is precisely eq. 6 we are interested in from stability point of view. For T >> 1, this equation can be seen as a following singular perturbation d

x(t) = g (x(t))

dt

f (x(t

1))

(7)

of general nonlinear difference equation with continuous argument given by g (x(t)) = f (x(t

1)); t

0

0

1)); t

0

(9)

where F () = g (f ()). For the solution of eq. 9 to be continuous for t 1, along with the continuity of F and (), which is the initial function, a so-called consistency condition limt! 0 (t) = F (( 1)) is required [5], [16]. It turns out that a great deal about the asymptotic stability of eq. 7 can be learned from the asymptotic behavior of following difference equation, with Z+ denoting the set of positive integers: xn+1 = F (xn ); n 2 Z+ (10) 1

Some of relevant previous work on these equations is presented in the following section. III. P REVIOUS W ORK In this section we summarize some of relevant work presented in [4]. Consider a nonlinear delay differential equation of the following form: x_ (t) = f (x(t

))

g (x(t))

(11)

where functions f and g are continuous for g (x) in (0; x) and f (x) < g (x) in (x; +1). Eq. 11 can be written in a singular perturbation form by change of coordinates t = s and = 1 . d dt

x(t) = f (x(t

1))

g (x(t))

IV. R ATE C ONTROL

(12)

Now define F (x) := g 1 (f (x)). Invariance and global stability of one dimensional map F can be translated to those of eq. 11 for arbitrary time delay [5] as described here.

WITH

F EEDBACK D ELAY

We study the rate allocation problem in Kelly’s optimization framework described in Section II [7] with the following class of price functions. p(y ) =

(8)

where g (x) = xU (x) and f (y ) = yp(y ) in the context of eq. 6. Under certain natural invertibility conditions on g (), it leads to much studied equation [16]. x(t) = F (x(t

Let I 0

(13)

This kind of marking function arises if the resource is modeled as M=M=1 queue with a service rate C packet per unit time and a packet receives a mark with a congestion indication signal if it arrives at the queue to find at least b packets in the queue. Generally, any strictly increasing continuous function should suffice for our purpose. The class of utility functions we consider here has the form 1 1

Ua (x) =

; a > 0: a xa

(14)

In particular, a = 1 has been found useful for modeling the utility function of Transmission Control Protocol (TCP) algorithms [10]. We say that a user u1 with utility function Ua1 (x) is greedier than another user u2 with utility function Ua2 (x) if a2 > a1 . One can interpret the notion of greed here using the notion of elasticity of demand [17]. With the utility functions of the form in eq. 14 one can easily show that the elasticity of the demand decreases with increasing a as follows. Given a price p, the optimal rate x (p) of the user that maximizes the net 1 utility Ua (x) x p is given by p 1+a . The price elasticity of the demand, which measures how responsive the demand is to a change in price, is defined to be the percent change in demand divided by the percent change in price [17]. In our case the price elasticity of demand is given by p

x (p)

dx (p) dp

= =

1 +1a p

p p

1+1 a 1

1+a

:

1+1 a

1

(15)

Therefore, one can see that price elasticity of the demand decreases with a, i.e., the larger a is, the less responsive the demand is. The model used for design of end-user rate control algorithm in [8] does not explicitly address the case where the total demand of the users exceeds the link capacity. In practice total rate of the users is limited by the link capacity. In order to handle this shortcoming of the model we make following natural assumption:

CDC 2003

3

Assumption 1: Suppose that there are N; N 1; homogeneous users, i.e., users with the same feedback delay and rate. We assume that the rate of each user is bounded from above by C N

C Clearly, allowing more than N may exceed the total throughput momentarily leading to severe penalties for all the users. This bound also makes sense due to the fact that networked users have a tendency to synchronize [6]. In the presence of time delay T , N homogeneous end users with their utility function in eq. 14, each users rate dynamical equation is given by

1

x_ (t) = k

x(t

x(t)a

T)

N x(t

T)

b ! ;

C

(16)

By substituting t = T s

1

x_ (s) =

x(s

x(s)a

1)

N x(s

1)

b

(17)

C

where = T1k . In order to apply the theorems in Section III, we can compare the forms in eq. 11 with that of eq. 16 where x b g (x) = k x1a and f (x) = kx C . It is clear that although eq. 16 looks similar to eq. 11, it does not satisfy all the assumptions required to apply these theorems. In particular, these functions have their range in negative real numbers. It turns out that by making a simple substitution we can make eq. 17 resemble the well studied eq. 11. Consider the following substitution: 0

g (x(t)) = x(t)U (x(t)) := y (t); b x(t) f (x(t)) = x(t) C

and

(18) (19)

We first make the following assumptions on the functions g (x) and f (x). Assumption 2: (i) The function g (x) as given in eq. 18 is 0 strictly decreasing with g (x) > 0 for all x > 0. (ii) The function f (x) is increasing for all x > 0 (iii) Both g (x) and f (x) are Lipschitz continuous on strictly positive real axis. For our results in this paper the exact form of users’ utility functions or the resource price function needs not be given by eq. 14 and 13, respectively, but rather must satisfy Assumption 2. This allows us the following change of coordinate: 1

x(t) = g x_ (t) =

(y (t));

0

(20)

y_ (t)

(21)

1 (y (t)))

g (g 0

1

y_ (t) = g (g

where the inverse g

(y (t)))(y (t) 1

f (g

1

(y (t

1))))

(22)

()

exists from assumption 2. Let Clearly, (y (t)) > 0 under assumption 2. Using this substitution in eq. 22 we get the following form which resembles eq. 11 closely, except for a multiplicative state-dependent gain (y (t)). (y (t)) :=

0

g (g

1

(y (t))).

y_ (t) = (y (t)) f (g

1

(y (t

1)))

y (t)

(23)

It is eq. 23 which we wish to study and show that there is a close correspondence between invariance and global stability properties of map yn+1 = f (g

1

(yn )) := F (yn )

(24)

and those of eq. 23 for fixed and periodic orbits. In particular, we wish to prove that if yn+1 = F (yn ) has a period two fixed point then eq. 23 will have a periodic solution for large enough time delay T if the initial function’s range is contained in the immediate basin of attraction of this period two fixed point. The proofs are based on the invariance property of the underlying map F () given by eq. 24 and the monotonicity of function g (). The map F (y ) is decreasing because g 1 (y ) is strictly decreasing under assumption 2 and a composition of an increasing and a strictly decreasing function (f is increasing from assumption 2) is a decreasing function. This is a much studied scenario for delay differential equations for oscillations where a typical re0 quirement is F < 0 or xF (x) < 0 around the origin. Assumption 3: Suppose now that I 0g is a closed invariant interval under F . In particular let I = [a b℄ be compact. Let X := C ([ 1; 0℄; 0 and 2 XJ0 , (t) = y . limt!1 y Here we are interested in the case where the map defined by eq. 24 goes through a period doubling bifurcation with its eigen value := dF dx jx=x < 1 where x is an unstable fixed point of map F . In the following subsection we describe how the instability of underlying discrete-time map is translated to the instability of delay-different equation in eq. 23. B. Linear Instability Assuming that the map F given by eq. 24 is locally smooth, it is possible to find conditions for linear instability of the fixed point of the map y and that of constant function y (t) = y for the delay-differential equation in eq. 23. In order for y (t) =

CDC 2003

4

y

to be locally asymptotically stable for all T 0, following variational equation should have its zero solution stable. 0

z (t)

T ))

0

= (y (t))F (y (t +

y =y

0

(y (t)) [F (y (t

T ))

T ))

0 = (y (t))F (y (t

y =y

z (t

T)

y℄ z (t

(y (t))

z (t)

j

T)

(y (t)) y=y z (t)

(because F (y ) = y ) := Bz (t

y =y

T ) + Az (t)

(25)

where B = (y )F (y ) and A = (y ). Now to determine the stability of y (t) = y , we can apply well known results [14]. (1) It is known that eq. 25 is stable for all T 0 only if: 0

A

0

and

A

jB j

j j

and T

T := osB

1 2

(

A B) : A2

(27)

For our case it follows from eq. 25 that (y ) is always negative, which holds due to fact that () is always positive. 0 The second condition jF j is crucial to stability of eq. 23. 0 Clearly, for the case when F < 1 (period doubling condition for the map F ) the linear stability condition given by eq. 26 is violated and for a large enough T the constant solution y (t) = y will not be stable. Thus, we know that in unstable situation solutions will be more complex than a constant function and will stay within the interval they initially start from due to the invariance results given by theorem 3. Next we show that in case of instability the bounds on the solution of delay-differential equation will be given by the period two solution of underlying discrete time map F . Theorem 5: Let I := [a; b℄ be a closed interval such that F (I ) := [a1 ; b1 ℄ I . Let the initial condition (t) 2 YI , where YI = f 2 Y j (s) 2 I 8 s 2 [ 1; 0℄g and Y := C ([ 1; 0℄; A

(1) If the solution y (t) is strictly monotone then m = M because of the boundedness of solutions.

V.

APPLICATION

Suppose that there are N; N 1; homogeneous users in the system. Since users are assumed to be homogeneous, we denote the rate of a user by x(t). We assume that utility function of the users is of the form in eq. 14 and the price function used at the resource is that of eq. 13. Then, the end user algorithm is given by x_

(N )

(t)

= k = k

x

(N )

0 (t)Ua (t)

1

x(N ) (t)a

x

x

(N )

(N )

(t

(t

T ) p(N

x N (t

)

T)

N

(

x N (t (

)

T)

T)

(28)

b !

C

;(29)

where a superscript (N ) is used to denote the dependence on N . The underlying discrete time difference equation is given by (N ) yn+1

1 (N ) xn+1

=

N C

(N )

a = xn

(N ) xn+1

b

=

b+1

yn a N

:= F

xnN

(C=N )b (N ) b+1 xn

(

C

)

(N )

(N ) (yn )

(30)

!b ; x(nN ) > 0

(31)

! a1

(32)

C a+b+1 , and the Then, from eq. 32 the fixed point x(N ) is N eigenvalue is given by (N ) (x(N ) ) = b+1 a and is independent of N . Therefore, the stability of the system does not depend on the number of users in the system. This can also be explained using the price elasticity of demand. Since, given a utility function of the form in eq. 14 for some a > 0, the price elasticity of the demand is constant for all x > 0 from eq. 15, one would expect the stability of the system to be independent of the operating point, i.e., the fixed point, and capacity, but only on the choices of the utility and price functions that determine the responsiveness of the users and resource, respectively. However, in the case of instability when a < b + 1, depending on the feedback delay T , it will oscillate. Clearly, the upper C of the bound on the solution is given by the self-imposed limit N b

CDC 2003

5

users to avoid any capacity mismatch. According to Theorem 5 a1 the lower bound will be given by F (C=N ) = N . Hence, C link will see a wide fluctuation from full to very low utilization irrespective of the number of users. VI. N UMERICAL S IMULATIONS We take two homogeneous users with their utility functions of the form given by eq. 14 with a = 3 and price function as in eq. 13 with b = 5 and link capacity C to be 5. It is clear that for these values of a and b rate control algorithm is unstable since 5+1 = 2 > 1. The optimal rates for both users in the absence 3 91 2 5 of delay will be given by x = = 1:6637. Their self 5 imposed upper rate limit will be C=2 = 2:5. The lower limit on the solution according to the period two orbit of map F will be 1 given by F (2=5) = 2 3 = 0:7368. 5

2.5

2.6

2.4

2

2.2

1.5

1.8

2

1.6

1

1.4

cation delays. In particular, these periodic orbits remind of a particular periodic solution class devised specifically for delay-differential equations, namely Slowly Oscillating Periodic (SOP) orbits [13], [16]. Roughly, an SOP is a periodic orbit with its consecutive zeros (zero corresponds to the fixed point y in our case) separated by more than one normalized time unit. The time unit used in our context corresponds to a round-trip time, which arises naturally as a measure for network performance and stability. This also supports the view that round-trip time may be the most useful time scale from the point of view of stability and oscillations [2]. For dynamical eq. 24 we have following conjecture regarding the existence of an SOP: Conjecture 1: SOP: For all 0 < < 1=T0 , where T0 is given by eq. 27 in linear stability context, eq. 23 has at least one slowly oscillating periodic solution with period P ( ) > 2. Moreover, T ( ) ! 2 as ! 0. Although proving the existence of an SOP is technically complicated and its asymptotic behavior is even more challenging, we believe that these slowly oscillating periodic orbits are useful for the study of networks and networked control systems to understand the stability and oscillation behavior in the presence of non-negligible delays.

1.2

0.5

VII. C ONCLUSION

1

0.8

0

0

0.5

1

1.5

2

2.5

0.6

0

20

40

60

80

(a) 2.6

2.4

2.4

2.2

2.2

2

2

1.8

1.8

1.6

1.6

1.4

1.4

1.2

1.2

1

1

0.8

0.8

0

20

40

60

80

100

(c)

120

140

160

180

200

300

350

400

450

500

(b)

2.6

0.6

100

120

140

160

180

200

0.6

0

50

100

150

200

250

(d)

Fig. 1. (a) Map given by eq. 24 for above scenario, (b) Rate waveform for the delay of 1 time unit, (c) for the delay of 10 time units, and (d) for the delay of 50 time units

Fig. 1(a) shows the rate waveform for a delay of one time unit which is not sufficient to send the system into the unstable mode, and hence both rates converge to their optimal value of 1.6637 (Fig. 1(b)). However, when delay is increased to 10 time units, systems is already oscillating as shown in Fig. 1(c). The upper limit of 2.5 and lower limit of 0.7368 can be verified. Finally, in Fig. 1(d), which shows the same waveform for a delay of 50 time units, the waveform is more squarelike compared to last figure. In the limit with increasing delay this waveform approaches a square waveform oscillating between the period two orbit of corresponding map. It is also evident that in both of the oscillating cases the period of the waveform is approximately twice of the delay and the interval between consecutive times when the waveforms cross y (t) = y = 1:6637 is more than the delay itself. Typically, these oscillating orbits are very difficult to describe as they vary from sinusoidal to square waves with increasing value of delay. This phenomena has been studied earlier in [1]. Clearly, these numerical solutions confirm the upper and lower limits for the trajectories for large enough communi-

We show that dynamical stability of a rate control mechanism between users and a resource is determined by the interaction of underlying utility and price functions. In particular, we show that when the users’ utility or resource price function is too responsive in relation to the other, it leads to network instability. We explicitly characterize this for the class of utility functions used. We also note that discrete-time framework arises as a natural tool to study the dynamics of delayed rate control schemes. It also hints the structure of periodic trajectories and their bounds. Finally, we believe that SOP orbits may be very relevant to study of the structure of periodic orbits arising in engineering applications. These periodic orbits have been studied extensively in mathematics community and also arise when the delay is state-dependent, which is a useful context in networking [2]. These SOP orbits support the earlier belief that the round-trip time may be the most relevant time scale for network stability studies. R EFERENCES [1] [2] [3] [4] [5] [6] [7] [8]

S. N. Chow, J. K. Hale, and W. Huang. From sine wave to square wave in delay equations. Proc. Royal Soc, Edinburgh, 120A:223–229, 1992. C. Hollot, V. Misra, D. Towsley, and W. Gong. A control theoretic analysis of RED. In Proc. of IEEE INFOCOM, Anchorage AK, 2001. A. F. Ivanov, E. Liz, and S. I. Trofimchuk. Global stability of a class of scalar nonlinear delay differential equations. Accepted for Publication, Preprint from Author, 2003. A. F. Ivanov, M. A. Pinto, and S. I. Trofimchuk. Global behavior in nonlinear systems with delayed feedback. Proc. Conference of Decison and Control, Sydney, 2000. A. F. Ivanov and A. N. Sharkovsky. Oscillations in singularly perturbed delay equations. Dynamics Reported, 1:165–224, 1991. R. Johari and D. Tan. End-to-end congestion control for the internet: Delays and stability. IEEE Transactions on Networking, 2000. F. Kelly. Charging and rate control for elastic traffic. European Transactions on Telecommunications, 8(1):33–7, January 1997. F. Kelly. Mathematical modelling of the internet. In Bjorn Engquist and Wilfried Schmid (Eds.), Mathematics Unlimited – 2001 and Beyond. Springer-Verlag, Berlin, 2001.

CDC 2003

[9] [10] [11] [12] [13] [14] [15] [16] [17]

F. Kelly, A. Maulloo, and D. Tan. Rate control for communication networks: shadow prices, proportional fairness and stability. Journal of the Operational Research Society, 49(3):237–252, March 1998. S. Kunniyur and R. Srikant. End-to-end congestion control schemes: Utility functions, Random losses and ECN marks. In Proc. of IEEE INFOCOM, Tel-Aviv, Israel, 2000. S. Kunniyur and R. Srikant. A time scale decomposition approach to adaptive ecn marking. In Proc. of IEEE INFOCOM, Anchorage, AK, 2001. S. Low and D. Lapsley. Optimization flow control. IEEE/ACM Transactions on Networking, 7(6):861–874, June 1997. J. Mallet-Paret and R. D. Nussbaum. Global continuation and asymptotic behaviour for periodic solutions of a differential-delay equation. Annali di Matematica Pura ed Applicata, 145(4):33–128, 1986. S. Niculescu, E. I. Verriest, L. Dugard, and J. Dion. Stability and robust stability of time delay systems: A guided tour. Stability and Control of Time-Delay System, LNCS: 228:16–17, 1997. P. Ranjan, E. H. Abed, and R. J. La. Greed, delay and stability tradeoffs in network economics. Submitted to Globecom, 2003. A N Sharkovsky, Y. L. Maistrenko, and E. Y. Romaneko. Difference Equations and their applications: (Trans. from Russian). Kluwer Academic, 1995. Hal R. Varian. Intermediate Microeconomics. Norton, New York, 1996.

6

1

Communication Delay and Instability in Rate-Controlled Networks Priya Ranjan, Eyad H. Abed, and Richard J. La Abstract— We analyze the optimization framework for rate allocation problem proposed by Kelly and characterize the oscillations in terms of underlying map’s period two orbit in the instability regime with arbitrary communication delay. We show the oscillations can be fairly large by computing the bounds explicitly for large return trip times.

I. I NTRODUCTION With the unprecedented growth and popularity of the Internet the problem of rate/congestion control is emerging as a more crucial problem. Poor management of congestion can render one part of a network inaccessible to the rest and significantly degrade the performance of networking applications. Kelly has proposed an optimization framework for rate allocation in the Internet [7]. Using the proposed framework he has shown that the system optimum is achieved at the equilibrium between the end users and resources. Based on this observation researchers have proposed various rate-based algorithms that solve the system optimization problem or its relaxation [7], [11], [12]. However, the convergence of these algorithms has been established only in the absence of feedback delay, and the impact of feedback delay has been left open as well as any trade-off that may exist between stability and selected utility and cost functions. In particular the issue of stability has not been systematically investigated with large communication delays. In our earlier [15] we have established a global stability criterion for system optimization problem in the presence of arbitrary delay for simple one resource problem with multiple homogeneous flows. This stability condition was derived using the invariance-based global stability results for nonlinear delaydifferential equations [13], [5], [4], [3]. This kind of global stability results are different from that based on Lyapunov or Razumikhin theorems in the sense that in our approach also provides us with insight on the structure of emerging periodic orbits, e.g., their periodicity and amplitude, in the case of loss of stability. This loss of stability and characterization of ensuing periodic orbit are main topics of this paper. Generally speaking, our results tell us that if the user and resource curves have a stable market equilibrium, then corresponding dynamical equation for flow-optimization will converge to the optimal solution in the presence of arbitrary delay. This result essentially shows that stability is related to utility and price curves in a fundamental way. In particular, for a given price curve, it possible to design stable user utility functions such that the corresponding dynamical system converges to the optimal flow irrespective of communication delay. Conversely, if the underlying market equilibrium is unstable then it is possible to find a large enough delay for which the optimal The authors are with the Department of Electrical and Computer Engineering and the Institute for Systems Research, University of Maryland, College Park, MD 20742 USA. Email: [email protected]

point loses its stability and gives way to oscillations. This paper studies the oscillatory orbits by explicitly giving the bounds on their amplitude. It is also shown that these bounds are derived from an underlying discrete time map which goes through a period doubling bifurcation with the loss of stability. These results provide an interesting perspective for designing end user algorithms and active queue management (AQM) mechanisms. It is also worth noting that in general characterizing the exact conditions for stability with a delay is difficult. Hence, our result provides a simple and yet robust way of dealing with the problem of widely varying feedback delay in communication networks through a clever choice of the user utility function and price functions. This paper is organized as follows. Section II describes the optimization problem for rate control. Relevant previous work on the stability criterion of a system given by a delayed differential equation is given in Section III. Our main results on instability are presented in Section IV, which is followed by numerical examples in Section VI. We conclude the paper in Section VII. II. BACKGROUND In this section we briefly describe the rate control problem in the proposed optimization framework. Consider a flow traversing a single resource. The rate control problem can be formulated as the following net utility optimization problem from the end user’s point of view [7]: max x

U (x) x

s. t.

x p(x)

(1)

C

where x is the rate, U (x) is the utility of the user when it receives a rate of x, p(x) is the price per unit flow the user has to pay when the rate is x, and C is the capacity of the resource. The proposed end user algorithm in the absence of delay is given by the following differential equation [9]. d dt

x(t) = k (w(t)

x(t)(t))

(2)

where w(t) is the price per unit time user is willing to pay, (t) = p(x(t)), and k; k > 0, is a gain parameter. The case where w(t) is a fixed constant, i.e., U (x) = log (x), is studied in [8]. In this paper we assume that w(t) = x(t) 0 U (x(t)) with any monotonically decreasing concave family of utility functions [9] with their decrease rate smaller than x1 . Now, suppose that congestion signal generated at the shared resource, i.e., p(x(t)), is returned to the user after a fixed and common round trip time T . In the presence of delay the interaction is given by the following delayed differential equation d dt

x(t) = k (w(t)

x(t

T )(t

T ))

(3)

CDC 2003

2

=k

0

x(t)U (x(t))

x(t

T )p(x(t

After normalizing time by T and replacing t becomes: d T ds d ds

T)

= s

(4)

T , eq. 4

0

x(s) = k x(s)U (x(s)) 0

x(s) = x(s)U (x(s))

x(s

x(s

1)p(x(s

1)p(x(s

1))

1))

(5) (6)

where = T1k . It is precisely eq. 6 we are interested in from stability point of view. For T >> 1, this equation can be seen as a following singular perturbation d

x(t) = g (x(t))

dt

f (x(t

1))

(7)

of general nonlinear difference equation with continuous argument given by g (x(t)) = f (x(t

1)); t

0

0

1)); t

0

(9)

where F () = g (f ()). For the solution of eq. 9 to be continuous for t 1, along with the continuity of F and (), which is the initial function, a so-called consistency condition limt! 0 (t) = F (( 1)) is required [5], [16]. It turns out that a great deal about the asymptotic stability of eq. 7 can be learned from the asymptotic behavior of following difference equation, with Z+ denoting the set of positive integers: xn+1 = F (xn ); n 2 Z+ (10) 1

Some of relevant previous work on these equations is presented in the following section. III. P REVIOUS W ORK In this section we summarize some of relevant work presented in [4]. Consider a nonlinear delay differential equation of the following form: x_ (t) = f (x(t

))

g (x(t))

(11)

where functions f and g are continuous for g (x) in (0; x) and f (x) < g (x) in (x; +1). Eq. 11 can be written in a singular perturbation form by change of coordinates t = s and = 1 . d dt

x(t) = f (x(t

1))

g (x(t))

IV. R ATE C ONTROL

(12)

Now define F (x) := g 1 (f (x)). Invariance and global stability of one dimensional map F can be translated to those of eq. 11 for arbitrary time delay [5] as described here.

WITH

F EEDBACK D ELAY

We study the rate allocation problem in Kelly’s optimization framework described in Section II [7] with the following class of price functions. p(y ) =

(8)

where g (x) = xU (x) and f (y ) = yp(y ) in the context of eq. 6. Under certain natural invertibility conditions on g (), it leads to much studied equation [16]. x(t) = F (x(t

Let I 0

(13)

This kind of marking function arises if the resource is modeled as M=M=1 queue with a service rate C packet per unit time and a packet receives a mark with a congestion indication signal if it arrives at the queue to find at least b packets in the queue. Generally, any strictly increasing continuous function should suffice for our purpose. The class of utility functions we consider here has the form 1 1

Ua (x) =

; a > 0: a xa

(14)

In particular, a = 1 has been found useful for modeling the utility function of Transmission Control Protocol (TCP) algorithms [10]. We say that a user u1 with utility function Ua1 (x) is greedier than another user u2 with utility function Ua2 (x) if a2 > a1 . One can interpret the notion of greed here using the notion of elasticity of demand [17]. With the utility functions of the form in eq. 14 one can easily show that the elasticity of the demand decreases with increasing a as follows. Given a price p, the optimal rate x (p) of the user that maximizes the net 1 utility Ua (x) x p is given by p 1+a . The price elasticity of the demand, which measures how responsive the demand is to a change in price, is defined to be the percent change in demand divided by the percent change in price [17]. In our case the price elasticity of demand is given by p

x (p)

dx (p) dp

= =

1 +1a p

p p

1+1 a 1

1+a

:

1+1 a

1

(15)

Therefore, one can see that price elasticity of the demand decreases with a, i.e., the larger a is, the less responsive the demand is. The model used for design of end-user rate control algorithm in [8] does not explicitly address the case where the total demand of the users exceeds the link capacity. In practice total rate of the users is limited by the link capacity. In order to handle this shortcoming of the model we make following natural assumption:

CDC 2003

3

Assumption 1: Suppose that there are N; N 1; homogeneous users, i.e., users with the same feedback delay and rate. We assume that the rate of each user is bounded from above by C N

C Clearly, allowing more than N may exceed the total throughput momentarily leading to severe penalties for all the users. This bound also makes sense due to the fact that networked users have a tendency to synchronize [6]. In the presence of time delay T , N homogeneous end users with their utility function in eq. 14, each users rate dynamical equation is given by

1

x_ (t) = k

x(t

x(t)a

T)

N x(t

T)

b ! ;

C

(16)

By substituting t = T s

1

x_ (s) =

x(s

x(s)a

1)

N x(s

1)

b

(17)

C

where = T1k . In order to apply the theorems in Section III, we can compare the forms in eq. 11 with that of eq. 16 where x b g (x) = k x1a and f (x) = kx C . It is clear that although eq. 16 looks similar to eq. 11, it does not satisfy all the assumptions required to apply these theorems. In particular, these functions have their range in negative real numbers. It turns out that by making a simple substitution we can make eq. 17 resemble the well studied eq. 11. Consider the following substitution: 0

g (x(t)) = x(t)U (x(t)) := y (t); b x(t) f (x(t)) = x(t) C

and

(18) (19)

We first make the following assumptions on the functions g (x) and f (x). Assumption 2: (i) The function g (x) as given in eq. 18 is 0 strictly decreasing with g (x) > 0 for all x > 0. (ii) The function f (x) is increasing for all x > 0 (iii) Both g (x) and f (x) are Lipschitz continuous on strictly positive real axis. For our results in this paper the exact form of users’ utility functions or the resource price function needs not be given by eq. 14 and 13, respectively, but rather must satisfy Assumption 2. This allows us the following change of coordinate: 1

x(t) = g x_ (t) =

(y (t));

0

(20)

y_ (t)

(21)

1 (y (t)))

g (g 0

1

y_ (t) = g (g

where the inverse g

(y (t)))(y (t) 1

f (g

1

(y (t

1))))

(22)

()

exists from assumption 2. Let Clearly, (y (t)) > 0 under assumption 2. Using this substitution in eq. 22 we get the following form which resembles eq. 11 closely, except for a multiplicative state-dependent gain (y (t)). (y (t)) :=

0

g (g

1

(y (t))).

y_ (t) = (y (t)) f (g

1

(y (t

1)))

y (t)

(23)

It is eq. 23 which we wish to study and show that there is a close correspondence between invariance and global stability properties of map yn+1 = f (g

1

(yn )) := F (yn )

(24)

and those of eq. 23 for fixed and periodic orbits. In particular, we wish to prove that if yn+1 = F (yn ) has a period two fixed point then eq. 23 will have a periodic solution for large enough time delay T if the initial function’s range is contained in the immediate basin of attraction of this period two fixed point. The proofs are based on the invariance property of the underlying map F () given by eq. 24 and the monotonicity of function g (). The map F (y ) is decreasing because g 1 (y ) is strictly decreasing under assumption 2 and a composition of an increasing and a strictly decreasing function (f is increasing from assumption 2) is a decreasing function. This is a much studied scenario for delay differential equations for oscillations where a typical re0 quirement is F < 0 or xF (x) < 0 around the origin. Assumption 3: Suppose now that I 0g is a closed invariant interval under F . In particular let I = [a b℄ be compact. Let X := C ([ 1; 0℄; 0 and 2 XJ0 , (t) = y . limt!1 y Here we are interested in the case where the map defined by eq. 24 goes through a period doubling bifurcation with its eigen value := dF dx jx=x < 1 where x is an unstable fixed point of map F . In the following subsection we describe how the instability of underlying discrete-time map is translated to the instability of delay-different equation in eq. 23. B. Linear Instability Assuming that the map F given by eq. 24 is locally smooth, it is possible to find conditions for linear instability of the fixed point of the map y and that of constant function y (t) = y for the delay-differential equation in eq. 23. In order for y (t) =

CDC 2003

4

y

to be locally asymptotically stable for all T 0, following variational equation should have its zero solution stable. 0

z (t)

T ))

0

= (y (t))F (y (t +

y =y

0

(y (t)) [F (y (t

T ))

T ))

0 = (y (t))F (y (t

y =y

z (t

T)

y℄ z (t

(y (t))

z (t)

j

T)

(y (t)) y=y z (t)

(because F (y ) = y ) := Bz (t

y =y

T ) + Az (t)

(25)

where B = (y )F (y ) and A = (y ). Now to determine the stability of y (t) = y , we can apply well known results [14]. (1) It is known that eq. 25 is stable for all T 0 only if: 0

A

0

and

A

jB j

j j

and T

T := osB

1 2

(

A B) : A2

(27)

For our case it follows from eq. 25 that (y ) is always negative, which holds due to fact that () is always positive. 0 The second condition jF j is crucial to stability of eq. 23. 0 Clearly, for the case when F < 1 (period doubling condition for the map F ) the linear stability condition given by eq. 26 is violated and for a large enough T the constant solution y (t) = y will not be stable. Thus, we know that in unstable situation solutions will be more complex than a constant function and will stay within the interval they initially start from due to the invariance results given by theorem 3. Next we show that in case of instability the bounds on the solution of delay-differential equation will be given by the period two solution of underlying discrete time map F . Theorem 5: Let I := [a; b℄ be a closed interval such that F (I ) := [a1 ; b1 ℄ I . Let the initial condition (t) 2 YI , where YI = f 2 Y j (s) 2 I 8 s 2 [ 1; 0℄g and Y := C ([ 1; 0℄; A

(1) If the solution y (t) is strictly monotone then m = M because of the boundedness of solutions.

V.

APPLICATION

Suppose that there are N; N 1; homogeneous users in the system. Since users are assumed to be homogeneous, we denote the rate of a user by x(t). We assume that utility function of the users is of the form in eq. 14 and the price function used at the resource is that of eq. 13. Then, the end user algorithm is given by x_

(N )

(t)

= k = k

x

(N )

0 (t)Ua (t)

1

x(N ) (t)a

x

x

(N )

(N )

(t

(t

T ) p(N

x N (t

)

T)

N

(

x N (t (

)

T)

T)

(28)

b !

C

;(29)

where a superscript (N ) is used to denote the dependence on N . The underlying discrete time difference equation is given by (N ) yn+1

1 (N ) xn+1

=

N C

(N )

a = xn

(N ) xn+1

b

=

b+1

yn a N

:= F

xnN

(C=N )b (N ) b+1 xn

(

C

)

(N )

(N ) (yn )

(30)

!b ; x(nN ) > 0

(31)

! a1

(32)

C a+b+1 , and the Then, from eq. 32 the fixed point x(N ) is N eigenvalue is given by (N ) (x(N ) ) = b+1 a and is independent of N . Therefore, the stability of the system does not depend on the number of users in the system. This can also be explained using the price elasticity of demand. Since, given a utility function of the form in eq. 14 for some a > 0, the price elasticity of the demand is constant for all x > 0 from eq. 15, one would expect the stability of the system to be independent of the operating point, i.e., the fixed point, and capacity, but only on the choices of the utility and price functions that determine the responsiveness of the users and resource, respectively. However, in the case of instability when a < b + 1, depending on the feedback delay T , it will oscillate. Clearly, the upper C of the bound on the solution is given by the self-imposed limit N b

CDC 2003

5

users to avoid any capacity mismatch. According to Theorem 5 a1 the lower bound will be given by F (C=N ) = N . Hence, C link will see a wide fluctuation from full to very low utilization irrespective of the number of users. VI. N UMERICAL S IMULATIONS We take two homogeneous users with their utility functions of the form given by eq. 14 with a = 3 and price function as in eq. 13 with b = 5 and link capacity C to be 5. It is clear that for these values of a and b rate control algorithm is unstable since 5+1 = 2 > 1. The optimal rates for both users in the absence 3 91 2 5 of delay will be given by x = = 1:6637. Their self 5 imposed upper rate limit will be C=2 = 2:5. The lower limit on the solution according to the period two orbit of map F will be 1 given by F (2=5) = 2 3 = 0:7368. 5

2.5

2.6

2.4

2

2.2

1.5

1.8

2

1.6

1

1.4

cation delays. In particular, these periodic orbits remind of a particular periodic solution class devised specifically for delay-differential equations, namely Slowly Oscillating Periodic (SOP) orbits [13], [16]. Roughly, an SOP is a periodic orbit with its consecutive zeros (zero corresponds to the fixed point y in our case) separated by more than one normalized time unit. The time unit used in our context corresponds to a round-trip time, which arises naturally as a measure for network performance and stability. This also supports the view that round-trip time may be the most useful time scale from the point of view of stability and oscillations [2]. For dynamical eq. 24 we have following conjecture regarding the existence of an SOP: Conjecture 1: SOP: For all 0 < < 1=T0 , where T0 is given by eq. 27 in linear stability context, eq. 23 has at least one slowly oscillating periodic solution with period P ( ) > 2. Moreover, T ( ) ! 2 as ! 0. Although proving the existence of an SOP is technically complicated and its asymptotic behavior is even more challenging, we believe that these slowly oscillating periodic orbits are useful for the study of networks and networked control systems to understand the stability and oscillation behavior in the presence of non-negligible delays.

1.2

0.5

VII. C ONCLUSION

1

0.8

0

0

0.5

1

1.5

2

2.5

0.6

0

20

40

60

80

(a) 2.6

2.4

2.4

2.2

2.2

2

2

1.8

1.8

1.6

1.6

1.4

1.4

1.2

1.2

1

1

0.8

0.8

0

20

40

60

80

100

(c)

120

140

160

180

200

300

350

400

450

500

(b)

2.6

0.6

100

120

140

160

180

200

0.6

0

50

100

150

200

250

(d)

Fig. 1. (a) Map given by eq. 24 for above scenario, (b) Rate waveform for the delay of 1 time unit, (c) for the delay of 10 time units, and (d) for the delay of 50 time units

Fig. 1(a) shows the rate waveform for a delay of one time unit which is not sufficient to send the system into the unstable mode, and hence both rates converge to their optimal value of 1.6637 (Fig. 1(b)). However, when delay is increased to 10 time units, systems is already oscillating as shown in Fig. 1(c). The upper limit of 2.5 and lower limit of 0.7368 can be verified. Finally, in Fig. 1(d), which shows the same waveform for a delay of 50 time units, the waveform is more squarelike compared to last figure. In the limit with increasing delay this waveform approaches a square waveform oscillating between the period two orbit of corresponding map. It is also evident that in both of the oscillating cases the period of the waveform is approximately twice of the delay and the interval between consecutive times when the waveforms cross y (t) = y = 1:6637 is more than the delay itself. Typically, these oscillating orbits are very difficult to describe as they vary from sinusoidal to square waves with increasing value of delay. This phenomena has been studied earlier in [1]. Clearly, these numerical solutions confirm the upper and lower limits for the trajectories for large enough communi-

We show that dynamical stability of a rate control mechanism between users and a resource is determined by the interaction of underlying utility and price functions. In particular, we show that when the users’ utility or resource price function is too responsive in relation to the other, it leads to network instability. We explicitly characterize this for the class of utility functions used. We also note that discrete-time framework arises as a natural tool to study the dynamics of delayed rate control schemes. It also hints the structure of periodic trajectories and their bounds. Finally, we believe that SOP orbits may be very relevant to study of the structure of periodic orbits arising in engineering applications. These periodic orbits have been studied extensively in mathematics community and also arise when the delay is state-dependent, which is a useful context in networking [2]. These SOP orbits support the earlier belief that the round-trip time may be the most relevant time scale for network stability studies. R EFERENCES [1] [2] [3] [4] [5] [6] [7] [8]

S. N. Chow, J. K. Hale, and W. Huang. From sine wave to square wave in delay equations. Proc. Royal Soc, Edinburgh, 120A:223–229, 1992. C. Hollot, V. Misra, D. Towsley, and W. Gong. A control theoretic analysis of RED. In Proc. of IEEE INFOCOM, Anchorage AK, 2001. A. F. Ivanov, E. Liz, and S. I. Trofimchuk. Global stability of a class of scalar nonlinear delay differential equations. Accepted for Publication, Preprint from Author, 2003. A. F. Ivanov, M. A. Pinto, and S. I. Trofimchuk. Global behavior in nonlinear systems with delayed feedback. Proc. Conference of Decison and Control, Sydney, 2000. A. F. Ivanov and A. N. Sharkovsky. Oscillations in singularly perturbed delay equations. Dynamics Reported, 1:165–224, 1991. R. Johari and D. Tan. End-to-end congestion control for the internet: Delays and stability. IEEE Transactions on Networking, 2000. F. Kelly. Charging and rate control for elastic traffic. European Transactions on Telecommunications, 8(1):33–7, January 1997. F. Kelly. Mathematical modelling of the internet. In Bjorn Engquist and Wilfried Schmid (Eds.), Mathematics Unlimited – 2001 and Beyond. Springer-Verlag, Berlin, 2001.

CDC 2003

[9] [10] [11] [12] [13] [14] [15] [16] [17]

F. Kelly, A. Maulloo, and D. Tan. Rate control for communication networks: shadow prices, proportional fairness and stability. Journal of the Operational Research Society, 49(3):237–252, March 1998. S. Kunniyur and R. Srikant. End-to-end congestion control schemes: Utility functions, Random losses and ECN marks. In Proc. of IEEE INFOCOM, Tel-Aviv, Israel, 2000. S. Kunniyur and R. Srikant. A time scale decomposition approach to adaptive ecn marking. In Proc. of IEEE INFOCOM, Anchorage, AK, 2001. S. Low and D. Lapsley. Optimization flow control. IEEE/ACM Transactions on Networking, 7(6):861–874, June 1997. J. Mallet-Paret and R. D. Nussbaum. Global continuation and asymptotic behaviour for periodic solutions of a differential-delay equation. Annali di Matematica Pura ed Applicata, 145(4):33–128, 1986. S. Niculescu, E. I. Verriest, L. Dugard, and J. Dion. Stability and robust stability of time delay systems: A guided tour. Stability and Control of Time-Delay System, LNCS: 228:16–17, 1997. P. Ranjan, E. H. Abed, and R. J. La. Greed, delay and stability tradeoffs in network economics. Submitted to Globecom, 2003. A N Sharkovsky, Y. L. Maistrenko, and E. Y. Romaneko. Difference Equations and their applications: (Trans. from Russian). Kluwer Academic, 1995. Hal R. Varian. Intermediate Microeconomics. Norton, New York, 1996.

6