Towards Stateless Core: Improving SIP Proxy Scalability Mauricio Cortes
Jairo O. Esteban
Hyewon Jun
Bell Labs, Lucent Technologies 600 Mountain Ave, Rm 2D-517 Murray Hill, NJ +1 908 582 4656
Bell Labs, Lucent Technologies 101 Crawford Corner Rd, Rm 4G-626 Holmdel, NJ 07733 +1 732 949 3587
Georgia Institute of Technology 801 Atlantic Dr Atlanta, GA 30332 +1 404 894-3152
[email protected]
[email protected]
[email protected]
ABSTRACT The Third Generation Partnership Project (3GPP) specified the IP Multimedia Subsystem (IMS) standard. This standard uses the Session Initiation Protocol (SIP) to setup and teardown multimedia sessions through a potentially large chain of proxies. The IMS standard allows some core elements to be deployed as transaction stateful or stateless proxies. Moreover, SIP standard allows proxies decide when to store transaction information on a per request basis. In this paper, we examine the implementation of these core IMS proxies. We propose an algorithm that determines for each incoming request when to store transaction information. The goal of this algorithm is to increase throughput by reducing the number of transactions without compromising call setup times. Our algorithm uses information such as the incoming SIP request, CPU usage, and the number of retransmitted messages. We have implemented this algorithm in our IMS simulation tool. The results show that the system throughput can be improved by as much as 46%.
General Terms Algorithms, Performance, Measurement
Keywords SIP, Proxy, Transaction Stateful, Transaction Stateless
1. INTRODUCTION The Third Generation Partnership Project (3GPP) [1] [2] has specified the IP Multimedia Subsystem (IMS) that enables end-user devices to setup, modify, and teardown multimedia sessions in managed networks. The end-user devices and the IMS core network use the Session Initiation Protocol (SIP) messages as its signaling control protocol. SIP is a client/server application protocol, where client applications send SIP request messages to create, change, or destroy a multimedia session. Server applications reply back one or more responses for each SIP request. RFC 3261 [9] describes an intermediary element called a SIP proxy. Proxies help process and route SIP messages from/to SIP endpoints. A SIP transaction represents a request and their corresponding responses exchanged between two adjacent SIP elements. To support transaction, SIP elements must store information(state) to handle message retransmissions with adjacent elements.
SIP proxies have the choice to support or not transaction functionality on a per-request basis. We abide by the following convention. A transaction stateful proxy is a SIP element that supports transactions for all incoming requests, while a transaction stateless proxy does not store transaction information for any request. A hybrid transaction proxy determines on a per-request basis how to process each new incoming SIP request. SIP proxies have been implemented as stateless or stateful proxies. To date, we have no knowledge of any hybrid proxy implementation and the impact on their message throughput. Each transaction stateful request requires additional memory, CPU cycles, timer handling, and network resources compared to a transaction stateless request. State-of-the-art computers, operating systems, and networks can accommodate the memory, timing support, and bandwidth requirements by SIP transactions. However, SIP is a CPU-bound protocol as reported by Cortes et.al. [3]. If we decrease the number of transactions processed per request, we are able to increase the proxy throughput measured in SIP requests per second. However, tension arises with lossy connections where stateless elements cannot retransmit, leaving this responsibility to stateful elements. These stateless elements introduce additional delays, possibly exceeding the maximum call setup time. Thus, the goal is to minimize the use of SIP transactions to improve throughput without compromising the call setup time. In this paper, we describe an algorithm for a hybrid transaction proxy that decides which SIP requests will create a SIP transaction. The algorithm examines information stored in the incoming SIP request, operating system (e.g. CPU usage), and network links (e.g. retransmissions). We added our algorithm to our IMS simulation tool [4]. The results show that the algorithm improves the overall system throughput by about 46%. The roadmap of this paper is as follows. The next section presents the main features of SIP and IMS. Section 3 describes the algorithm that decides when an incoming SIP request creates a transaction. Section 4 presents the results of our IMS simulator, showing the significant impact of SIP transactions on the overall IMS capacity. We conclude this paper with some final remarks.
2. BACKGROUND This section introduces SIP and IMS terminology. The first subsection summarizes SIP main network components. Later, we present key IMS components. We conclude with a related work subsection.
2.1 SIP The Session Initiation Protocol (SIP) is an IETF application-layer signaling protocol, which allows enduser devices to create, modify, and terminate sessions.
2.1.1 SIP elements Endpoints in a SIP signaling path are called User Agents (UA), and can play the role of either User Agent Client (UAC) when they originate requests, or User Agent Server (UAS) as the destination of the request. SIP proxies are routing elements that decide the message’s next hop(s) based on information coming from DNS servers, network traffic, etc. The nature of SIP proxies is to route SIP messages to one or more terminating User Agents (UAs), possibly adding value in the signaling path, for example, by authenticating users. Zero or more SIP proxies can be in the signaling path between UAC and UAS.
2.1.2 SIP Abstractions A SIP message can be subdivided in two main categories: requests and responses. Requests are originated by UAC in order to invoke a service. INVITE, BYE, CANCEL, OPTIONS, and REGISTER are the basic request messages. Responses are originated in UAS triggered by incoming requests. A response can be either provisional or final. The former (1XX message) indicates progress has been made on the request, while the latter indicates successful (2XX) or unsuccessful (3XX-6XX) completion of the request. 1 . IN V IT E
T im e
2 . 1 0 0 T r y in g 3 . IN V IT E 4 . 1 0 0 T r y in g UAC 6 . 1 8 0 R in g i n g
S IP P ro x y
5 . 1 8 0 R in g in g 7. 200 O K
UAS
8. 200 O K 9 .A C K 10. A C K
Figure 1. Basic SIP call setup flow Figure 1 depicts the basic flow to setup a call through a SIP Proxy. These messages include one request (INVITE), two provisional responses (100 Trying and 180 Ringing), and a final message (200 OK). The handshake between UAC and UAS is completed with an ACK message, confirming that the call has been established. SIP messages can be exchanged using UDP, TCP, or SCTP as their underlying transport protocol. Even with reliable transport protocols, SIP elements might retransmit
messages. Some retransmissions are handled between adjacent path elements, others are handled end to end. SIP transactions as described in RFC 3261 support message retransmissions, among other functionality. A transaction is created by a request, and is typically terminated by a final response. Transaction stateful elements will store information about the request and associated responses to absorb incoming retransmissions and to generate new retransmissions from/to adjacent SIP elements. Figure 1 shows that a transaction is created with the arrival of the INVITE request, and is destroyed after processing the ACK message. The dash line denotes the transaction lifetime. SIP transactions enable other functionality such as forking or forwarding messages using a different transport protocol. In these two cases, the proxy must store transaction information. All other cases, proxies can decide to store or not transaction related data. In this paper, we focus on the common case of requests that do not fork and use the same transport protocol to forward messages. RFC 3261 does not prescribe the co-location of transaction stateful and stateless functionality, in fact, we have found that SIP specifications [9] state that “A proxy can operate in either a stateful or stateless mode for each new request“, allowing SIP proxies to behave either way on a per-request basis. SIP message processing entails reading, parsing, processing, and responding/forwarding operations. Cortes,et.al. report that parsing and processing use the largest fraction of computational resources. Message processing in a transaction stateful proxy includes the creation, lookup, update, and destruction of transaction state. In addition, transactions need to create 8 timers in average for each call. Given the asynchronous nature of these timers, an important amount of resources are dedicated to timer support. Therefore, any reduction in the number of transactions in a SIP proxy has a significant impact in the overall system throughput.
2.2 IMS IMS is a 3GPP standard that specifies the IP network elements and protocols to offer a wide range of telephony and converged services in a managed network. In this subsection, we will describe the main IMS elements and the message flow to setup a basic call.
2.2.1 IMS functional elements IMS end-user devices must behave as standard SIP UA plus a few SIP extensions such as 100rel, and UPDATE. The first extension adds reliability support to provisional responses. The second extension allows end-user devices to modify multimedia settings before establishing a call. IMS specifications describe a number of SIP proxy servers including Call Session Control Function (CSCF). These proxies help control and coordinate the services
offered by the IMS provider. There are three types of CSCFs. Proxy-CSCF (P-CSCF) is an edge proxy for the IMS network. The Serving-CSCF (S-CSCF) controls (e.g. registration, authentication) and coordinates IMS services for end-users and their devices. The Interrogating-CSCF (I-CSCF) balances the load between two or more S-CSCF. It is mandatory for the I-CSCF to be a transaction stateful proxy. In contrast, the standard does not impose any restrictions on S- and P-CSCFs. Application Servers (ASes) implement telephony, data, or converged services. Examples of ASes include voice mail and Push-To-Talk servers. S-CSCFs coordinate the usage of ASes. Each request and associated responses can be forwarded by the S-CSCF to zero or more ASes.
2.2.2 IMS Flows IMS components exchange a large number of SIP messages to establish and tear down a session. Figure 2 shows the message flow to establish a call between two IMS subscribers including one AS in the signaling path. A total of 40 messages (20 incoming / 20 outgoing) and 6 transactions are needed to setup one call. Each AS in the signaling path adds 20 (10/10) messages and 3 transactions, increasing the amount of work done by SCSCF to process a single call. 1.INVITE UAC 2. 100 Trying
UAC
7.INVITE 8. 100 Trying 9. 183 Progress 12.183 Progress 13.PRACK 16.PRACK 17.200 [PRACK] 20.200 [PRACK] 21.180 Ringing 24.180 Ringing 25.PRACK 28.PRACK 29.200 [PRACK] 32.200 [PRACK] 33.200 [INVITE] 36.200 [INVITE] 37.ACK
UAS
40.ACK
UAS UAC UAS UAC UAS UAC UAS UAC UAS
3.INVITE 4. 100 Trying 5.INVITE 6. 100 Trying
S-CSCF
10.183 Progress 11.183 Progress 14.PRACK 15.PRACK 18.200 [PRACK] 19.200 [PRACK] 22.180 Ringing 23.180 Ringing 26.PRACK 27.PRACK 30.200 [PRACK] 31.200 [PRACK] 34.200 [INVITE] 35.200 [INVITE] 38.ACK 39.ACK
AS
Figure 2. IMS Basic Message Flow Compared to a basic SIP call setup, an IMS call without an AS in the signaling path needs twice the number of messages and three times the number of transactions. The difference is due to a provisional message (183 Progress) and the 100rel extension. This extension triggers one PRACK request and one transaction for each provisional message. The number of incoming messages grows linearly with the number of ASes in the signaling path, 10 + 10N, where N is the number of ASes. Likewise, the number of transactions also grows at a rate of 3 + 3N.
In a 3G wireless network, end-devices communicate with P-CSCF proxies using the air interface where bit error rates (BER) are higher than in wired networks. Further, PCSCFs can also exchange SIP traffic with a wide range of wired elements using DSL, Cable, or other access technologies. Typically, S-CSCFs communicate with its neighbors using high bandwidth, low latency, and low BER networks such as Fast/Giga Ethernet. However, SCSCF can potentially exchange messages with remote ASes connected to unmanaged network. Therefore, P-CSCF and S-CSCF can exchange messages using a heterogeneous collection of networks. Both elements will benefit from our proposal of colocating stateful and stateless functionality. These proxies will pick the appropriate functionality depending on the network and local computational resources available.
2.3 Related Work Other authors have shown concern about the expected performance and capacity of IMS components, especially the S-CSCF. In particular, Gurbani and Jain [5] acknowledge the need to reduce the amount of state stored at the S-CSCF. They proposed a technique that applies to ASes. Their technique couples S-CSCF with the ASes, using an API or a lightweight RPC mechanism to communicate between these elements. Our approach differs in that we allow S-CSCF and ASes use SIP as proposed in the standards. We leave to the S-CSCF the decision on how much state to store on a per request basis with ASes and other SIP elements. O’Regan and Pesch [7] used a simulation to estimate the mean delay for a call setup on an IMS environment. In their case, they used a stochastic, event-driven simulator, and evaluated a proposed architecture for Push To Talk (PTT) that integrates UMTS and IMS. Our simulation differs in that we added to our model different levels of statefulness, our message flow includes additional IMSspecific messages (e.g. 183 Progress, PRACK), and we add Application Servers to the signaling path. Pous, Pesch, and Foster [8] also used a simulator to estimate end-to-end signaling delays in a SIP-based UMTS network. They proposed optimizations to improve response times, like using SIP compression and integrating IMS elements in the same node. Our proposed optimizations are local to the SIP proxies. Again, we also include other metrics and estimate the system’s capacity. Typically, SIP proxies can be configured to limit the amount of transaction state they can store. However, these configuration parameters usually limit the proxy to always behave transaction stateless xor transaction stateful. Remarkably, the SIP Express Router (SER) [6] provides a scripting language that allows the programmer to decide on a per-request basis. Our approach could be partially implemented in SER. However, the decision point in SER
takes place at message arrival. We believe that proxies should be able to make a final decision some time later.
3. OUR APPROACH SIP proxies cannot anticipate the message load and network conditions. CPU, memory, and network usage increases with the number of incoming messages. A proxy can reduce resource consumption by behaving as a stateless proxy. However, if the network conditions deteriorate, stateless proxy exacerbates retransmission delays, introducing longer delays and impacting ringing and call setup times. Therefore, it is reasonable to colocate stateless and stateful functionality within the same proxy. The proxy can decide which functionality to use depending on the runtime conditions. We propose a hybrid transaction IMS core where S- and P-CSCFs can determine how to process each incoming request. In particular, we argue that proxies exchanging messages using low bit error rate networks should not store transaction state for most or all of their incoming requests. If a proxy detects a link or connection with high error rate, it should examine if enough CPU cycles are available to support a SIP transaction. Two proxies exchanging messages using a highly reliable network will rarely need to retransmit. In this scenario, transaction state is a high overhead cost with little benefit. In contrast, proxies exchanging SIP messages over lossy networks are better off storing transaction state and retransmitting messages as often as needed to meet the timing constraints and to reduce message load (e.g. retransmissions). However, an overloaded proxy needs to manage its load to lower the CPU usage. One approach is to store transaction information to a fraction of the new SIP requests needing transaction support. Notice that INVITE transactions tend to have longer lifetime and a higher message count compared to all other transactions. This implies that INVITE transactions have a higher chance to lose a message. A first step is to limit new transactions to those triggered by INVITE or BYE requests. If the CPU overload persists, the proxy can set a limit on the number of new INVITE transactions. In summary, the CPU usage and the underlying network connection are used to increase proxy throughput. By taking advantage of high bandwidth, low bit error rate, and low latency connections, we can skip transaction processing. Likewise, if the machine has only a few CPU cycles available, the proxy should try to shed some CPU load by skipping transaction processing for some requests, such as PRACK-related transactions. It is important to point out that proxies can switch modes between two requests dealing with the same call. For example, proxies can process the initial INVITE request in transaction stateful mode, but process the BYE request in stateless mode, or vice-versa. This flexibility provides
the SIP application with the ability to react to sudden changes in its environment. For instance, mobile devices can initiate a call under excellent network conditions. As the user moves to a different location, network conditions can change, forcing a P-CSCF to create transactions for new SIP requests. Figure 3 presents a macro-algorithm to process a request message within a SIP proxy. This algorithm decides which requests need to create a transaction based on the following factors: 1. Link quality: Applications can monitor the quality of all links. TCP, high bandwidth, low-packet-loss links have a higher reliability and therefore don’t require the use of transactions. On the other hand, low bandwidth, high-packet-loss (e.g. mobile radio) connections, may require the use of transactions to support retransmissions. 2. Actual CPU load: As a protection mechanism against CPU overload, the algorithm decides to process new requests without an associated transaction, as a previous step to a more severe protection measure, like rejecting all new INVITE messages. 3. Request type: The algorithm inspects the message type of the incoming request. When the machine is starting to be overloaded, the algorithm creates transactions for INVITE and BYE requests only. 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13.
if (forking-request) request.transaction = create-transactions() if (incoming-socket==lossy or outgoing-socket==lossy) { if (CPU-load.isLow()) request.transaction = create-transactions() else if (CPU-load.isMedium() and request==INVITE|BYE) request.transaction = create-transactions() else // high load request.transaction = no-transaction } else request.transaction = no-transaction
Figure 3. Message processing algorithm Figure 3 depicts a simplified version of the proposed algorithm that determines which request needs to create a transaction. Line 1 checks if requests need to fork, if so transactions are created (Line 2). If the incoming or outgoing connection experiences some loss in the recent past, the algorithm allocate a transaction as follows. If CPU cycles are available (line 4), create a transaction (line 5). Otherwise, check if the CPU is heavily loaded, then create a transaction if an INVITE or a BYE request has arrived (line 8). If the CPU is overloaded (line 10) or the incoming and outgoing connections have a low loss rate, no transaction is created (line 13).
4. PERFORMANCE EVALUATION In this section, we evaluate the impact of our proposed algorithm on the system throughput using our IMS simulator, Diabelli [4]. Diabelli implements not only network resources, but also computational resources, such as CPU and threads. Thus, the accurate delay (networking delay + processing delay) can be simulated. The detailed simulation methodology appears in [4]. To operate our algorithm, we set 60% and 75% CPU usage as two thresholds to tune whether to create transactions or not. That is, if the CPU usage is less than 60%, S-CSCF creates transaction states. If the CPU usage is greater than 75%, it does not. When the CPU usage lies in between, S-CSCF checks the current trend of the CPU usage. If it is increasing, S-CSCF does not create transaction states. Otherwise, S-CSCF creates transaction states. We also set 10-5 message error rate as a threshold to determine a lossy link. That is, only if an average loss rate on a link is higher than the threshold, the link is considered as a lossy link. This decision may change based on the loss rate measurement during the simulation. In our simulation, all wired links have an average loss rate of 10-9 BER, while varying the average loss rate on the wireless links as shown in Figures 4-6. In this evaluation, we consider the CPU usage of a SCSCF and an average ringing time that is defined as time between when a caller generates an INVITE message and when the caller receives an 180 Ringing message, where the call arrival rate per second (cps) varies from 100 to 260 and the loss rate (LR) varies from 10-10 to 10-4 in terms of the bit error rate on the wireless link. 100 90
%cpu
80 70 60 50 40 30 100 110 120 130 140 150 160 170 180 190 200 210 220 230 240 250 260 cps LR=10^-10, stateful
LR=10^-8,stateful
LR=10^-6, stateful
LR=10^-10, stateless
LR=10^-8, stateless
LR=10^-6, stateless
Figure 4. Impact of loss rate (LR) on CPU usage without our proposed algorithm Figure 4 depicts the CPU usage of two extreme implementation of S-CSCF, i.e., a transaction stateful or stateless proxy. The results show that a transaction stateful
implementation of S-CSCF reaches a 75% CPU usage with a sustained load of 140 cps. On the other hand, a transaction stateless implementation reaches the same CPU usage level at 205 cps, an improvement over the stateful implementation by 46%. Figure 5 shows the CPU usage of the hybrid implementation of S-CSCF with our algorithm. In this case, the hybrid S-CSCF keeps the number of transactions small. When the loss rate is either 10-10 or 10-8, S-CSCF behaves close to a transaction stateless proxy and is able to reach 75% CPU usage at a load of 196cps, slightly less than that of the transaction stateless proxy by only 4%. When the loss rate is 10-6, S-CSCF demonstrates the transition from creating transaction states to ignoring the states due to the relatively poor quality of the links. In this case, it reaches the 75% CPU usage at 149cps. As it reaches this threshold, S-CSCF processes more messages in the stateless mode and is able to reach 200cps at the same CPU usage level. 100 90 80 % cpu
Notice that the algorithm takes into account both the incoming and the outgoing connections. It is not sufficient to look only at the incoming connection. If the proxy has experienced loss responses from the next hop, a transaction help retransmit final responses as well.
70 60 50 40 30 100 LR=10^-10
150 LR=10^-8
200
250
cps
LR=10-6
Figure 5. CPU usage with our proposed algorithm Notice that reasonable bit error rate in actual networks is in the order of magnitude of 10-8, which makes this application behave mostly on the transaction stateless mode. As a response to sudden changes in link quality, the application is able to progressively turn itself into a stateful mode, until it reaches a limit in the CPU usage, when it turns back to stateless mode. Figure 6 shows the average ringing time when S-CSCF is implemented as a transaction stateful proxy or a hybrid proxy as we vary the call load. When the call load is less than 150 cps, the average ringing time for all scenarios is almost constant. The messages are arriving at slower rates than that the S-CSCF can process. Once the call arrival rate reaches beyond 150 cps, the ringing time of the transaction stateful scenarios increases suddenly, meaning that messages arrive close to or faster than the rate that SCSCF can process and introduce more delay in CPU. Meanwhile the ringing time of the hybrid scenarios is stable for extended call arrival rates, reaching beyond 200 cps. Therefore, our hybrid implementation achieves higher throughput than the transaction stateful implementation without compromising the ringing time.
This figure also shows the behavior of the two approaches network as we vary the loss rates. Between 10-6 and 10-10, the impact is negligible up to 150 cps. The average ringing and call setup time slightly fluctuate. With higher loads, our hybrid algorithm supports up to 250 cps with an acceptable ringing time. We decided not to include the curve for 10-8 since its behavior follows the same curve for 10-10 ringing time. If the network loss rate is 10-4, the ringing time is greatly affected in both stateful and hybrid approach. In this case, message retransmissions become a major factor on the ringing time. The hybrid algorithm performs better than the stateful proxy because the former takes advantages of stateless processing. The aggregate transaction processing cost is still higher than the retransmission cost. Recall that IMS networks have been designed to support wireless access networks (e.g. CDMA) where the network loss is within this range. 4.4
seconds
3.9 3.4 2.9 2.4 1.9 1.4 100
140
145
150
200
205
250
cps
LR=10^-10, stateful
LR=10^-6,stateful
LR=10^-4,stateful
LR=10^-10, hybrid
LR=10^-6,hybrid
LR=10^-4,hybrid
Figure 6. Average ringing time With loss rates below 10-4, the number of lost messages and retransmissions affects the ringing and call setup time. Neither the stateful nor the hybrid approaches were able to complete 98% of the calls successfully.
5. CONCLUSION This paper shows the impact of transactional support in SIP proxies. We showed that a significant amount of computational resources is needed to handle each SIP transaction. Therefore, proxies should minimize transaction usage. SIP proxies can decide to use transactions based on network and CPU usage, as well as the type of the incoming message request. It is highly recommended for SIP proxies to colocate stateful and
stateless functionality. Both the performance advantage of the stateless functionality and the reliability provided by the stateful functionality, allow the SIP proxy to adapt to network and CPU load conditions.
6. ACKNOWLEDGMENTS We like to thank our colleague Volker Hilt for his insightful comments in the early stages of the present work.
7. REFERENCES 1. 3rd Generation Partnership Project, “Network architecture” Rel.6, TS 23.002, V6.7.0, March 2004, www.3gpp.org/ftp/Specs/htmlinfo/SpecReleaseMatrix.h tm. 2. 3rd Generation Partnership Project, “IP Multimedia Call Control Protocol based on Session Initiation Protocol (SIP) and Session Description Protocol (SDP) (Stage 3)” Rel.6, TS24.229, V6.6.0, March 2005, www.3gpp.org/ftp/Specs/htmlInfo/SpecReleaseMatrix.h tm. 3. Cortes, M., Ensor, J.R., Esteban, J., On SIP Performance, Bell-Labs Technical Journal, Vol. 9, No. 3, pp. 155-172, November, 2004. 4. Cortes, M., Esteban, J., Jun, H., Diabelli: An IMS Simulation Tool, Bell-Labs Technical Journal, Vol. 10, No. 4, pp. 255-259, Winter 2006. 5. Gurbani, V., Jain, R., Contemplating Some Open Challenges in SIP, Bell-Labs Technical Journal, Vol. 9, No. 3, pp. 255-269, November, 2004. 6. Iptel.org, SER: SIP Express Router, http://www.iptel.org/ser/ 7. O’Regan, E., Pesch, D., “Performance Estimation of a SIP based Push-to-Talk Service for 3G Networks”, Proc. of Fifth European Wireless Conference, 2004, http://research.ac.upc.edu/EW2004/papers/144.pdf 8. Pous, M., Pesch, D., Foster, G., “SIP-based Applications in UMTS: A Performance Analysis”, Proc. of Fifth European Wireless Conference, 2004, http://research.ac.upc.edu/EW2004/papers/143.pdf 9. Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, A., Peterson, J., Sparks, R., Handley, M., Schooler, E., "SIP: Session Initiation Protocol", RFC 3261, June 2002.
