Swing

Swing: Generating Realistic Background Packet Traces for Large-Scale Internet Topologies Kashi Vishwanath and Amin Vahdat University of California, San Diego kvishwanath,vahdat @cs.ucsd.edu

Abstract

sign space for these applications, it is important to have an understanding of how application communication requirements might interact with a variety of global network characteristics. That is, it is difficult to predict the behavior and robustness of applications sharing communication resources across a network characterized by high variability and rapid evolution. This problem is exacerbated by the pace at which communication patterns change. For example, packet-level traces from the Wide working group [4] show in Figure 1 significant shifts in application popularity within a 28 month period.

One important question facing both network designers and distributed application developers is the interaction of the communication of a single network service or application with aggregate background traffic conditions. Unfortunately, network services are evaluated with typically no competing background traffic or, in the best case, ad hoc and simplistic traffic models. A related question is the effect that widely varying network traffic patterns have on network planning and resource allocation. For instance, the predicted growth of one application versus another (e.g., multimedia versus instant messaging) may have dramatically different impacts on a network’s traffic matrix. Yet, there is currently no way to quantify such effects.

Usage of link over time 60 HTTP NNTP SMTP Napster

Thus, this paper proposes Swing, a flexible workload generator that accurately captures the packet interactions of a wide variety of applications based on a simple set of parameters. Our initial implementation includes settings for HTTP, P2P, and multimedia applications. We verify the accuracy of our application models by extracting application characteristics from available large-scale packet traces and by validating our generated traces against these same traces. We run Swing in ModelNet, a scalable and accurate network emulator to evaluate the interactions of distributed applications with a variety of traffic mixes. We show how to use these runs to measure dynamically changing router queue occupancy to “play back” realistic background traffic scenarios without actually generating any traffic. Finally, we use Swing to model the effect of traffic scenarios and topology on aggregate network load, for instance, as a network planning tool.

50

Percentage by bytes

40

30

20

10

0

0 Time from 01/01/2001 − 04/04/2004

1 Introduction

Figure 1: Trends in usage of protocol by bytes over time for Mawi Traces

Recently, there has been tremendous interest in developing the technologies and techniques necessary to characterize and to understand the Internet. The scale, decentralization, and lack of available instrumentation make it difficult to characterize the network and to plan for its continued growth. At the same time, such missing information makes it difficult to design robust, large-scale distributed services, such as web services, application-layer multicast, and multimedia services. Given the large de-

Thus, the long term goal of our work is to develop a realistic model of background traffic characteristics. For example, for a given network topology we wish to understand the load that individual routers in the topology may experience as a function of a given communication workload. This workload is partially specified by characteristics of currently popular applications, user behavior, and available network resources. By changing assumptions about each of these variables, characteristics of the router work1

load changes. And such characteristics may have significant impact on the design of large-scale distributed services, on network planning, and even for lower-level network protocols (e.g., congestion control or routing protocols). Consider the following motivating scenarios:

the overhead associated with obtaining accurate results. Such tools would benefit from an understanding of the range of representative background traffic conditions and of the limitations beyond which the approaches break down. Robust Distributed Systems: Finally, distributed systems delivering functionality such as applicationlayer multicast [24, 30], must adapt to changing network characteristics. The performance of such structures often depends on making good choices for application-level topology. One significant challenge facing such systems is the ability to rapidly adapt to changing network conditions to maintain robustness and high levels of performance. Once again, an understanding of background traffic characteristics would allow such systems designers to better understand and design their services.

Capacity Planning: Given a fixed capital budget, ISPs must plan upgrades for their network likely to result in the biggest performance improvements for their customers. Making such a decision requires an understanding of current communication patterns as well as how such communication is likely to evolve. A number of research efforts are exploring techniques for efficiently determining the former, e.g., in developing accurate traffic matrices [37]. However, it is unclear how to map projected evolution in user, network, or application characteristics into changing traffic matrices or network workloads.

We present the design, implementation, and evaluation of Swing, a tool for generating a variety of background traffic conditions for existing simulation environments such as ns [7] and for emerging large-scale emulation environments such as Emulab [50] and ModelNet [49]. The initial goal is to subject routers in such simulation and environments to a user-defined set of background traffic conditions, for example, to enable the usage scenarios described above. While recreating the Internet is a difficult task, Floyd and Paxson [17] emphasize that understanding the Internet requires generating workloads capturing the various applications on the Internet.

Web Service Performance: Given the importance of ensuring high quality of service for its customers, designers of web services must understand the endto-end performance delivered to users. For instance, two recent industrial efforts consider extracting endto-end performance information from server packet traces [18] and playing back a given access workload to Google’s web servers [16]. Another recent study [39] shows that wide-area network characteristics can significantly impact web service performance, even for well-provisioned services. Creating a variety of background traffic scenarios would allow network service designers to understand the impact of evolving wide-area characteristics on the performance perceived by its end users and to react appropriately.

While a variety of efforts (enumerated in Section 2) attempt to measure or model various network characteristics in isolation and workload generators exist for a number of applications, we take the novel approach of targeting a simple spanning set of parameters able to capture the end-to-end packet interactions of a range of applications. This approach allows us to experiment with a variety of “what if” scenarios, for example, considering changes in application characteristics (what happens if the distribution of P2P file size shifts in a given manner), a variety of network topologies and link bandwidth distributions, and mixes in application popularity (what if multimedia streaming grows to account for 30% of network sessions).

High-bandwidth Trace Generation: Generating such traces that are representative of a range of scenarios would be invaluable to a variety of activities. For instance, consider high-speed router/switch design where the distribution of packet sizes and destination address (and hence output port) can significantly impact the hardware design. Similarly, efforts into active queue management [32] would benefit from realistic/representative packet traces traveling across individual routers.

This paper makes the following contributions. We present the first unified framework that achieves structural modeling of a wide variety of application traffic. This allows us to not only reproduce existing traces but, by varying the application mix, we can extrapolate to a variety of representative network conditions. We present a detailed validation of our approach, showing that our structure model is promising for capturing the characteristics of a wide

Bandwidth Estimation: There has been significant recent interest in developing tools for measuring the available bandwidth or capacity on an end-to-end path [21, 26, 43]. One of the principal challenges facing these efforts is highly variable background traffic that can pollute measurements and increase 2

range of applications. Next, we generate high bandwidth network traces of an order of magnitude larger than previously produced traces. Finally, we demonstrate the utility of our approach, studying the effect of a variety of background traffic scenarios on end-to-end web server performance and on individual routers within a large-scale topology.

workloads [11] and studies about the way CDNs see traffic [31, 44]. Synthetic traffic Trace Generation: One of the ways to study statistical properties is using traces. A number of live traces are available through CAIDA [12], NLANR [5] and LBL [2]. However, since all of these traces are based on past measurements, it is difficult to extrapolate to other workload and topology scenarios. The challenges associated with obtaining traces from high speed links are discussed in [25]. In [28] the authors discuss a technique to generate a high speed Internet traffic trace based on RAMP [15]. However [15] models only two applications, HTTP and FTP. Moreover we do not make any assumptions about which point the underlying trace is from etc. Here our contribution is that we have been able to generate traffic traces for such high bandwidths.

The rest of the paper is organized as follows. In Section 2 we discuss related work. We present our generic sourcelevel Internet traffic model in Section 3. We then discuss various ways to populate our parameters in Section 4 and to validate our approach in Section 5. We show how Swing can be employed to generate a variety of packet traces in Section 6 and detail a number of case studies in Section 7 to demonstrate the practicality and generality of our approach. We discuss a number of limitations of our approach and present our conclusions in Section 8.

Structural Model: A number of recent efforts [51, 17] stress the need for a structural model of network traffic. GenSyn [22] generates synthetic Internet traffic using stochastic models. Netspec [33] builds source models to generate traffic for telnet,ftp, voice, video and WWW but the authors do not demonstrate if the generated traffic is representative of anything. Finally, [38] describes and application-level structural model for RealAudio Traffic.

2 Related Work Our approach is to build a generic workload characterization framework that can be used to generate packet workloads for a given application and, when used to simultaneously capture the characteristics of multiple application classes, to generate realistic aggregate background traffic workload characteristics. Thus, our goals are related to efforts in application-specific workload generation, synthetic traffic trace generation, network traffic generators, modeling and reproducing of Internet characteristics. We discuss the relationship of our work to these efforts in turn below.

Source level modeling of HTTP [14] has been done but the model consists of a large number of parameters which is a mix of both application dependent and network dependent characteristics. Hence, it is unclear how to interpret the results and apply it to different scenarios. Other traffic generators: There are a number of other available traffic generators like iperf [3], netperf [6], flowgen [1] and TG [8]. However, none of them attempt to capture realistic Internet communication patterns, rather leaving the task of parameterizing the traffic generator to the end user. In some sense, our work can be viewed as targeting appropriate parameters and application mixes for existing traffic generators.

Application-Specific Workload Generation: There have been many attempts to design application-specific Workload generators. Examples include Surge [10] for HTTP following the lessons learned from the empirical models for web traffic proposed in the literature [36, 41]. Cao et.al [14] perform source level modeling of HTTP. However, they attempt to parameterize the RTTs seen by the client rather than capture it empirically. We distinguish between the responsibilities of the traffic generator and the underlying topology, enabling analysis of the impact of application usage patterns under varying topologies.

Swing is not just aimed at capturing application specific behavior and replaying it. It is also aimed at analyzing the correlation of a number of applications simultaneously competing for network resources. Our work is complementary to a number of research efforts including modeling usage pattern of the Internet, generating accurate background traffic, modeling of network characteristics like congestion, etc.

Likewise there have been attempts to understand the P2P workload at the packet [44, 20, 45] and flow level [46]. However, we are not aware of any real workload generator except for simulations. We employ these workload characterizations to populate our model of traffic generation for P2P systems. Similarly, we leverage multimedia workload characterization by MediSyn[48] and Real Audio [38] to inform our own models of multimedia traffic. Finally, there have been attempts to model SMTP

Capturing all applications in one: There have been attempts to classify applications based on the patterns in communication [13, 47]. The authors employ this approach to identify clusters of applications that show similar behavior. We consider this work complementary to

3

ours [23].

ability to scale to large-scale environments (tens of thousands of routers with aggregate bisection bandwidth measured in gigabits per second running on a commodity cluster of UNIX workstations). Given an appropriate model for the number, location, and communication pattern of a target mix of applications exchanging packets over a target network topology, we can measure the effective load placed on individual router links by instrumenting the emulation environment.

Modeling Congestion: [19] tries to model the observed congestion in background routers. We have a source level approach to it. The higher level goal of the workload generator is to to facilitate the study of metrics like queue occupancy in routers. Thus, once we are able to model queue occupancy according to realistic workloads, we no longer need to generate synthetic traffic and can instead have routines which delay packets in the “router” appropriately.

At this point, our methodology may be used in a number of different scenarios. For instance, a target application of interest (a web service, a Grid application, a peer-topeer system, etc.) may be run in the emulation or simulation environment to allow application developers to determine the resilience of their application to bursty cross traffic. Another option is to use a run of background traffic in isolation to attempt to model the characteristics of background traffic based on particular application mixes or topology characteristics, e.g., perhaps routers at different locations in the network (“transit” versus “stub” or “long-haul” versus “access”) may have different queue occupancy characteristics that can be measured and modeled. These models can be used to influence queue management policies or may be used in subsequent emulation runs to execute an application in isolation with modeled background traffic on a per-link basis. Such modeling would enable larger-scale emulations because the actual per-packet effects, traffic sources and sinks need not be emulated. A final use of our methodology is for capacity planning. Consider that while holding the distribution of clients, servers, and applications constant, we can vary the emulated topology. Thus, for instance, an ISP may employ this approach to determine which links are likely to see congestion as its user population grows and to experiment with a variety of “what-if” scenarios to determine appropriate additions to its topology for best return on investment. Similarly, the ISP may project a variety of application workload scenarios, e.g., what if multimedia traffic were to grow by 30% over the next year?

Background traffic: One recent effort [39] studies the effect of WAN conditions on web server performance and concludes that losses and delays can affect response times. One goal of our work is to both generalize and standardize such studies, e.g., to determine the effect on end-client perception of web server performance for a variety of traffic conditions and network topologies. [52, 53] have examined the temporal stability of Internet performance through active measurements. Lucas et al. [35, 34] have characterized the nature of wide area IP traffic and have developed a traffic model called (M,P,S) to generate traffic for simulation. Until now, no one now has generated realistic traffic and used it to measure performance on applications. One goal of our work is to minimize the resources spent in producing accurate background traffic characteristics, naturally corresponding to an increase in the available scale of emulation and simulation-based studies. Scaling Simulations: There have been attempts to scale the scope of simulations. [40, 42] look at sampling the input traffic and feeding it into a scaled down simulator and later extrapolating the results. Although we consider this work complementary to ours there are two issues this exposes: i) Simulation does not capture congestion accurately and ii) it is not clear how to scale background traffic.

3 Generic Internet Workload Model

Fundamental to the above methodology is developing a generic parameterizable model for the communication patterns for distributed applications. We do not wish to execute “real” applications because of the CPU, memory, and storage overhead associated with such applications, the difficulty in obtaining the source or executable for the applications in certain cases, and in configuring the applications to match a target workload. Instead, we take the approach of developing a structural model for packet communication based on the characteristics of a broad range of applications. Thus, we wish to ask questions that determine the transport protocol, the size of a “request” and “response”, number of requests per session,

The overall goal of Swing is to generate realistic background traffic levels on a per-router link basis for a given topology and application mix scenario. To first extract and to then generate target background traffic models, we take the approach of emulating the per-packet interactions of a broad range of applications spread across a target userspecified topology. While our technique is general to any network simulation or emulation environment, we employ the ModelNet [49] emulation environment because of its ability to host unmodified applications running on unmodified end hosts, our familiarity with the code base, and its 4

pause time between requests, the distribution of the client population, the relative popularity of the server population, etc. The goal is to develop a simple, spanning set of parameters that can be applied to a broad range of applications, faithfully capturing the packet interactions and enabling exploration of the communication space as workload characteristics change. Note that we are not concerned with the contents of any packet and we need not perform any application-layer processing. We simply care about the communication load placed on the network by a given type of application. One significant benefit of this approach is that it is topology independent. Hence, we need not be concerned about modeling packet arrival rates at individual routers, simply the behavior of individual applications and the placement of application endpoints (clients, servers, etc.) in a target topology.

Response characteristics: Finally, we characterize individual responses with the packet size distribution, whether it involves constant bit rate communication, etc. Another interesting question for certain applications is whether a response is causally related to a subsequent request/response pair, perhaps to a different destination [9]. We do not consider this last question for the purposes of this study. Thus, each Swing application consists of a number of user sessions, whose characteristics we summarize in Table 1. We view each user session as a series of RequestResponse Exchanges (RREs). Each RRE consists of a number of flowpairs separated by inter-RRE think times. Each flow pair is characterized by a server (to request a response from), request size, a server think time (before it acts upon the request), and a response size. There are bitrates and packet size distributions corresponding to responses from the server. The transport can be either TCP or UDP.

The goal of this section is to motivate and describe the initial structural model that we have developed for Swing and to demonstrate how we map a number of important existing application classes to this model based on both network and end-application traces.

While a number of models for parameterizing traces have been proposed in the past, to the best of our knowledge our approach is the first to capture the characteristics of a broad range of applications in a unified manner. We do not claim that our chosen set of parameters is complete, but in conjunction with our validation below it does illustrate that a relatively simple model can adequately capture the characteristics of disparate application classes. We next describe the procedure to populate the parameters of our model.

We divide our task of finding suitable parameters into four categories: Users: The communication characteristics of a variety of applications are determined by end users. Important questions include how often the user becomes active, which remote sites the user visits and with what distribution, think time between individual requests, etc. Note that certain applications (such as SNMP) may not have a human in the loop, in which case we use this category as a proxy for any regular, even computer-initiated behavior.

4 Populating Application Models 4.1

Sessions: Given an understanding of how often a user becomes active, the next category considers the characteristics of individual sessions. For instance, does an activity correspond to downloading multiple images in parallel, are different chunks of the same mp3 downloaded in parallel, etc. Thus, one important question regards the number and target of individual connections corresponding to a single session.

Determining Parameter Values

There are a number of ways to populate the parameters of our source level traffic model. We describe some of them here. One approach is to use the results of analytic models developed in the research literature for each of the parameters in Table 1. There are a number of problems with such an approach. First, it would amount to determining the characteristics of each of the parameters from an independent study. Although there are a number of studies of for instance, HTTP and P2P application characteristics, there is limited information available for some of the other application classes. Further, fitting observed data into an analytic model limits the scope of experiments. We cannot for example make conclusions about the effects of scaling individual parameters, considering different topologies, or projected changes in application behavior.

Connections: Next, we consider the characteristics of individual connections within a session, such as the target of the connection, the number of request/response pairs within a connection, the size of the request and corresponding response, wait time before generating a response (e.g., corresponding to CPU and I/O processing overhead at the endpoint), spacing between requests, the type of transport (e.g., TCP vs. UDP) involved, etc.

Alternatively one could derive CDF’s of application char5

Layer Client characteristics

RRE characteristics FlowPair characteristics

Packet characteristics (request/response)

Variable Client ID Number of RREs Inter RRE think time Number of FlowPairs Server ID Number of Request/Response Exchanges per connection Request size Server think time Response size Transport packet size distribution packet arrival distribution

Our Model Client IP num exchange User think-time num connections Server IP numpairs Req Size Server think time Resp size TCP for HTTP,P2P, UDP for MultiMedia packet size brate for udp

Table 1: Structural Model of Traffic

fer the number of RREs and the number of flowpairs per RRE we track the total number of simultaneous flows at any point. This value corresponds to different flowpairs in our model. If after the end of the last flow in such a flowpair 60 seconds elapse without the origin of any other flow then we mark it as the end of a session. However if a flow starts then it corresponds to a new RRE.

acteristics from packet traces to drive experiments. The advantage of this approach is that it faithfully reproduces whatever is seen across the network. Swing gives the user the ability to either choose from any of the available well known distributions or to choose from the CDF. For the experiments in this paper, we chose to populate the models using the distributions observed in available widearea network traces. We begin with network level traces with no application level headers from NLANR, CAIDA, and WIDE. We also employ P2P application studies [44, 20], MediSyn logs for Multimedia [48] and Surge data for HTTP [10].

In the following subsections, we describe how we parameterize our model for three popular application classes, HTTP, peer-to-peer applications, and streaming media. We extract network characteristics as cumulative distribution functions based on a variety of available large-scale packet traces. For this paper we focus on the Mawi [4] traces from a trans-Pacific 100Mbps link. As discussed earlier, we could also populate our parameters using available analytical models for various parameters. While we cannot claim that our model and parameters are sufficient to capture the communication characteristics of all existing and future applications, we do believe that the model does show significant promise as a starting point for developing a true minimal and spanning set of parameters.

We wrote simple tools to extract relevant information from the traces. For instance, we mapped packets to individual applications based on the TCP port number, e.g., ports 80 and 443 for HTTP, and 25 for SMTP. We aggregate the characteristics of applications that we do not explicitly consider into a single “other” class. We discuss some of the limitations with this approach in Section 8. Extracting application-level characteristics from packet traces is a well-studied area [47] and we omit the details of our tool for brevity. Rather, we focus on how we employ our tools to determine appropriate values for the parameters outlined in Table 1. It is straightforward to determine the distribution of origin and destination IP addresses from a given packet trace. We use this same distribution to guide the selection of IPs in our workload generator. Alternatively, we could use the results of analytical studies [29] for this parameter. The number of request/response exchanges can be inferred by measuring the number of times requests and responses alternate in a single connection. Packet size, bit rate, and transport protocol distributions can similarly be inferred in a straightforward manner from packet-level traces. To in-

4.2

HTTP

To parameterize our variables for HTTP, we must determine: i) the client initiating the session, ii) the total number of web pages the user visits, iii) the think time between page accesses, iv) the total number of connections the client opens, v) the destination address for each connection, vi) the processing time at the destination to generate the response, vii) whether subsequent requests are made on the same connection, and viii) the transmission rate for the response. We collect all this information from our traces and plot a subset of the CDF’s for these values below. As discussed earlier, we use these CDFs to inform the distribution of client/server pairs and their communi6

cation patterns during live emulation runs on a variety of topologies. In each case, we also validated Swing’s ability to generate background traffic with similar characteristics during such emulation. The results of these experiments, which are described in detail in Section 5, are plotted in the Figures below with the curve labeled “Swing”. They are compared to “Caida” or “Mawi” as the case maybe.

Empirical CDF 1

0.9

Cumulative fraction of response

0.8

Response size: Figure 2 and Figure 3 compare the CDF of the response sizes generated for the Mawi trace and the Caida trace. The x-axis shows response size in bytes and the y-axis plots cumulative probability. In both the cases the curves overlap each other showing that the distribution generated is the same as the distribution we start with.

0.7

0.6

0.5

0.4

0.3

0.2

0.1 Swing_http Mawi_http 0 0 10

Request size: Figure 4 and Figure 5 compare the CDF of the request sizes generated for the Mawi trace and the Caida trace. The x-axis shows request size in bytes and the y-axis plots cumulative probability. Once again the graphs show that the distribution generated is the same as the distribution we start with.

1

2

10

10

3

4

5

6

10 10 10 Response Size in bytes (log scale)

7

10

10

8

10

Figure 2: Response size comparisons Swing and Mawi Empirical CDF 1

0.9

Num RRE: Figure 10 and Figure 11 compare the CDF of the number of RRE per session for the Mawi trace and the Caida trace. The x-axis shows number of RREs and the y-axis plots cumulative probability.

0.8

Cumulative probability

0.7

Num Conn: Figure 6 and Figure 7 compare the CDF of the number of simultaneous connections generated for the Mawi trace and the Caida trace. The x-axis shows number of simultaneous connections and the y-axis plots cumulative probability.

0.5

0.4

0.3

0.2

Num Pairs per connection: Figure 8 and Figure 9 compare the CDF of the number of request response exchanges on a single connection simultaneous connections generated for the Mawi trace and the Caida trace. The x-axis shows number of simultaneous connections and the y-axis plots cumulative probability.

0.1 Swing_http Caida_http 0 0 10

1

10

2

3

4


5

10

Figure 3: Response size comparisons Swing and Caida

Spread of clients: Figure 12 and Figure 13 compare the spread of clients over sessions for the Mawi trace and the Caida trace. The x-axis shows the fraction of clients and the y-axis plots cumulative probability.

the network. For instance, it may be impossible to reconstruct file size distribution from a packet trace because each node may retrieve a single file from many destinations. Fortunately, from our perspective, we are simply interested in the network communication effects of individual applications rather than application-level behavior. However, one limitation imposed by such limited knowledge is our inability to, for instance, scale projected file size in peer to peer file sharing applications (perhaps signaling a transition from music to video) in a straightforward manner. Based on the traces available, we are able to determine when a user submits a query to find locations to retrieve a file from. This may be to multiple servers or a single server. Once the server replies, the user typically transmits multiple requests to download

Spread of servers: Figure 14 and Figure 15 compare the spread of servers over sessions for the Mawi trace and the Caida trace. The x-axis shows the fraction of servers and the y-axis plots cumulative probability.

4.3

0.6

P2P

We next consider parameterization of peer-to-peer workloads. One difficulty with considering this workload is that it is difficult to reconstruct a number of applicationlevel characteristics from packet-traces in the middle of 7

6

10

Empirical CDF 1

0.9

0.9

0.8

0.8

0.7

0.7

Cumulative fraction of FlowPairs

Cumulative fraction of requests

Empirical CDF 1

0.6

0.5

0.4

0.3

0.2

0.6

0.5

0.4

0.3

0.2

0.1

0.1 Swing_http Mawi_http

0 1 10

2

10

3

4

5

10 10 Request Size in bytes (log scale)

Swing_http Mawi_http 0 0 10

6

10

10

1

10

2

3

4

10 10 Number of simulateous connections (log scale)

10

Figure 4: Request size comparisons Swing and

Figure 6: Number of simultaneous connections

Mawi

comparison Swing and Mawi Empirical CDF 1

0.9

0.9

0.8

0.8

0.7

0.7 Cumulative probability


Empirical CDF 1

0.6

0.5

0.4

0.6

0.5

0.4

0.3

0.3

0.2

0.2

0.1

0.1

Swing_http Caida_http 0 0 10

1

10

2

3

4

10 10 10 Request Size in bytes (log scale)

5

10

Swing_http Caida_http 6

10

0 0 10

1

10

2

3


4

10

Figure 5: Request size comparisons Swing and

Figure 7: Number of simultaneous connections

Caida

comparison Swing and Caida

non-overlapping portions of the same file from multiple destinations. Based on this model, we present CDF’s populating our peer to peer application model below.

Request size: Figure 4 compares the CDF of the request sizes generated for the Mawi trace. The x-axis shows request size in bytes and the y-axis plots cumulative probability.

Response size: Similar to HTTP, we want to measure the size of generated responses. Note that the response size for our purposes is the total number of bytes the server sends in response to a P2P request. Whether this happens to be an entire file or just a portion of the same is not of consequence. We consider Napster traffic in this case, as it was the popular peer to peer file sharing application at the time of this trace. Figure 16 shows on the x-axis the response size in bytes on a log scale and the cumulative fraction on the y-axis.

Num RRE: Figure 10 compares the CDF of the number of RRE per session for the Mawi trace. The x-axis shows number of RREs and the y-axis plots cumulative probability. Num Conn: Figure 6 compares the CDF of the simultaneous connections generated for the Mawi trace. The x-axis shows number of simultaneous connections and the y-axis plots cumulative probability. 8

5

10

Empirical CDF 1

0.9

0.9

0.8

0.8

0.7



Empirical CDF 1

0.6

0.5

0.4

0.6

0.5

0.4

0.3

0.3

0.2

0.2

0.1


0 0 10

1

2

3

4

Swing_http Mawi_http 0 0 10

5

10 10 10 10 Number of Request/Response exchanges on a connection (log scale)

10

1

2

3

Figure 8: Number of request response exchanges on

Figure 10: Number of Request Response exchanges

a connection comparison Swing and Mawi

per session comparison Swing and Mawi

10

Empirical CDF

1

1

0.9

0.9

0.8

0.8

0.7



Empirical CDF

0.6

0.5

0.4

0.6

0.5

0.4

0.3

0.3

0.2

0.2

0.1

0.1 Swing_http Caida_http

0 0 10

4

10 10 10 Number of Request Response exchanges (log scale)

1

2

3

4

10 10 10 10 Number of Request/Response exchanges on a connection (log scale)

Swing_http Caida_http 0 0 10

5

10

1

2

3

10 10 10 Number of Request Response exchanges (log scale)

Figure 9: Number of request response exchanges on

Figure 11: Number of Request Response exchanges

a connection comparison Swing and Caida

per session comparison Swing and Caida

characteristics of P2P workloads as observed and reported in [44, 20]. Figure23 compares the spread of clients over requests between the UWash data and Swing generated traces. Similarly Figure24 compares the spread of clients over requests between the UWash data and Swing generated traces. These are the only two parameters for which we had access to the raw data.

Num Pairs per connection: Figure 8 compares the CDF of the number of request response exchanges on a single connection generated for the Mawi trace. The x-axis shows number of simultaneous connections and the y-axis plots cumulative probability. Spread of clients: Figure 12 compares the spread of clients over sessions. The x-axis shows the fraction of clients and the y-axis plots cumulative probability.

4.4

Spread of servers: Figure 14 compares the spread of servers over sessions. The x-axis shows the fraction of servers and the y-axis plots cumulative probability.

MMStream

Finally, we consider parameterization of streaming media applications. This application class does not currently comprise a significant fraction of any packet trace that we had available. Further, it demonstrates significantly different characteristics than either of the two application

Comparison to Uwashington data: Here we demonstrate the ability of Swing to reproduce 9

4

10

1

0.9

0.9

0.8

0.8

0.7



1

0.6

0.5

0.4

0.6

0.5

0.4

0.3

0.3

0.2

0.2

0.1


0

0

0.1

0.2

0.3

0.4 0.5 0.6 Cumulative fraction of clients

0.7

0.8

0.9

Swing_http Mawi_http 1

Figure 12: Spread of clients over sessions compari-

0

0.1

0.2

0.3

0.4 0.5 0.6 Cumulative fraction of servers

0.7

0.8

1

0.9

0.9

0.8

0.8

0.7


1

0.6

0.5

0.4

1

0.6

0.5

0.4

0.3

0.3

0.2

0.2

0.1

0.1 Swing_http Caida_http

0

0.9

Figure 14: Spread of server over sessions comparison Swing and Mawi

son Swing and Mawi


0

0

0.1

0.2

0.3


0.7

0.8

0.9

Swing_http Caida_http 1

Figure 13: Spread of clients over sessions compari-

0

0

0.1

0.2

0.3


0.7

0.8

0.9

Figure 15: Spread of server over sessions comparison Swing and Caida

son Swing and Caida

Response bitrate: Next, we measure the file playback bitrate. Figure 25 shows on the x-axis the bit rate in bytes per second and on the y-axis the cumulative fraction.

classes we considered earlier. To demonstrate the generality of our approach, we collected media server logs from a large US-based firm. The log entries consist of, among other things, the client IP address, the duration of file playback, and the average sustained bandwidth. We use this information to recreate the traffic. In this model, a user first sends a request to a streaming media server. The server then starts sending data over UDP at a fixed bitrate. In the network, we consider a sequence of UDP packets from a server to a particular client and infer that it comprises multimedia data for a single file. We plot our measurements of this trace below.

5 Validation All our experiments were run on top of ModelNet [49]. The experiments consisted of 15 Linux machines and 1 FreeBSD machine interconnected with 1 Gbps Ethernet switches. We configured the linux machines to route all their packets to the FreeBSD machine. We run multiple instances of our workload generator on each physical machine. The FreeBSD machine acts as an emulator, capturing the per-hop bandwidth, latency, queueing of a user-specified large-scale topology. For many of our experiments, we use a 5000-router INET topology [27] with

Response duration: This measures the total file playback time. Figure 26 shows on the x-axis the file play duration in secs and on the y-axis the cumulative fraction of flows. 10

1

Empirical CDF 1

0.9

0.9

0.8

0.8

0.7



Empirical CDF 1

0.6

0.5

0.4

0.6

0.5

0.4

0.3

0.3

0.2

0.2

0.1

0.1 Swing_P2P Mawi_P2P

0 0 10

1

10

2

10

3

4

5


6

10

7

10

Swing_P2P Mawi_P2P 0 0 10

8

10

1

10

2

3

4

10 10 10 Request Size in bytes (log scale)

5

10

6

10

Figure 16: Response size comparisons Swing

Figure 17: Request size comparisons Swing

traffic sources and sinks assigned to 1000 random points in the topology.

We attempt to set bandwidth and latency values for the links connecting to the dumbbell from individual traffic generators based on observed characteristics in the trace, as follows. For latency, we calculate the time difference between the SYN+ACK sent by the server and the ACK for the first byte of request sent by the server for every connection. The minimum of these values over a large number of connections is an approximation of the RTT between the client and server. Figure 28 shows the distribution of RTTs as observed in the trace. Similarly for bandwidth we take the time elapsed between the first and the last byte of response and divide that by the total time elapsed. Figure 29 shows the distribution of bandwidths as observed in the trace.

We first set out to determine whether the application characteristics produced from our emulation runs matched the observed characteristics of our available Internet packet traces. Our methodology is as follows. We first populate the parameters of the source level model and the traffic mix using the techniques described in section 4. We then feed in this mix to the generator so that it can generate a proportional mix of the applications. Next, we run our workload generator through ModelNet using tcpdump on the FreeBSD core to capture the aggregate bandwidth generated by the workload generator. We compare this trace against the publicly available traces initially used to populate our models. Our metrics for comparison include the degree to which: i) generated traffic possesses the same properties as that of the trace we started from, ii) end-to-end application characteristics, such as web-page download latency match, and iii) the aggregate properties of the trace are the same.

The accuracy of this analysis is of course compromised by the sometimes small number of samples for a particular client/server pair and the presence of multiple dynamically changing points of congestion. While we believe our validation is an appropriate initial approximation, it is clear that the packet trace across our emulated bottleneck in the dumbbell cannot fully match the characteristics of our available packet traces for a single link in a much larger topology. Nevertheless, the results of our validation are promising.

Initially, it may seem trivial that these characteristics should match in all cases given that we use measurements from a given trace to populate the models that drive our traffic generation. However, note that our traces are for a single link in a much more complex network environment while our experiments are run end-to-end in our own target topology. To match trace characteristics to the extent possible, we employ a simple dumbbell topology for our emulation where a single bottleneck link, with capacity set to the largest value seen in the trace, connects traffic generation sources and sinks on either side of the dumbbell. Figure 27 depicts this configuration.

5.1

Trace

Given the above general methodology, we now present our validation of the Mawi trace, taken from February, 6th 2002 [4] and the Caida trace taken from August 14, 2002. Mawi Trace Validation: We begin by filtering the trace for only HTTP packets. 11

Empirical CDF 1

0.9

0.9

0.8

0.8

0.7



Empirical CDF 1

0.6

0.5

0.4

0.6

0.5

0.4

0.3

0.3

0.2

0.2

0.1


0 0 10

1

2


Swing_P2P Mawi_P2P 0 0 10

3

10

1

2

3

10 10 10 Number of Request/Response exchanges on a connection (log scale)

4

10

Figure 18: Number of simultaneous connections comparison

Figure 19: Number of request response exchanges on a con-

Swing

nection comparison Swing

The total size of responses in this trace divided by the total time elapsed (1253 seconds) results in an average bandwidth of 7.79 Mbps for HTTP. Next we run the background traffic generator to determine if the total number of useful bytes received at the clients is the same. In our generator we get a total response bandwidth of 7.47 Mbps when running for 93 seconds.

all, we are satisfied with Swing’s ability to generate application packet characteristics matching either analytical models or measured distribution functions.

6 Generating a variety of traces The real power of Swing comes from its ability to generate traces for a wide variety of scenarios. These scenarios correspond to projected trends in usage of the Internet in the future. For instance increase in the percentage of dynamic web content corresponds to an increase in the computing overhead at the server as oppose to static content. This can manifest in increase latency to download files etc. Similarly increase in the sizes of response. There can also be a relative change in popularity of applications and in such scenarios we want to be able to predict the effect of this on end users as well as routers for instance.

Caida Trace Validation: Again we filter the trace for only HTTP packets. The total size of the responses and requests in this trace was divided by the total time elapsed to get an average bandwidth of Mbps. Next we run the background traffic generator to determine the total number of useful bytes received at the clients. In our generator after 29.67 the total sizes of request and response resulted in avg bandwidth of Mbps.

To our best knowledge Swing is the first effort that generates realistic traces corresponding to a wide range of scenarios not just of the past or present but also looking into the future. Thus it is an invaluable tool to experiment with the application characteristics, the users as well as the underlying topology at the same time.

Figures 2- 22, first described in Section 4, also include runs of Swing in the above experimental setup using ModelNet to emulate a dumbbell topology. Visually, the CDFs for various distributions in the Mawi(Caida) trace match the same characteristics for our Swing-generated packet traces for both HTTP and P2P workloads. More rigorously, the results of KSTests are in most cases is 0 (with a significance level of 0.05), indicating that the original Mawi(Caida) trace and the Swing-generated results are from the same CDF family. Our results for validation of the multimedia traces from the end-server perspective in Figures 25- 26 also match similarly well. Of course, there are a number of potential sources of error to our validation, explaining the discrepancies where present. Over-

In this section we demonstrate how we can use Swing to generate these different traces and also show that how these traces are different to each other.

6.1

Varying application characteristics

First, we consider what happens when we change the characteristics of applications. This involves introducing 12

1

Empirical CDF 1

0.9 0.9 0.8 0.8 0.7 Cumulative probability


0.7

0.6

0.5

0.4

0.6

0.5

0.4

0.3 0.3 0.2 0.2 0.1 Swing_P2P Mawi_P2P

0.1 Swing_P2P Mawi_P2P 0 0 10

1

2

10 10 Number of Request Response exchanges (log scale)

0

0

0.1

0.2

0.3

3

10


0.7

0.8

0.9

1

Figure 21: Spread of clients over sessions comparison Swing

Figure 20: Number of Request Response exchanges per session comparison Swing

1

0.9

new applications as well as changing the properties of existing ones.

0.8


0.7

6.1.1 Varying file size We take one example in this category as changing the response size generated, beginning with the simple case of file size distribution for HTTP. For example,we want to answer questions like what happens if the average file size

increases by either a factor of or a factor of .

0.6

0.5

0.4

0.3

0.2

0.1 Swing_P2P Mawi_P2P 0

We run the following tests using Swing on top of ModelNet. We start with a simple dumbbell topology with 500 nodes attached to either side of the dumbbell to act as clients/servers. The bottleneck link has a bandwidth of 100Mbps and latency of 1ms. The clients/servers are attached to this link using 10Mbps links with a latency of 0.5 ms. We configured our routes in ModelNet such that all communication between pairs of nodes in this topology necessarily used the bottleneck link. Thus the RTT between any pair of hosts is 4ms.

0

0.1

0.2

0.3


0.7

0.8

0.9

1

Figure 22: Spread of server over sessions comparison Swing

We next configure Swing to generate background traffic as a result of HTTP. The parameters of HTTP are chosen based on the Mawi trace. We also show how simply changing the average response size changes the average bandwidth of trace generated. File Size: First we show the CDF for different files sizes in Figure 31. Latency: Next, we plot the total connection time of objects in Figure 30. 13

Bandwidth: Finally, we measured the aggregate bandwidth in the trace across the bottleneck as a function of the file size. For normal file sizes, the bandwidth is 2.16Mbps. For two times the file size, the bandwidth predictably jumps to 5.33Mbps while it increases to 13.32Mbps for five times the file size.

6.2

Changing user characteristics

We also consider how changing the characteristics of the end user manifests itself in network-level properties. As one example, we consider shifting the popularity of applications from Web objects to Multimedia. In our hypothetical scenario, we vary the mix between HTTP, P2P and Multimedia sessions. We generate a total of 200 user sessions in 30 seconds. We consider three different cases

1

0.9

0.9

0.8

0.8

0.7



1

0.6

0.5

0.4

0.6

0.5

0.4

0.3

0.3

0.2

0.2

0.1


0

0

0.2

0.4

0.6 0.8 Cumulative fraction of clients

1

1.2

Swing_P2P Mawi_P2P 0

1.4

Figure 23: Spread of clients over sessions comparison of Swing and Uwash data

0

0.2

0.4

0.6 0.8 Cumulative fraction of servers

1

1.2

1.4

Figure 24: Spread of servers over sessions comparison of Swing and Uwash data

7 Case Studies of popularity with 190 of these 200 sessions belonging to HTTP, P2P and Multimedia respectively and the remaining 10 sessions equally divided among the other two applications. In the face of this background traffic, we configure clients to fetch 4KB files from an Apache Webserver both with and without background traffic, as shown in Figure 32. Interestingly, the mix of background traffic significantly impacts client-perceived latency, which may have implications for the design of web services in the face of a variety of network conditions.

6.3

Sensitivity to network topology

Finally, we consider how changing the underlying topology while keeping the communication pattern constant affects network level measures such router queue occupancy. Figure 33 shows the results for varying the latency across the bottleneck link in the dumbbell topology. There are two lines corresponding to the topologies where the RTT between client/server pairs is 30 ms and 90 ms respectively. If we consider INET topologies then the comparison is shown in Figure 34 for file download latencies for files. The three INET topologies differ in the way bandwidths are assigned to the links. Inet06 is the most constrained of the topologies with bandwidths varying between 1-10Mbps, Inet04 is in the middle with bandwidth assigned at 10-100 Mbps and Inet09 effectively has unlimited bandwidth, with all links set to a bandwidth of 1 Gbps.

We now present a number of cases studies to demonstrate the range of scenarios where our results apply. We are largely making the case that the presence of background traffic significantly impacts the observed performance and that the performance also depends on the characteristics of the generated background traffic.

7.1

Queue Sizes in routers

ModelNet allows us to observe the queue occupancy in routers as a function of input traffic on that link. We want to see how this depends on background traffic. For this experiment, we run the background traffic generator and observe the average number of queue slots occupied for any incoming packet for a given link, outputting this statistic for all links each second. We next modify ModelNet to read the queue occupancy recorded earlier and to play back the corresponding queue occupancy by marking each link’s queue as virtually occupied based on the earlier measurements. Thus, for every incoming packet, if the total of the already queued packets plus the virtual queue exceeds the capacity of the queue then the packet is dropped. Additionally, if the packet is enqueued, it is then subject to an additional queueing delay because of the virtually queued packets. To demonstrate the validity of this approach we perform the following experiment. First, we have clients making requests to a Webserver to fetch files of varying sizes. We record the time taken to download these files. Next, we run our background traffic generator while requesting the same files in the same pattern. We also run the back-

14

Empirical CDF

Empirical CDF

1

1 MMServer Swing

0.9

0.9

0.8

0.8

0.7



MMServer Swing

0.6

0.5

0.4

0.6

0.5

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0 3 10

4

10

5

0 0 10

6

10 File bitrate (bytes/sec), logscale

10

Figure 25: Comparison of file bitrates

Swing lets us generate high bandwidth traces by carefully extrapolating the parameters of the original trace we start with. We demonstrate this by doing the following experiment. We first learn the values of the parameters by starting with the Caida trace. Next we vary various parameters like user population, response size etc. to generate high bandwidth traces. Reproduce original OC-48 trace.

3

4

10 10 File play duration in secs , logscale

5

10

6

10

7

10

Figure 27: The link under interest connects the two client/server clouds Decrease the number of sessions per unit time by

Mbps. . Set request size to a constant of 1000 bytes and response size to a constant 320000 bytes. Set number of request response exchanges per connection to 10 and the number of parallel connections per session to 10. Thus resulted in an overall trace with avg utiliza tion of Mbps and at that stage we were limited by the CPU saturating at core.

High Bandwidth Trace Generation

2

10

Figure 26: Comparison of file play durations

ground traffic generator, but without the requests for the web files while recording the router queue occupancy for the links of interest. Finally, we use this statistic to feed into the new module that accounts for background traffic using router queue occupancy. We perform the file downloads while emulating the network with this new module. Figure 35 shows these result for downloading a 984KB file. The curve for no background traffic shows the improved performance of the system when the network is clear. When background traffic is introduced, the file download slows by approximately 30% in the median case (note that the x-axis does not start at 0). Finally, the curve labeled “Router Changes” demonstrates our ability to accurately emulate a target level of background traffic without actually playing back the trace. While these results are initial, we believe they have significant implications for our ability to scale large-scale emulations, for end-to-end application performance evaluation, active queue management policies, etc.

7.2

1

10

8 Discussion and Conclusions

Mbps.

There are many avenues for future for the framework presented in this paper. For instance, Swing suffers from a number of limitations including:

Increase the response size and request size by times each. Mbps 15

Empirical CDF 1

0.9

0.9

0.8

0.8

0.7


Cumulative fraction of source/destination pairs

Empirical CDF 1

0.6

0.5

0.4

0.6

0.5

0.4

0.3

0.3

0.2

0.2

0.1

0.1

Normal File Size File Size * 2 File Size * 5

Latency between src/dest pairs 0 0 10

1

10

2

3

10 End−to−End Latency in ms

0 −3 10

4

10

10

−2

10

−1

0

10 10 Download latency in sec

1

10

2

10

Figure 30: Comparison of download latencies when varying

Figure 28: Distribution of RTTs in the mawi trace

avg response size of HTTP objects Empirical CDF 1

not be able to derive complete application characteristics or generalize to some existing or emerging application class.

Cumulative fraction of source/destination pairs

0.9

0.8

0.7

The CDFs that we use to populate our structural model are derived from a single trace and hence are not necessarily representative of a particular application class. We argue that the value of our work lies not in the realism or representativeness of a particular trace but the ability to faithfully capture the characteristics of a particular application scenario, whether realistic based on a particular trace or projected based on speculation of a new application domain or the evolution of an existing application domain.

0.6

0.5

0.4

0.3

0.2

0.1 Bandwidth between src/dest pairs 0 −2 10

−1

10

0

10

1

2

3

10 10 10 End−to−End Bandwidth in kbps

4

10

5

10

6

10

Figure 29: Distribution of Bandwidth in the mawi trace We have currently only validated that our model is appropriate for HTTP, P2P, and streaming media applications. While we believe that our approach is general to a variety of applications, we leave validation of other application classes including SMTP, SNMP, NTTP, FTP, etc. to future work.

We use port number to categorize applications. However, a number of disparate application classes are running over port 80, e.g., tunneled over HTTP. We cannot distinguish between multiple sessions and single sessions employing multiple connections based solely on packet traces. While we do lose some accuracy in modeling user behavior, we believe that the resulting packet communication load should remain accurate.

In conclusion however, we believe that this paper takes an important step in the developing realistic background traffic models with significant benefit to: i) simulation/emulation experiments, ii) high speed packet traces, iii) available bandwidth and capacity measurement efforts, iv) active queue management policies, and v) high speed router design. To this end, we present a first principles structural model with a number of simple parameters that capture the communication characteristics of a broad range of applications. We demonstrate the gener-

We do not have an accurate way of validating our generated background traffic against existing traces because existing traces are typically taken at a single point in the network. Our underlying methodology and parameters might 16

Empirical CDF 1

0.9

0.9

0.8

0.8

0.7



Empirical CDF 1

0.6

0.5

0.4

0.6

0.5

0.4

0.3

0.3

0.2

0.2

0.1

0 2 10

3

10

4

10

5

10 Download latency in sec

6

7

10

0 0 10

8

10

No Background Traffic HTTP dominated P2P dominated MMStream dominated

0.1

Normal File Size File Size * 2 File Size * 5 10

1

10

2

3

10 10 Download latency of file in msec (log scale)

4

10

Figure 32: Comparison of CDFs of time to download 4KB files

Figure 31: CDF of response sizes in HTTP

with and without background traffic.

ality of our approach by validating our model against existing large-scale packet traces for three currently popular application classes, HTTP, P2P, and streaming media. We believe that our model will generalize to other existing application classes. Finally, we demonstrate the utility of our approach by showing how Swing generated background traffic can generate packet traces with a variety of characteristics and how it enables studies of router queue occupancy as a function of network topology and offered levels of background traffic.

[13] F. H. Campos, K. Jeffay, F. D. Smith, S. Marron, and A. Nobel. Methodology for developing empirical models of tcp-based applications (extended abstract). In SIGCOMM Internet Measurement Workshop, 2001. [14] J. Cao, W. Cleveland, Y. Gao, K. Jeffay, F. D. Smith, and M. Weigle. Stochastic models for generating synthetic http source traffic. In IEEE Infocomm, 2004. [15] K. chan Lan and J. Heidemann. Rapid model parameterization from traffic measurements. In ISI Technical Report ISI-TR-561, 2002. [16] Y.-C. Cheng, U. Hoelzle, N. Cardwell, S. Savage, , and G. M. Voelker. Monkey see, monkey do: A tool for tcp tracing and replaying. In USENIX Technical Conference, 2004.

References

[17] S. Floyd and V. Paxson. Difficulties in simulating the internet. In IEEE/ACM Transactions on Networking, 2001.

[1] Flowgen. http://www.cs.unc.edu/ jeffay/dirt/FAQ/flowgen.html. [2] The internet traffic archive, lawrence berkeley national laboratory http://ita.ee.lbl.gov.

[18] Y. Fu, L. Cherkasova, W. Tang, and A. Vahdat. EtE: Passive Endto-End Internet Service Performance Monitoring. In Proceedings of the USENIX Annual Technical Conference, June 2002.

[3] Iperf. http://dast.nlanr.net/Projects/Iperf . [4] Mawi working group http://tracer.csl.sony.jp/mawi/.

traffic

archive.

[19] J. Gast and P. Barford. Modeling congestion in backbone routers. In UW-Madison CS Techincal Report 1451, october 2002.

[5] The national laboratory for applied network research (nlanr ) http://www.nlanr.net.

[20] K. P. Gummadi, R. J. Dunn, S. Saroiu, S. D. Gribble, H. M. Levy, and J. Zahorjan. An analysis of internet content delivery systems. In Proc. of the 10th ACM Symposium on Operating Systems Principles (SOSP 2003)., 2003.

[6] Netperf. http://www.netperf.org/netperf/NetperfPage.html .

[7] The network simulator ns-2. http://www.isi.edu/nsnam/ns . [21] K. Harfoush, A. Bestavros, and J. Byers. Measuring bottleneck [8] Tg - traffic generator. http://www.caip.rutgers.edu/ arni/linux/tg1.html. bandwidth of targeted path. In IEEE INFOCOM, 2003. [9] M. K. Aguilera, J. C. Mogul, J. L. Wiener, P. Reynolds, and [22] P. E. Heegaard. Gensyn - a java based generator of synthetic interA. Muthitacharoen. Performance Debugging for Distributed Sysnet traffic linking user behaviour models to real network protocols. tems of Black Boxes. In Proceedings of the ACM Symposium on In ITC Specialist Seminar on IP Traffic Measurement, Modeling Operating Systems Principles (SOSP), October 2003. and Management, Sep 18-20, 2000. [10] P. Barford and M. Crovella. Generating representative web workloads for network and server performance evaluation. In Measurement and Modeling of Computer Systems, pages 151–160, 1998.

[23] F. Hernndez-Campos, A. B. Nobel, F. D. Smith, and K. Jeffay. Statistical clustering of internet communication patterns. In 35th Symposium on the Interface of Computing Science and Statistics, July 2003.

[11] L. Bertolotti and M. C. Calzarossa. Models of mail server workloads. In Performance Evaluation - An International Journal, 46(2-3):65-76, 2001.

[24] Y. hua Chu, S. G. Rao, S. Seshan, and H. Zhang. Enabling Conferencing Applications on the Internet using an Overlay Multicast Architecture. In Proceedings of ACM SIGCOMM, August 2001.

[12] CAIDA. http://www.caida.org .

17

Empirical CDF 1

0.9

0.9

0.8

0.8

0.7


Cumulative fraction of flows

Empirical CDF 1

0.6

0.5

0.4

0.6

0.5

0.4

0.3

0.3

0.2

0.2

0.1

0.1

Inet06 Inet04 Inet09

Dumbbell30 Dumbbell90 0

0

0.1

0.2

0.3

0.4 0.5 0.6 End−to−End Latency in sec

0.7

0.8

0 0 10

0.9

1

10

2

3

4

10 10 10 File download latencies in msec (log scale)

5

10

6

10

Figure 33: Comparison of router queue occupancy as a func-

Figure 34: Comparison of file download latencies as a function

tion of the RTT of host/destination pairs in the dumbbell topology

of the underlying topology

[36] B. A. Mah. An empirical model of HTTP network traffic. In INFOCOM (2), pages 592–600, 1997.

[25] G. Iaaccone, C. Diot, I. Graham, and N. McKeown. Monitoring very high speed links. In ACM Internet Measurement Workshop, 2001.

[37] A. Medina, N. Taft, K. Salamatian, S. Bhattacharyya, and C. Diot. Traffic Matrix Estimation: Existing Techniques and New Directions. In Proceedings of ACM SIGCOMM, August 2002.

[26] M. Jain and C. Dovrolis. End-to-end available bandwidth: Measurement methodology, dynamics, and relation with tcp throughput. In Proceedings of ACM SIGCOMM, 2002.

[38] A. Mena, J. H. Mena, and J. Heideman. Empirical study of real audio traffic. In IEEE Infocomm, Tel-Aviv, Israel, pages p. 101– 110, March, 2000.

[27] C. Jin, Q. Chen, and S. Jamin. Inet: Internet topology generator. 2000.

[39] E. M. Nahum, M.-C. Rosu, S. Seshan, and J. Almeida. The effects of wide-area conditions on WWW server performance. In SIGMETRICS/Performance, pages 257–267, 2001.

[28] P. Kamath, K. chan Lan, J. Heidemann, J. Bannister, and J. Touch. Generation of high bandwidth network traffic traces. In Proceedings of the International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems, pages 401–410, Fort Worth, Texas, USA, October 2002. USC/Information Sciences Institute, IEEE.

[40] R. Pan, B. Prabhakar, K. Psounis, and D. Wischik. Shrink: A method for scalable performance prediction and efficient network simulation. In IEEE INFOCOM 2003. [41] V. Paxson. Empirically derived analytic models of wide-area TCP coections. IEEE/ACM Transactions on Networking, 2(4):316–336, 1994.

[29] E. Kohler, J. Li, V. Paxson, and S. Shenker. Observed structure of addresses in ip traffic. In ACM SIGCOMM Workshop on Internet Measurement, Marseille, France, 2002.

[42] K. Psounis, R. Pan, B. Prabhakar, and D. Wischik. The scaling hypothesis: Simplifying the prediction of network performance using scaled-down simulations. In ACM HOTNETS, October 2002.

[30] D. Kosti´c, A. Rodriguez, J. Albrecht, and A. Vahdat. Bullet: High Bandwidth Data Dissemination Using and Overlay Mesh. In Proceedings of the 19th ACM Symposium on Operating System Principles, October 2003.

[43] V. Ribeiro, R. Riedi, R. Baraniuk, J. Navratil, and L. Cottrell. pathChirp: Efficient Available Bandwidth Estimation for Network Paths. In Passive and Active Measurement Workshop, 2003.

[31] B. Krishnamurthy, C. Wills, and Y. Zhang. On the use and performance of content distribution networks, Nov. 2001.

[44] S. Saroiu, K. Gummadi, R. Du, S. Gribble, and H. Levy. An analysis of internet content delivery systems. In Proc. of the Fifth Symposium on Operating Systems Design and Implementation (OSDI 2002)., 2002.

[32] L. Le, J. Aikat, K. Jeffay, and F. D. Smith. The Effects of Active Queue Management on Web Performance. In Proceedings of ACM SIGCOMM, August 2003. [33] B. O. Lee, V. S. Frost, and R. Jonkman. Netspec 3.0 source models for telnet, ftp , voice, video and WWW traffic. In Technical Report ITTC-TR-10980-19, University of Kansas, January,1997. [34] M. T. Lucas, B. J. Dempsey, D. E. Wrege, and A. C. Weaver. (m,p,s) - an efficient background traffic model for wide-area network simulation. [35] M. T. Lucas, D. E. Wrege, B. J. Dempsey, and A. C. Weaver. Statistical characterization of wide-area IP traffic. In Proceedings of Sixth International Conference on Computer Communi cations and Networks (IC3N’97), 1997.

[45] S. Saroiu, P. K. Gummadi, and S. D. Gribble. A measurement study of peer-to-peer file sharing systems. In Proceedings of Multimedia Computing and Networking 2002 (MMCN ’02), San Jose, CA, USA, January 2002. [46] S. Sen and J. Wang. Analyzing peer-to-peer traffic across large networks. In Proceedings of the second ACM SIGCOMM Workshop on Internet measurment workshop, 2002. [47] F. D. Smith, F. Hernandez-Campos, K. Jeffay, and D. Ott. What TCP/IP protocol headers can tell us about the web. In SIGMETRICS/Performance, pages 245–256, 2001.

18

Comparison for file file4 1

0.9

0.8

Cumulative fraction of flows

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0 800

Background traffic No Background Traffic Router Changes 1000

1200

1400

1600

1800 2000 Latency in ms

2200

2400

2600

2800

Figure 35: Comparison of Webserver file fetch latencies with background traffic and using router queue policy for file size 984 KB

[48] W. Tang, Y. Fu, L. Cherkasova, and A. Vahdat. Medisyn: a synthetic streaming media service workload generator. In Proceedings of the 13th international workshop on Network and operating systems support for digital audio and video, 2003. [49] A. Vahdat, K. Yocum, K. Walsh, P. Mahadevan, D. Kostic, J. Chase, and D. Becker. Scalability and accuracy in a large-scale network emulator. In Proceedings of the Fifth Symposium on Operating System Design and Implementation (OSDI), December 2002. [50] B. White, J. Lepreau, L. Stoller, R. Ricci, S. Guruprasad, M. Newbold, M. Hibler, C. Barb, and A. Joglekar. An Integrated Experimental Environment for Distributed Systems and Networks. In Proceedings of the 5th Symposium on Operating Systems Design and Implementation (OSDI), December 2002. [51] W. Willinger, V. Paxson, and M. S. Taqqu. Self-similarity and Heavy Tails: Structural Modeling of Network Traffic. 1998. [52] Y. Zhang, N. Du, e Paxson, and S. Shenker. the constancy of internet path properties. In ACM SIGCOMM Internet Measurement Workshop, 2001. [53] Y. Zhang, V. Paxson, and S. Shenker. The stationarity of internet path properties: Routing, loss, and throughput. ACIRI Technical Report, 2000.

19