Improving the communication performance of ...

The Journal of Systems and Software 97 (2014) 178–191

Contents lists available at ScienceDirect

The Journal of Systems and Software journal homepage: www.elsevier.com/locate/jss

Improving the communication performance of distributed animation rendering using BitTorrent file system夽 Ekasit Kijsipongse a,1 , Namfon Assawamekin b,∗ a b

National Electronics and Computer Technology Center, Pathumthani 12120, Thailand University of the Thai Chamber of Commerce, Bangkok 10400, Thailand

a r t i c l e

i n f o

Article history: Received 31 October 2013 Received in revised form 19 June 2014 Accepted 21 July 2014 Available online 28 July 2014 Keywords: Animation rendering Distributed file system Peer-to-peer

a b s t r a c t Rendering is a crucial process in the production of computer generated animation movies. It executes a computer program to transform 3D models into series of still images, which will eventually be sequenced into a movie. Due to the size and complexity of 3D models, rendering process becomes a tedious, timeconsuming and unproductive task on a single machine. Accordingly, animation rendering is commonly carried out in a distributed computing environment where numerous computers execute in parallel to speedup the rendering process. In accordance with distribution of computing, data dissemination to all computers also needs certain mechanisms which allow large 3D models to be efficiently moved to those distributed computers to ensure the reduction of time and cost in animation production. This paper presents and evaluates BitTorrent file system (BTFS) for improving the communication performance of distributed animation rendering. The BTFS provides an efficient, secure and transparent distributed file system which decouples the applications from complicated communication mechanism. By having data disseminated in a peer-to-peer manner and using local cache, rendering time can be reduced. Its performance comparison with a production-grade 3D animation favorably shows that the BTFS outperforms traditional distributed file systems by more than 3 times in our test configuration. © 2014 Elsevier Inc. All rights reserved.

1. Introduction Animation rendering is a process that transforms 3D models into hundred thousands of image frames to be composed into a movie. Due to the size and complexity of 3D models, rendering process is very computing intensive and time consuming. Rendering a single frame of an industrial-level animation can even take several hours on a commodity machine. Accordingly, to speedup the animation production, the rendering process is typically distributed to a set of machines in a network where each frame is rendered independently on a different machine to reduce the overall rendering time. There have been different distributed rendering strategies existed in the present day ranging from render farms to volunteerbased rendering. A render farm consists of a cluster of computers

夽 This paper is an extended version of our previous conference publication Assawamekin and Kijsipongse (2013). The extensions include the implementation details, security, load-balancing and fault-tolerance mechanisms as well as various additional experimental results. ∗ Corresponding author. Tel.: +66 02697 6506; fax: +66 02277 7007. E-mail addresses: [email protected] (E. Kijsipongse), namfon [email protected], [email protected] (N. Assawamekin). 1 Tel.: +66 02564 6900; fax: +66 02564 6901. http://dx.doi.org/10.1016/j.jss.2014.07.050 0164-1212/© 2014 Elsevier Inc. All rights reserved.

connecting to a locally high-speed network, like Ethernet switch, and it is exclusively utilized for the rendering process. Render farms are only found in large companies where users are able to afford the procurement and operation cost of the systems. In contrast, distributed rendering can happen on a set of non-dedicated machines such as the animator’s workstations in the company. He works on his machine to create 3D models during the day time and when he leaves he can turn it into a rendering machine at night time. On the other end of the spectrum, community/volunteer-based rendering (Distributed Rendering) allows the owners of personal computers to donate the idle time of their computers for rendering. People participating in the volunteer-based rendering could be friends, friends of friends, or even any trusted persons who are willing to share. By volunteering, it means they can cease the participation to help rendering at any time. For example, Renderfarm.fi, a largescale volunteer-based rendering service, distributes rendering tasks across personal computers over the public Internet. Our work is intended for volunteer-based distributed rendering which a large number of volunteer computers are located in wide area network. It is possible that some of them are co-located in the same network vicinity since they are owned by classmates, for example. Generally, volunteer-based distributed rendering consists of a few central servers and a number of volunteer computers

E. Kijsipongse, N. Assawamekin / The Journal of Systems and Software 97 (2014) 178–191

called clients (Volunteer Computing). The central servers are used to store and distribute data to all clients. There is no notion of data exchange between clients. To proceed with the volunteer-based rendering, users submit a job to the servers which there exists a scheduler to dispatch the job to many clients. Each job is associated with a rendering program, necessary arguments and the 3D models as well as the number of frames to render. These 3D model files have to be transferred to the clients before rendering process can start on the clients. However, the transfer time is not trivial due to the latency of the public Internet which delays the rendering process. Besides, if there are too many clients requesting for the data, the central servers can become overloaded, amplifying the problem. In fact, a rendering job that is dispatched to run on different clients requires the same input data. So, when a client needs the input data for the job, the data may already have been existed on other clients in whole or in part. Since the same data are used by several clients at almost the same time, it is a great opportunity to coordinate the data transfer among clients in the peer-to-peer (P2P) manner. We apply the BitTorrent (Cohen, 2003) P2P file sharing protocol as the means to disseminate the rendering data to all clients. A client who has already downloaded a whole or parts of the file from the central servers can directly share the file to other clients. As a result, fewer requests are sent to the central servers. Thus, it reduces the data transfer time and load on the central servers can be alleviated. To let any rendering applications access and exchange data in accordance with the BitTorrent protocol without having the applications modified, the P2P file sharing service must be made transparent. Transferring the files from/to several peers across the network must be invisible to the applications and the files could be treated the same way as usual files on local disks. As a result, it is necessary to implement the P2P file sharing service at the file system layer of an operating system which will isolate the applications from the complication of communication mechanism. In our previous work, we introduced BitTorrent file system (BTFS) (Assawamekin and Kijsipongse, 2013) as an efficient, transparent and distributed file system based on the BitTorrent protocol. The BTFS was successfully applied to improve the communication performance of the volunteer-based distributed rendering. In this paper, we extend our earlier work in many important aspects. Firstly, we address the security concern. As opposed to the typical P2P file sharing applications which can exchange music and video to any anonymous users, BTFS requires that the data have to be exposed to only authorized users. The access control mechanism in BTFS prevents clients (peers) to access the rendering data that they do not have permission. Furthermore, to keep unintended data disclosure minimal, data are always stored and transferred in encrypted format. Decryption keys are handed over to clients in encrypted channels. Secondly, we take into account the performance and fault-tolerance concern. Data are partitioned and replicated to multiple servers so that they help increasing the performance of BTFS when there are not enough peers in sharing data. Similarly, fault-tolerance in BTFS is guaranteed by duplicating data and executable components of BTFS on different servers. Lastly, we carry out extensive performance evaluation of BTFS and present its expanded results. The rest of this paper is organized as follows. Section 2 presents background knowledge and some existing research related to our work. The BTFS architecture and all modules constituting in this architecture are elaborately described in Section 3. In Section 4, we describe the experiments of using BTFS in distributed rendering as well as the evaluation results to show the advantages of BTFS in a practical use. Finally, we conclude the paper with the discussion of our contributions and ongoing work in Section 5.

179

2. Background and related work As part of our attempt in improving the communication performance of distributed animation rendering using BitTorrent file system, we have investigated some existing approaches and tools to resolve this problem. The aim of this section is to give some background knowledge and review the previous research studies relating to our approach. We point out their common limitations in which the techniques presented in Section 3 are proposed to overcome. 2.1. Distributed file system Distributed file system (DFS) is a technique of storing and accessing files from remote storages. Typically, DFS uses one or more servers to store files. Any number of clients can read or update a file on the server in the same ways as if it is stored locally on the client machines. DFS can provide larger storage space than client own storages. DFS beneficially makes multiple clients sharing a file easier. The DFS may provide redundancy by replicating files to many locations to protect users from data loss in the case of failure. DFS usually organizes files into a hierarchical structure (directories) following the common convention in local files. We give a brief explanation of some well-known DFSs as follows. Network File System (NFS) (Sandberg, 1985) is one of the first generation DFSs mostly used in Unix environment. It is implemented on a simple client/server model which allows users on client computers to transparently access remote files shared from a centralized server. NFS is not easy to scale to a large number of clients and the server poses a single point of failure. Likewise, Server Message Block/Common Internet File System (SMB/CIFS) is another popular DFS based on the client/server model for sharing files across both Unix and Windows platforms over Intranet and Internet. Similar to NFS, SMB/CIFS has the problems of scalability and single point of failure. In the next generation DFSs, such as Andrew File System (AFS) (Howard et al., 1988) and Coda, there are a set of servers to store files and they address the problem of location-transparency for all clients to see a globally unique file namespace. Both AFS and Coda also make use of persistent cache on clients to speedup the access time for both file and directory data. Server replication is used to protect from data loss. More advanced DFSs decouple data and metadata storages. The metadata provides information where the data are stored. The metadata is managed by single or a few metadata servers; while data is handled by a larger set of data servers. Typically, each client performs small operations to metadata servers to locate the data servers for a file. Then, a large portion of data communication happens directly between the client and the data servers. This design allows DFSs to scale better to large file transfers. Modern DFSs are designed with this principle. For example, Hadoop distributed file system (HDFS) (Shvachko et al., 2010) offers fault-tolerance and support storing very large files. Files are partitioned into blocks, each of which is stored on a different data server. HDFS replicates file blocks in many data servers for redundancy. The metadata server in HDFS, known as NameNode, tracks the location of all blocks. HDFS may consist of hundreds or thousands of data servers. When one of these machines crashes, it can automatically recover from failure. However, the NameNode of HDFS presents the single point of failure. Another major drawback is that HDFS files are immutable, i.e. once files are created they cannot be changed later. MooseFS has a similar design as HDFS. Each file is divided into chunks which are stored (and possibly replicated) on different servers. MooseFS is transparent and adopts the POSIX file system semantics. Unfortunately, there is only single metadata server in HDFS and MooseFS, so the metadata server inevitably presents a bottleneck and a single point of failure.

180


To resolve a single point of failure, MogileFS, an open source DFS which emphasizes on data archiving and deployment on commodity hardware, spreads data and metadata over different servers. The metadata servers must be configured as a high availability cluster. Each file is replicated depending on its replication level defined by applications. However, MogileFS is not designed to be transparent to client applications. It provides the application-level APIs to access files and so it requires application modification. MogileFS does neither support the POSIX file system semantics nor hierarchical namespace. GlusterFS, another highly fault-tolerant DFS having POSIX compliance, completely eliminates the use of metadata servers by applying hashing algorithm to track the location of the files in the data servers. Its design allows the performance to linearly scale with the number of data servers. GlusterFS requires that all data servers must operate in trusted network environment. Moreover, the access control is enforced at the client side which is easy to circumvent. 2.2. BitTorrent file sharing protocol BitTorrent (Cohen, 2003) is one of the most popular Peer-toPeer (P2P) file sharing protocol to distribute and share media files (music, videos, games and e-books) over the Internet. When the users want to download a file, they use BitTorrent software to locate other computers having the same file and begin downloading the file from several computers, known as peers, in parallel. The users who download files can also upload the files to others when requested. BitTorrent protocol operates on three main components, i.e. .torrent, tracker and peer, each of which is described below. 2.2.1. .torrent A user who wants to disseminate a file creates a small file called .torrent that acts as the key to initiate the sharing of the file. The .torrent does not contain the content of the file; but it rather contains metadata about the file such as its length, the hashing information for verifying integrity and the URL of the tracker(s). When another user wants to download the file, he/she firstly obtains the corresponding .torrent file, and then opens that .torrent file in a BitTorrent software to start exchanging file with peers. Since, the BitTorrent protocol does not specify its own facilities to deposit and search for the .torrent files, users are required to distribute the .torrent files by other conventional means (like web blog, email, news feed, etc.). 2.2.2. Tracker Trackers in the BitTorrent protocol play an important role in peer communication. It helps peers locate one another by keeping track of peers that are interested in the same file. Peers find out the tracker URL from the .torrent file, and then communicate with the tracker to get a list of peers who are participating in that file sharing. This list may not contain all the possible peers, but only some randomly chosen peers in order to even the load on all peers. Once the list of peers is obtained, the file exchange (download/upload) process begins. The tracker neither directly involves in any data transfer nor have a copy of the file. Peers must report own statistics information to the tracker periodically and receive updated information about new peers to which they can connect, or about peers which have left in return. BitTorrent trackers are commonly implemented as HTTP/HTTPS servers. 2.2.3. Peer A peer is a computer running an instance of BitTorrent software to exchange files with other peers. Each file is exchanged in pieces such that a peer is not necessary to have all pieces of the file; only some pieces. A peer reports its IP address and port to the

tracker for peer connections. Each peer polls the tracker for information about other peers to exchange pieces. Each piece can be downloaded concurrently from different peers. Peers continuously exchange pieces with other peers in the network until they have obtained every file piece needed to reassemble the original file. A peer who has the complete file is called a seeder. BitTorrent protocol is primarily designed to reduce the download time for large and popular files. For example, BitTorrent protocol gives an incentive for peers to share, i.e. peers that upload more should be able to download more. In addition, the order in which pieces are selected for download from other peers is based on the rarest pieces first algorithm so that it rapidly increases the number of replicas of those rare pieces. Several research studies (Cohen, 2003; Qiu and Srikant, 2004; Bharambe et al., 2006) have shown that the BitTorrent protocol can handle large distributions and scales well as the number of peers increases. Other P2P file sharing systems have an architectural design rather different than BitTorrent explained above. For example, Napster, the first commercial P2P file sharing system, makes use of the centralized servers to index and search for the files its clients have. However, the centralized servers limit its scalability and pose a single point of failure. Gnutella (Klingberg and Manfredi, 2013) is a fully decentralized P2P file sharing system. It has no centralized servers to index the files. The search for files is realized via message broadcasting which makes it suffer from the exponential growth of the network traffic. KaZaA is built on the hybrid architecture between Napster and Gnutella. KaZaA has no centralized servers; but it dynamically promotes some peers to be superpeers. Each superpeer becomes an index server for a number of ordinary peers that connect to it. An ordinary peer sends a request for files to its superpeer, which in turn communicates with other superpeers using Gnutella-like broadcasting to find the files. KaZaA scales better and offers more reliable than Napster and Gnutella. However, this approach still consumes resources and bandwidth of superpeers.

2.3. Related work This section gives a survey comparing relevant research work that attempts to utilize BitTorrent protocol for improving the performance of data transfer in applications other than music and video file sharing. Many of them have actively been carried out in the Grid and volunteer-based computing domains. Kaplan et al. (2007) propose GridTorrent framework to efficiently distribute data for scientific applications in Grid computing environment. The framework consists of many components inspired by BitTorrent protocol to handle task dispatching and file sharing. Another independent work from Zissimos et al. (2007), having the same name, GridTorrent integrate BitTorrent with Globus Grid middleware to provide a P2P file transfer mechanism which outperforms the GridFTP protocol for large files. They eliminate the use of “.torrent” metadata files with necessary information from the Replica Location Service (RLS). The RLS also functions like a BitTorrent tracker to find the locations of file replicas. Costa et al. (2008) apply BitTorrent to optimize data distribution in BOINC (Anderson, 2004), the middleware for volunteering computing. They show that BitTorrent can reduce the network load at servers significantly while having minimal impact to the computing time at clients. Wei et al. (2005, 2007) implement BitTorrent in the computational Desktop Grid platforms like XtremWeb (Cappello et al., 2005) to collaboratively distribute data among users in solving scientific problems. They point out that even though BitTorrent protocol has more overhead than typical file transfer protocols, it can outperform when distributing large files to a high number of nodes. BitDew (Fedak et al., 2008), a data management framework


181

Table 1 Comparison of BitTorrent-based data dissemination in distributed computing environments. Work

Application transparency

Name space

Access control

Torrent indexing and tracking

Application and testing platform

Kaplan et al. (2007)

No

Hierarchy

Unix-like file permission

Zissimos et al. (2007)

No

GridTorrent URL

Grid security infrastructure

Costa et al. (2008)

No

Hierarchy

Not specified

A synthetic application on 3 nodes A synthetic application on 18 PlanetLab nodes A synthetic application on 312 nodes from Grid5000 testbed

Wei et al. (2005, 2007)

No

Flat file name (UUID)

Not specified

Fedak et al. (2008)

No

Hierarchy

Not specified

Our approach

Yes

Hierarchy

Access control list for create/read/write/delete

No torrent indexing and use a variant of BitTorrent tracker Use Globus Replica Location Service (RLS) Catalog service for torrent indexing and a native BitTorrent tracker Catalog service for torrent indexing and a native BitTorrent tracker Catalog service for torrent indexing and a native BitTorrent tracker Catalog service for torrent indexing and a native BitTorrent tracker

for Desktop Grid, also relies on BitTorrent in its data distribution service. Although the aforementioned works have similar objectives to ours, they remain distinct. Table 1 highlights these differences. Firstly, all of them do not implement the BitTorrent-based data distribution mechanism in the file system layer. So, application transparency is not accomplished. It is required to modify the applications with the given APIs. Secondly, the namespace, which refers to how users logically identify their data, varies. Some works (Wei et al., 2005, 2007) employ flat namespace. The hierarchical namespace as adopted in (Kaplan et al., 2007; Costa et al., 2008; Fedak et al., 2008) provides more semantics to users than flat namespace since the organization of files in terms of directory hierarchies are naturally recognized for most users. URLs are also used (Zissimos et al., 2007). Thirdly, different access control mechanism is implemented. For example, Kaplan et al. (2007) support the Unix-like file permission, i.e. public, group and user level access. Zissimos et al. (2007) rely on Grid security infrastructure. Others do not specify. Next, torrent indexing and Tracking describes how a client discovers the .torrent metadata and peer information, respectively. Most previous works (Costa et al., 2008; Wei et al., 2005, 2007; Fedak et al., 2008) implement own catalog service to store .torrent and use native BitTorrent trackers. Kaplan et al. (2007) generate .torrent from task descriptions and use a variant of BitTorrent tracker that includes additional features like access control. Zissimos et al. (2007) utilize the Replica Location Service (RLS) in Globus for both torrent indexing and tracking services. Lastly, their evaluations were carried out with scientific or even synthetic applications. No one explores the advantages of using BitTorrent in distributed animation rendering. For volunteer-based distributed animation rendering such as Renderfarm.fi, BURP and vSwarm, the rendering program and its input files are downloaded from the centralized servers or from a set of mirror servers. According to the current information, the BitTorrent protocol has not been implemented to speed up the data distribution to clients yet. 3. Design and implementation We design BitTorrent file system (BTFS) to function at file system layer in the Linux operating system to provide applications the scaleable, fault-tolerant and distributed file system with transparent P2P data dissemination and persistent local cache. The BTFS is intended to improve the performance of data communication in distributed animation rendering where the same dataset is repeatedly used over multiple remote machines. We assume the BTFS to

A synthetic application on 64 nodes A bioinformatics application on over 400 nodes from Grid5000 testbed Animation rendering on 8 nodes from 5 distributed sites

work with an external job scheduler which is capable to dispatch jobs and manage the membership of remote machines. The BTFS consists of 4 main components: metadata server, seeder, tracker and BTFS client, as shown in Fig. 1. The operations of these components are further described below. 3.1. Metadata server The metadata server provides information about files and directories such as attributes (e.g. file size or last modified time), as well as the torrent information of the file. It does not store the content of the files. The BTFS client interacts with the metadata server to get/set the file metadata. Since files in BTFS are organized into a hierarchical namespace which files are located under a particular directory, the metadata server is also responsible for such directory listing. The torrent information associated with each file include filename, file size, hash information, number of pieces, seeder and tracker URLs like other .torrent files used in BitTorrent file sharing software. The BTFS clients are allowed to access the file metadata only if the permission of the requested file is granted. We use Apache ZooKeeper 3.4.3 to implement the metadata server. ZooKeeper is a distributed coordination service for distributed systems. It store data in a data structure called znode which can be organized into a hierarchy similar to directory tree. We create the/btfs and/config znodes at the top level of the hierarchy. The/btfs znode is used to hold the root directory of a BTFS namespace. This is where file and directory entries of all BTFS users are placed. They will appear as the normal files and directories under the mount point on the BTFS client machines. The/config znode is used to store the globally system configuration for BTFS clients. We manage some client configuration through the/config znode so that the configuration updates can be distributed to all clients easily. Each file and directory in the BTFS namespace is associated with a descendant of the/btfs znode. The BTFS file (or directory) attributes are stored in the corresponding znode data. The essential attributes include the Universally Unique IDentifier (UUID), last modification time, file size and encryption key. These attributes will partially be mapped into the appropriated fields in the POSIX stat structure on the BTFS client machine as described later. Each file has a different UUID which is used internally by BTFS components as a unique ID to refer to the particular file (similar to what inode number is). Fig. 2 shows the znode tree structure and the corresponding BTFS files and directories. The “.torrent” znode in the znode tree is created as the child of a user file. This znode is invisible to the users and stores the torrent information of the file to be shared in BitTorrent protocol.

182


Fig. 1. BitTorrent file system architecture.

3.2. Seeder

seeder is referenced by its UUID which is very unlikely to duplicate, there is no need to maintain directory structure of files in the seeder.

A seeder has responsibility like a central file server to store and serve the content of the files. Each file in BTFS must be uploaded into the seeder to ensure that the availability of the file is independent to the existence of BTFS clients. Files are always available on the seeder and the BTFS clients can retrieve the files from the seeder as a last resort. Under the normal operation, BTFS clients can download the files from both the seeder and other clients, whatever is best. Seeders should be located in the public network that any clients can reach. In a more complex deployment, there can be multiple (and possibly remote) seeders on which files are distributed such that the load on a single seeder is reduced. It is also a great advantage to replicate a file on multiple seeders as the file is eligible for parallel download and safe from single server failure. The BTFS clients manipulate (including upload and download) files on the seeder via the WEBDAV protocol. Since each file on the

3.3. Tracker According to the BitTorrent protocol, trackers are required to coordinate all BitTorrent peers to exchange files. This is also the case for BTFS clients which run the protocol. Any number of native BitTorrent trackers such as opentracker or public trackers can be used. 3.4. BTFS client BTFS client is a core component that glues other components together. It runs on client machines where rendering software execute a job dispatched from the job scheduler. The implementation of BTFS client is based on File System in Userspace (FUSE).

ZooKeeper Node

Directory

File

/

btfs

Mary

config Mary

John

tmp

/

model

John

tmp

model

a.blend

a.blend a.blend.torrent

(a) ZooKeeper tree structure

(b) BTFS file system

Fig. 2. Mapping from ZooKeeper data hierarchy to BTFS file system.

E. Kijsipongse, N. Assawamekin / The Journal of Systems and Software 97 (2014) 178–191 Table 2 Mapping from BTFS to POSIX attributes. BTFS attributes

POSIX attributes

Description

Size Mtime

st size st atime, st mtime, st ctime

UUID

–

IV Key

– –

File size in bytes Only the last modification time is maintained and it is copied to the last access and last status change for performance reason Universally unique identifier of each file Initialization vector Encryption/decryption key

BTFS clients intercept all file system calls such as open(), read() and write() from any applications which request access to the files in BTFS and act upon them accordingly. For reading a file, BTFS clients contact the metadata server for getting the attributes and torrent information of the file, and then create a BitTorrent client thread to download the file from seeders and peers. The downloaded file will locally be cached into the local storage of the client machines. The file handle of the cached file is passed to the requesting application for future operations. The cached file is kept as long as the cache space is available; otherwise, the cache replacement algorithm is invoked to clear unused files. We use the Least Recently Used (LRU) policy for cache replacement which will remove the least recently used file first. Subsequent file reading operations will be redirected to the cached file, if existed, to reduce the network traffic. The cached file is shared and exchanged with other BTFS clients by the BitTorrent protocol as well. Each BTFS client stores cache information in a small embedded database. When writing a file, BTFS clients perform the writing operation locally to the cached file first. After the written file has been closed, BTFS clients generate a new UUID and upload the file to the seeder with this UUID. Then, BTFS clients update the file metadata on the metadata server to complete the write operation. BTFS clients can work with multiple seeders or trackers to improve the efficiency and reliability as described later. Other basic file operations such as deleting a file, listing files, creating and removing a directory are also implemented which require the BTFS clients to interact with the metadata server only. 3.5. Mapping to POSIX semantics BTFS has the POSIX-like file system semantics. The attributes of files and directories which are stored in the corresponding znode data are mapped into the POSIX attributes on the BTFS clients if valid. Table 2 shows the mapping between BTFS and POSIX attributes including file size and modification time. The UUID, IV and key attributes are used internally by the BTFS components. Other POSIX attributes such as inode number and permission mode are not aligned with the BTFS so they are assigned with some default values in order to maintain the normal POSIX operations on client machines. 3.6. Consistency model To improve the performance of BTFS over the wide area network, we employ the weak consistency model which works sufficiently in distributed animation rendering where files are mostly

183

read, neither frequently nor simultaneously updated. The weak consistency model can greatly reduce the network latency; but with some trade-off in data consistency. Firstly, the attributes of the files obtained from the metadata server are maintained in memory cache at the BTFS clients until time-to-expire (60 s in the current implementation). So, reading a file is not guaranteed to get the latest version of the file due to the stale attributes in cache. Secondly, we deploy write back policy. The files are locally updated and then sent back to the metadata server and seeder after closed. It is possible for other BTFS clients to read the old version of the file if they access the file before the write back has completed. However, BTFS ensure that the file will never be corrupted since the updated file will always be assigned a new UUID. Both old and updated file content will coexist in the seeders until the next garbage collection which removes the old file from the seeders. Besides, when there are several BTFS clients trying to update the same file concurrently, they do not corrupt the integrity of the file; but last writer wins. When a file is updated or removed from the BTFS, the associated znode is also updated or removed, respectively. However, the files in the seeder remain intact for a period of time, not to disrupt any ongoing download. To reclaim the space on the seeder afterwards, there exists a process running periodically to perform garbage collection. The process collects all UUIDs from the metadata server and compares the UUIDs with all files in the seeders. For a file without reference from the metadata server and not being currently access, the process removes it from the seeder.

3.7. Security Since BTFS is intended to be used over a public network, it is necessary to guard against many security attacks such as unauthorized access and data corruption. The security mechanism of BTFS is explained as follows.

3.7.1. Authentication, authorization and access control BTFS needs all users to login before accessing the file metadata on the metadata server. The BTFS client sends the user credential (username and password) for authentication when establishing the connection with the metadata server. If the credential is valid, the user is authenticated. Currently, we develop a file-based authentication as a plug-in for ZooKeeper. The password file simply stores a list of valid username and password of all users. The group file maintains the group membership information of all users. In the future work, we plan to migrate the password and group information to an LDAP server for better management. To further authorize the user to access files or directories, BTFS relies on the ZooKeeper’s Access Control Lists (ACLs). The authenticated user will be inspected against the ACL to check if the access to the file metadata is granted. When a new file or directory is created, BTFS clients duplicate the ACL from parent directory by default. Another essential security setting is required on the seeder to allow the BTFS clients to only upload and download the files. Other WEBDAV operations like deleting a file and directory listing must strictly be prohibited from BTFS clients; otherwise, malicious users may create denial of service attacks by scanning and removing seed files from the seeder directly.

3.7.2. Data integrity The data integrity of the file is supported by the 20-byte SHA1 message digest (hash) of the file’s content available in the torrent information which the BTFS client obtains from the metadata server. Thus, according to the BitTorrent protocol, the BTFS client can always ensure the data integrity by validating the downloaded

184


file with the message digest. If it fails, the downloaded file could be corrupted or tampered and it should be downloaded again. 3.7.3. Confidentiality To protect the confidentiality of the data stored in the seeders and the data transmitted in the network during P2P file exchange, we encrypt the content of the file. The file is always stored as a cipher text in the seeders. When a BTFS client creates a file, it will randomly generate an initialization vector (IV) and a key for encrypting/decrypting the file, both of them are stored in the BTFS attributes at the metadata server. The content of the file is encrypted with the 128-bit AES CTR mode during write. This encrypted file is uploaded to the seeders and shared with other BTFS clients. For another BTFS client to read the file, it must know the matching IV and key which can only be obtained from the metadata server if it is authorized. The encrypted file is downloaded into the local cache on each BTFS client and decrypted on-the-fly while reading. The encryption/decryption is done in the FUSE file system layer and thus transparent to the applications. Similarly, the messages between BTFS clients and the metadata server which carry some sensitive information such as username and password as well as the key must not be sent in plaintext since they can easily be exposed to eavesdroppers. So, a secure connection must be established between BTFS clients and the metadata server. Unfortunately, the current ZooKeeper’s C client APIs do not support the secure SSL connections (only available in Java APIs; but we believe that it will be available in the future releases of ZooKeeper). A workaround that we employ for the moment is to wrap the connection between the metadata server and BTFS Clients with stunnel which can secure information and resist replay attacks. 3.8. Scalability and fault tolerance BTFS has been designed to tolerate from single point of failure. First, the metadata can be replicated to many servers by using ZooKeeper’s master/slave replication. At the present of some server failures, the operation can continue as long as the majority of the metadata servers (quorum) agree on the data consistency. Further on, BTFS clients can arbitrarily connect to one of any metadata servers for read, making the load evenly distributed to multiple metadata servers. In case of write, all the operations will be forwarded to the master server. Since in animation rendering, data are more read than written, file write will not cause too much load on the master server. For the seeders to scale, we can setup multiple seeders and let the BTFS client randomly choose one of them for uploading the file. So, the load on multiple seeders is likely to balance. If some seeders fail, they partially affect the BTFS in parts which the data are stored on those seeders, while the remaining parts are intact. Nevertheless, it should be emphasized that files on the failed seeders will not be lost if the files remain cached in online BTFS clients. We can further increase the level of fault-tolerance by replicating files on more than one seeder, in which case a BTFS client randomly selects to hold the replicas. Trackers are also scaleable. Each file can be associated with a different tracker to reduce the load on single tracker. It is also possible to associate an individual file with multiple trackers to alleviate the problem of tracker failure. The BitTorrent protocol specification extension suggests that the BTFS client will try to connect to any tracker that is still functioning randomly. However, it may happen that clients which share the same file may connect to a different tracker and see only the subset of peers. So, some of them cannot cooperate in file transfer for an optimal result. Regarding to this problem, our BTFS client is configurable to support a variation that contacts to all trackers at the expense of additional network traffic.

1 203.185.96.47 8080 /repository/default admin admin 203.185.96.48 8080 /repository/default admin admin http://203.185.96.47:6969/announce http://203.185.96.48:6969/announce Fig. 3. Global configuration file.

3.9. Global configuration management The global configuration of the BTFS is stored in the/config/.btfsrc.xml znode in the metadata server. Each client will read the content of this node during the initialization phrase. The configuration is written in an XML format which is exemplified in Fig. 3. The configuration is self-described, for instance, the sections define the information of seeders such as IP and port. The section defines the replication parameter which specifies the number of file replicas to store in different seeders. 3.10. Operation Since the BTFS is implemented as a file system in the user space, a normal user can mount it to a local directory tree where he/she has permission. The command to mount BTFS is as follows; where the first argument is the IP address of the metadata server and the second argument is the local mount point. $btfsmount 192.168.1.1 /home/user1/btfs 4. Evaluation and experiments This section describes the experimental setup and presents the results of micro and macro benchmarks to determine the overhead of BTFS read/write operations and the overall application performance when using BTFS in the volunteer-based distributed animation rendering, respectively. 4.1. Testbed system configuration We have carried out all experiments on a testbed system consisting of a set of servers located at a central site and multiple clients from different remote sites. The testbed represents the volunteerbased distributed animation rendering which users donate their desktop or notebook computers for rendering jobs in a specific project. The testbed system spans across 5 institutional and corporate sites, i.e. NECTEC, INET, CAT, UTCC and CSLOXINFO, as illustrated in Fig. 4. They all connect to the public Internet. NECTEC is chosen as the central site to place all the servers. We allocate 7 clients from the remaining sites. For hardware specification, each


185

Fig. 4. Testbed system.

client machine has 4 CPU cores, 4 GB RAM, and at least 50 GB free harddisk space. The CPU speed ranges between 2.2 and 2.4 GHz. All clients have a public IP address and connect to the Internet without any firewall. However, the original bandwidth between sites as measured by the ttcp network benchmarking tool is as high as 100 MB/s which is too fast when comparing to the current Internet speed of average home users. So, we set the bandwidth throttling for both egress and ingress traffic on all servers and clients to 10 Mb/s for being more realistic. Note that, we sometimes observe the network bandwidth between some particular sites as low as 2 MB/s during the experiments but it does not persist and occurs rarely just from time-to-time. All machines are installed with Linux CentOS 6.2.

Fig. 5 shows the time of all steps in milliseconds for reading files of different sizes except that the download time is presented in seconds by the secondary Y-axis. Clearly, as the file becomes larger, the download time increases proportionally. The accumulative time of all other steps is considered as the overhead and it grows in accordance with file size. Most of the overhead time comes from the decryption step. However, the overhead time is less than a second and grows much slower than the download time. For the file larger than 1 M, this overhead is very trivial. Although, this operation breakdown differs by many factors such as bandwidth, the number of peers and CPU speed, it still gives some useful insight. For write operation, each BTFS client carries out the following steps respectively:

4.2. Micro-benchmark To understand the overhead cost incurred by BTFS, we measure the time spent in each step of the read and write operations. In this experiment, we use only one seeder and one BTFS client. We vary the file size to read and write from 0.5 MB to 20 MB to see how the overhead grows. For read operation, each BTFS client executes the following steps respectively: (1) Get metadata (2) Get torrent (3) Download file (4) Update cache DB (5) Read local (6) Decrypt

Get file metadata from metadata server Get torrent information of the file from metadata server Download the file from the seeder and store it in local disk Update cache information in local embedded database Read the file locally Decrypt the file using the key obtain from the file metadata

Fig. 5. Read operation breakdown.

186


120

Frequency

100 80 60 40 20 0 0 Fig. 6. Write operation breakdown.

10

20

30

40

50

60

70

File ID Fig. 7. Frequency of file access.

(1) Get metadata (2) Write local (3) Encrypt (4) Update cache DB (5) Create torrent (6) Upload file (7) Update metadata

Get the parent’s ACL from the metadata server Write the file in local disk Encrypt the file using a generated key Update cache information in local embedded database Calculate file hash and create torrent information Upload file to a seeder Update file attribute and put torrent information into the metadata server

Similarly, Fig. 6 shows the overhead and the upload time of a BTFS client when writing a file of different sizes. The overhead is measured in milliseconds, whereas the upload time is presented in seconds by the secondary Y-axis. In overall, the overhead in writing a file is larger than that in reading a file but less than a second in all cases. The time used in encryption is the largest part of the overhead. The upload, encryption, writing locally and creating torrent steps are clearly grows as the file becomes larger. However, the growth rate of the upload time is higher than that of the overhead which is also trivial when writing a file larger than 1 M. 4.3. Macro-benchmark This section presents the performance evaluation of BTFS on the testbed using a distributed animation rendering application with real dataset. The characteristics of the benchmark dataset and experimental results are as follows. 4.3.1. Render data and software The data used in the experiments are from the Big Buck Bunny project which is the open animation movie initiated by the Blender software development team. The project has publicly released all 3D models, image and texture files that have been used during their production. There are over 400 files in 1.2 GB for the entire dataset. These files are organized into 13 scenes separated by the top level folders under the project directory. Each scene may further be broken into sub-scenes, each of which is composed of a main .blend file and references to many other files (including texture images and .blend files) in the project directory. Fig. 7 depicts the number of references (frequency) to each .blend file, sorted from the most frequent to the least, for all scenes. It is clear that many of the .blend files are commonly accessed as shown by high frequency. Therefore, it would be beneficial for the BTFS to share and cache these files for distributed rendering. The rendering process finally produces a number of images. Since different images have distinct computational requirements, some images can finish rendering in a few minutes and use only small memory; while others may take an hour with much larger memory. The final animation is 10 min long consisting of more than 15,000 image frames.

To test BTFS with different workload, we select scenes having distinct computational requirements to represent small, medium and large jobs as shown in Table 3. The input size is the total size of the main .blend file and all referenced files required for rendering. The average rendering time is measured when all files are stored in the local disk. Note that several scenes in the Big Buck Bunny require larger memory than what we have (4 GB) in the testbed, in which case they cause the rendering to fail. These too large scenes, such as 01 intro/01.blend, are initially precluded from our consideration. We deploy DrQueue 0.63.4, the job scheduler for distributed render farm, to dispatch jobs to client machines. DrQueue can detect client availability, support fault-tolerance by job reexecution, and has a superb interface for end users. Basically, to submit a job, users use DrQueue GUI to create a job description providing the path of the .blend file, the number of start frame and the end frame as well as other render parameters. Blender 2.49, the open source rendering software, is installed on all clients. For a client to execute a job, it is required that the input files (i.e. the .blend and all referenced files) of the job are always accessible from the client. Thus, an instance of DFS is used to allow the clients to transparently access all the required files from the network. 4.3.2. Performance comparison of BTFS and SMB file system We compare the performance of BTFS with those of SMB, one of the most widely used DFS. We setup a Samba version 3.5.10, an open source SMB server software, at the NECTEC site as a repository to hold the entire project dataset. For a client machine to access files from Samba, there are two possible SMB client software: FuseSMB and Linux CIFS. Both are different SMB client implementation. The former is a user-space SMB client to mount the SMB file system based on FUSE. The latter is the kernel-based SMB client that requires root permission to mount. We use a single BTFS seeder for this experiment. Jobs of the selected scenes are submitted to the testbed with 25% full HD resolution (480 × 270 pixels). Then, we measure the total rendering time and find the average of 3 runs. The total rendering time varies for each file system used. For small job, it spends approximately 1260, 1621 and 5040 s to finish under BTFS, Linux CIFS and Fuse-SMB, respectively in an ascending order as depicted in Fig. 8. Clearly, BTFS gives the best time; whereas Fuse-SMB is the worst. To elaborate how BTFS could help improve the performance, we plot the amount of data transfer from the server (seeder, in case of BTFS) over time from one such run as illustrated in Fig. 9. It shows that Fuse-SMB performs the worst since the rendering application requires data transfer mostly all the time, making the rendering process delayed. Linux CIFS is better than Fuse-SMB as the data transfer in Linux CIFS happens only at the early period of time (around the first 1000 s) and then reduces


187

Table 3 Characteristics of the testing dataset. Job Size

Scene

No. of frames

Small

12 peach/03.blend

28

650

1:30

90

Medium

01 intro/02.blend

93

2500

4:50

40

Large

02 rabbit/02.blend

91

3500

9:06

290

gradually until job finishes. This is due to the internal differences of their SMB implementation. Most of the network traffic reduction in the Linux CIFS comes from using memory pagecache which automatically caches files in free memory. The BTFS has the least data transfer from the server because of two main reasons. First, BTFS clients can share data among other peers making the data transfer happen only at the first 500 s. The other reason is that BTFS can effectively utilize the local disk cache. For medium job, it spends approximately 5837, 5740 and 7202 s to finish under BTFS, Linux CIFS and Fuse-SMB, respectively as depicted in Fig. 8. Similar to the small job, the Fuse-SMB performs the worst in terms of the rendering time and the amount of network traffic from the server as shown in Fig. 10. Its network traffic stays high all the time. Both BTFS and Linux CIFS transfer data from the server only at the beginning of time. While the BTFS has similar rendering time, it has half amount of traffic than Linux CIFS. Next, for the large job, the rendering process takes approximately 9248, 29,886 and 32,717 s under BTFS, Linux CIFS and Fuse-SMB, respectively as depicted in Fig. 8. The performance of Fuse-SMB is the worst due to the same reasons as mentioned for the small and medium jobs. It is observed that BTFS can reduce the

35,000

Rendering Time (s)

30,000 25,000 BTFS

20,000

Mem. (MB)

Avg. rendering time (min. per frame)

Input size (MB)

rendering time from that of Fuse-SMB more than 3 times. Interestingly, the Linux CIFS now spends rendering time close to the Fuse-SMB. Fig. 11 shows the traffic load of the server under different file system. It reveals that Linux CIFS has continuous network activity throughout the time. This is owing to the fact that the large job requires entire memory of the machine in rendering, so there is no free memory left for doing pagecache. As a result, Linux CIFS needs to reload files from the server for every image frame. On the contrary, BTFS stores files in persistent storage (disk) which is independent to pagecache. Thus, the rendering time and the traffic load from the server are greatly reduced by the BTFS. 4.3.3. Load balance in BTFS with multiple seeders In this experiment, we create multiple seeders for the purpose of load sharing. There are 4 seeders, each of which is capable of sending data at 1 kB/s. Data replication is disable such that a file is uploaded into a random seeder. The entire Big Buck Bunny project is copied into the BTFS. Then, we run 7 remote clients that continuously read a random file from the project. The P2P data exchange is turned off to make clients download data from the seeders only. Fig. 12 depicts the load of each seeder over time. The load varies and is spread across all seeders. However, load is not perfectly balanced as the load on seeder 1 and 3 is higher than others resulting in the actual aggregate bandwidth below 4 kB/s. From our observation, each seeder holds roughly 100 files but seeder 1 and 3 hold larger data as shown in Table 4. Ideally, the load can be more equal if the distribution of files involves file size rather than relying only on the randomness, which remains further investigation.

CIFS

15,000

Fuse SMB

10,000 5,000 0 Small

Medium

Large

Job Size Fig. 8. Rendering time under different file system.

4.3.4. BTFS replication performance In this experiment, we replicate files into multiple seeders and measure the time for a single BTFS client to retrieve the files from different number of seeders. With replication, the BTFS client can download different parts of the file in parallel from multiple seeders. We vary the number of seeders from 1 to 4. Fig. 13 shows the transfer time (not including the rendering time) for large job, i.e. 02 rabbit/02.blend and all 58 referenced files, from using multiple seeders. The transfer time reduces significantly when the

188


1,200

Data Transfer (KB/s)

1,000

800 BTFS Fuse-SM B

600

CIFS

400

200

0 0

500

1,000

1,500

2,000

2,500

3,000

3,500

4,000

4,500

Time (s) Fig. 9. Outbound network traffic from server (seeder) for small job.

1,200


1,000

800 BTFS Fuse-SM B

600

CIFS

400

200

0 0

1,000

2,000

3,000

4,000

5,000

6,000

7,000

8,000

Time (s) Fig. 10. Outbound network traffic from server (seeder) for medium job.

number of seeders increases (and thus having more replicas). The best performance is obtained when the number of seeders is 4. Fig. 14 demonstrates the transfer speedup over the number of seeders. Note that the transfer speedup is calculated as the ratio

of the transfer time using multiple seeders to the transfer time using single one. Although, the transfer speedup does not reach the ideal speedup due to some overhead incurred, it remains persuasive.

1,200


1,000

800 BTFS Fuse-SM B

600

CIFS

400

200

0 0

5,000

10,000

15,000

20,000

25,000

30,000

35,000

40,000

Time (s) Fig. 11. Outbound network traffic from server (seeder) for large job.


189

3,000


2,500

2,000 Seeder 4 Seeder 3

1,500

Seeder 2 Seeder 1

1,000

4,320

4,080

3,840

3,600

3,360

3,120

2,880

2,640

2,400

1,920

2,160

1,440

1,680

1,200

960

480

720

0

0

240

500

Time (s) Fig. 12. Accumulative load of multiple seeders.

350 300

Time (s)

250 200

Ideal BTFS

150 100 50 0 1

2

3

4

Number of Seeders Fig. 13. Data transfer time of multiple replicas.

4.5 4.0 3.5

Speedup

3.0 2.5

Ideal

2.0

BTFS

1.5 1.0 0.5 0.0 1

2

3

4

Number of Seeders Fig. 14. Speedup of multiple replicas.

4.3.5. Evaluating BTFS under volunteer volatility In this experiment, we demonstrate that BTFS can work under the volatility circumstance where nodes can dynamically join and leave the system at anytime during the job execution, and discuss how the volunteer volatility affects the rendering process. To

handle such the volatility situation gracefully, DrQueue job scheduler keeps monitoring the availability of the nodes. If a node leaves, fails or the client process on the node ceases, DrQueue will remove the node from the list of available nodes, and the job will be rerun on the other node. We emulate the volatility of a node by randomly

190


Table 4 Number of files and disk usages on each seeder.

No. of files Disk usage (MB)

Seeder 1

Seeder 2

Seeder 3

Seeder 4

Total

119 422

103 259

112 310

101 229

435 1220

Table 5 Performance of rendering process under different volunteer volatility. 2700 s

1800 s

Time (s) # Rerun Time (s) BTFS + DrQueue BTFS + BOINC

7525 7861

11 14

11, 347 10, 200

900 s # Rerun Time (s)

# Rerun

23 20

31 36

15, 840 15, 546

starting and killing the client process (including BTFS client and Blender) on the testing nodes. The On and Off time periods of each node are independently modeled by two random variables with exponential distribution. We vary the average On time to 900, 1800 and 2700 s, while fixing the average Off time to 1800 s. We report the average rendering time and the number of rerun jobs for the medium job in Table 5. Intuitively, the larger On period gives longer availability time which is more preferable for the volunteer-based distributed rendering than the smaller one as observed by lower rendering time and number of rerun jobs. Yet, the BTFS can work with DrQueue under the volatile environment to finish the jobs in all cases. 4.3.6. Using BTFS with BOINC volunteer computing platform This experiment shows that BTFS can also be used with BOINC (Volunteer Computing), the well-known volunteer computing platform, for a volunteer-based animation rendering system. Here, we setup own BOINC server at NECTEC site. The BOINC platform is only used for job and user management; while the data repository and distribution is delegated to BTFS. To avoid porting Blender to the BOINC platform, Blender is executed under the BOINC wrapper on client nodes. Rendering jobs are submitted to the BOINC as shell scripts that mount the BTFS, invoke Blender to render a specific frame using data from BTFS, and unmount the BTFS when finished. Note that almost all BOINC’s job execution parameters such as redundant computing, or resource requirements still function as normal. By using BTFS, users have an easier way to create jobs as they do not need to pack and stage all relevant files into jobs like native BOINC. Besides, the data can be more efficiently distributed among volunteer nodes reducing the BOINC server load. We emulate the volunteer volatility by the method described in Section 4.3.5. The average rendering time and the number of rerun jobs are placed in the bottom of Table 5. The results are not notably distinct from those of DrQueue which shows that the BTFS can work in tandem with the BOINC platform for volunteer-based distributed rendering. 5. Conclusions The BitTorrent file system, or BTFS, provides a viable way to improve the communication performance of volunteer-based distributed animation rendering. It allows multiple rendering clients to share and exchange data in the P2P manner so that the transfer time is greatly reduced allowing the rendering process to take place more rapidly. The experiments carried out on a testbed using a production-grade 3D animation show that BTFS can lessen the overall rendering time and lower traffic load on the server comparing to other traditional network file systems. BTFS is implemented in a file system layer to shield the rendering applications from such the complication in P2P communication mechanism. By being transparency, BTFS could be used in not only the rendering but also any other application domains which require disseminating a large

amount of common data to many distributed computers in a short time. Data in BTFS are stored securely by enforcing access control as well as encryption on files. BTFS is designed to have all components scalable and fault-tolerant. Replication is also supported to enhance the availability of data. Although, the core components of BTFS are built around well-developed standard protocols whose ability, maturity and compatibility have been proven, it remains challenged to deploy BTFS on very large-scale environments where there exist thousands of computers. In addition, the “last writer wins” consistency model used in BTFS might not be suitable for all applications. Release consistency model which requires lock and unlock operations when accessing files should further be implemented in BTFS as it is required in some applications. Besides, the partial request files and the data reduplication in BTFS are also worth investigation for future work.

References Anderson, D.P., 2004, November. BOINC: a system for public-resource computing and storage. In: Proceedings of the fifth IEEE/ACM International Workshop on Grid Computing (GRID 2004), Pittsburgh, PA, USA, pp. 4–10. Apache ZooKeeper. Available at http://zookeeper.apache.org/ (accessed 31.10.13). Assawamekin, N., Kijsipongse, E.,2013. Design and implementation of BitTorrent file system for distributed animation rendering. In: Proceedings of the 17th International Computer Science and Engineering Conference (ICSEC 2013), September 4–6. Windsor Suites Hotel, Bangkok, Thailand, pp. 68–72. Bharambe, A.R., Herley, C., Padmanabhan, V.N., 2006, April. Analyzing and improving a BitTorrent networks performance mechanisms. In: Proceedings of 25th IEEE International Conference on Computer Communications (INFOCOM 2006), Barcelona, Spain, pp. 1–12. Big Buck Bunny. Available from: http://www.bigbuckbunny.org/ (accessed 31.10.13). BitTorrent Multitracker Metadata Extension. Available from: http://www. bittorrent.org/beps/bep 0012.html (accessed 31.10.13). Blender. Available from: http://www.blender.org/ (accessed 31.10.13). BURP: the Big and Ugly Rendering Project. Available from: http://burp. renderfarming.net/ (accessed 31.10.13). Cappello, F., et al., 2005. Computing on large-scale distributed systems: XtremWeb architecture, programming models, security, tests and convergence with grid. Future Gener. Comput. Syst. 21 (March (3)), 417–437. Coda File System. Available from: http://www.coda.cs.cmu.edu/ (accessed 31.10.13). Common Internet File System (CIFS). Available from: http://www.cifs.com/ (accessed 31.10.13). Cohen, B., 2003, May. Incentives Build Robustness in BitTorrent. Workshop on Economics of Peer-to-Peer Systems, Berkeley, CA, USA. Costa, F., et al., 2008, April. Optimizing the data distribution layer of BOINC with BitTorrent. In: Proceedings of the 2008 IEEE International Parallel and Distributed Processing Symposium (IPDPS 2008), Miami, Florida, USA, pp. 1–8. Distributed Rendering. Available from: http://www.isgtw.org/visualization/ distributed-rendering/ (accessed 31.10.13). DrQueue, the Open Source Distributed Render Queue. Available from: http://www. drqueue.org/ (accessed 31.10.13). Fedak, G., He, H., Cappello, F., 2008. BitDew: a programmable environment for largescale data management and distribution. In: Proceedings of the 2008 ACM/IEEE Conference on Supercomputing (SC 2008), Austin. Free Rendering by the People for the People. Available from: http://www.renderfarm.fi/ (accessed 31.10.13). FUSE: Filesystem in Userspace. Available from: http://fuse.sourceforge.net/ (accessed 31.10.13). Globus Toolkit. Available from: http://www.globus.org/toolkit/ (accessed 31.10.13). GlusterFS. Available from: http://www.gluster.org/ (accessed 31.10.13). Howard, J.H., et al., 1988. Scale and performance in a distributed file system. ACM Trans. Comput. Syst. 6 (February (1)), 51–81. Kaplan, A., Fox, G.C., Laszewski, G.v., 2007, November. GridTorrent Framework: A High-Performance Data Transfer and Data Sharing Framework for Scientific Computing. Grid Computing Environments Workshop, Reno, NV, USA. KaZaA Lite. Available at http://kazaa-lite.en.softonic.com/ (accessed 31.10.13). Klingberg, T., Manfredi, R. Gnutella Protocol Development. Available from: http://rfc-gnutella.sourceforge.net/src/rfc-0 6-draft.html (accessed 31.10.13). Moose File System. Available from: http://www.moosefs.org/ (accessed 31.10.13). MogileFS. Available from: https://github.com/mogilefs/ (accessed 31.10.13). Napster. Available from: http://www.napster.com/ (accessed 31.10.13). opentracker – An Open and Free BitTorrent Tracker. Available from: http://erdgeist.org/arts/software/opentracker/ (accessed 31.10.13). PublicBitTorrent – An Open Tracker Project. Available from: http://publicbt.com/ (accessed 31.10.13). Qiu, D., Srikant, R., 2004. Modeling and performance analysis of BitTorrent-like peerto-peer networks. ACM SIGCOMM Comput. Commun. Rev. 34 (October (4)), 367–378.

E. Kijsipongse, N. Assawamekin / The Journal of Systems and Software 97 (2014) 178–191 Samba – Opening Windows to a Wider World. Available from: http://www.samba. org/ (accessed 31.10.13). Sandberg, R., et al., 1985. Design and implementation of the sun network filesystem. In: USENIX 1985 Summer Conference Proceedings, pp. 119–130. Shvachko, K., et al., 2010. The Hadoop distributed file system. In: 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST), May 6–7, Lake Tahoe, Nevada, USA, pp. 1–10. SMB for Fuse. Available from: http://www.ricardis.tudelft.nl/∼vincent/fusesmb/ (accessed 31.10.13). stunnel. Available from: https://www.stunnel.org/index.html/ (accessed 31.10.13). The Linux Kernel Archives. Available from: https://www.kernel.org/ (accessed 31.10.13). Volunteer Computing. Available from: http://boinc.berkeley.edu/trac/wiki/Volunteer Computing/ (accessed 31.10.13). vSwarm: Free Render Farm. Available from: http://www.vswarm.com/ (accessed 31.10.13). Wei, B., Fedak, G., Cappello, F., 2005, July. Collaborative data distribution with BitTorrent for computational desktop grids. In: Proceedings of the 4th International Symposium on Parallel and Distributed Computing (ISPDC 2005), France.

191

Wei, B., Fedak, G., Cappello, F., 2007, November. Towards efficient data distribution on computational desktop grids with BitTorrent. Future Gener. Comput. Syst. 23 (8), 983–989. Zissimos, A., et al., 2007, May. GridTorrent: optimizing data transfers in the grid with collaborative sharing. In: Proceedings of the 11th Panhellenic Conference on Informatics (PCI 2007), Patras, Greece. Ekasit Kijsipongse received his Ph.D. in computer science from Mahidol University, Thailand in 2009. He currently works as a researcher at the National Electronics and Computer Technology Center, Thailand. His research interests include distributed and parallel systems, as well as grid and cloud computing. Namfon Assawamekin received her Ph.D. in computer science from Mahidol University, Thailand in 2009. She is currently an assistant professor at School of Science and Technology, University of the Thai Chamber of Commerce, Thailand. Her main research interests are in the areas of software engineering, Web engineering, ontologies, data and knowledge engineering and distributed systems.