Intrusion Detection Techniques for Virtual Domains - IEEE Xplore

Download PDF
28 downloads 4115 Views 435KB Size Report
Comment
domain. There is a need to develop intrusion detection techniques to deal with different types of attacks in virtual domains. In this paper we consider the design ...
Intrusion Detection Techniques for Virtual Domains Udaya Tupakula

Vijay Varadharajan

INSS Research, Faculty of Science Macquarie University, Sydney, Australia {udaya.tupakula, vijay.varadharajan}@mq.edu.au Abstract—A virtual domain enables grouping of related virtual machines running on separate physical machine into a single network domain with a unified security policy. Since the virtual machines can be running different operating systems and applications, the attacker can exploit even a single vulnerability in any of the operating system or applications in a single virtual machine to attack other machines in the virtual domain. There is a need to develop intrusion detection techniques to deal with different types of attacks in virtual domains. In this paper we consider the design choices for attack detection and propose intrusion detection architecture to deal with attacks in virtual domains. Our architecture takes into account the specific features of the virtual machine as well as security policies of the virtual domains to deal with different types of attacks in virtual machines. We have described the operation of the proposed system architecture in detail. Finally we present how our model can efficiently deal with different types of attacks and performance analysis of our model. Keywords-Virtual Domains; Intrusion Detection Systems Architecture; Malware;

I.

INTRODUCTION

High performance, efficient resource utilization and high availability of the system resources are the major goals for the development of Virtualization and cloud computing technologies. A virtual machine monitor (VMM) [1] is an additional software layer that has complete control on the physical resources and enables to run multiple operating systems on a scalable computer. Cloud computing is one of the increasingly popular technologies where the cloud services providers allocate the computing resources to their customers (tenants) to host their data or perform their computing tasks. Cloud computing can be categorized into different services such as Software as a Service (SaaS), Platform as a Service (PaaS), and Infrastructure as a Service (IaaS). Virtualisation is one of the key technologies for the cloud. The current Internet environment is vulnerable to a range of different types of attacks such as [2-4] and several new types of attacks are appearing on a daily basis. Enlargements of computer infrastructure by creating virtual machine have raised the vulnerability of these systems to security threats, attacks and intrusions. Some specific examples of intrusions that concern system administrators include attempted break-in, running unauthorized program in virtual machine, Trojan horse, virus and Denial-of-Service. Intrusion detection techniques help to detect, prevent and as well as react to the attack and intrusions on computer

978-1-4673-2371-0/12/$31.00 ©2012 IEEE

Dipankar Dutta Amazon Development Center Bangalore, India [email protected] systems or any network based infrastructures. Intrusion detection can be defined as software or hardware systems that automate the process of monitoring the events occurring in a computer system or network, analyzing them for signs of security problems. The same definition can be applied to virtual machines. As the processes of a virtual machine are not fully controlled or monitored by the virtual machine monitor, there is a greater chance of vulnerability. Hence there is need for intrusion detection systems to detect attacks on the virtual machines. In this paper we consider a generic architecture for virtual domain where each customer hosts several virtual machines in the cloud. We consider an attacker model and propose novel intrusion detection architecture to deal with the attacks on the virtual machines in virtual domains. We will present the operation of our model and a detailed discussion on the components of our model. Our security architecture takes into consideration the features that are specific to each virtual machine and the domain wide security polices in which the virtual machine is a member of. The paper is organized as follows. Section II considers some related work, which are relevant for this paper. In Section III, we consider generic architecture for virtual domains and the attacker model for virtual domains. Then we propose intrusion detection architecture for dealing with attacks in virtual domains. Section IV presents implementation details and performance results of our model and Section V concludes the paper. II.

RELATED WORK

In this section, we present some of the important techniques that are related to our proposed architecture. Dunlap et al [5] proposed ReVirt architecture for secure logging by placing the logging tool inside the VMM. ReVirt claims to log enough information such as real time clock, keyboard, mouse events, user inputs and system calls, which enables the administrator to replay the execution of virtual machine. Since the logs are isolated from the virtual machine, they can be used to replay the logged information and analyse the attacks in case of compromise of the virtual machines. However additional tools are needed to detect attacks on the virtual machines. Garfinkel [6] proposed a Livewire intrusion detection system which makes use of the virtual machine monitor (VMM) to obtain the state of the virtual machine with additional commands for inspection, monitoring and

administration. The policy engine queries the OS library to interpret virtual machine system state and events from the VMM interface and decide whether or not the system is compromised. Lares [7] considers techniques for active monitoring by placing hooks in the virtual machine that needs to be monitored. When a program that needs to be monitored reaches the hook, the control is passed to the security tool to ensure the safety of the system. The main advantage of this technique is that since the hooks are placed in the virtual machine that needs to be monitored, it has more clear view of the state and hence the semantic gap is reduced. Lycosid [8] detects hidden process in the virtual machines by comparing the implicit guest view with the VMM image. The idea is to obtain the process running in the guest virtual machine and then obtain the process from the VMM. If the number of processes listed in the guest VM and the VMM are different then there is a hidden process. It does not address attacks generated by visible process. Although some of the attacks are possible with the hidden processes, there can be legitimate reasons for the occurrence of hidden process. For example since the reports are generated at different time intervals there is possibility for a new process to be initiated or terminated after the report is generated by the guest virtual machine and before the process report is generated by the VMM. Vigilante [9] is a collaborative where each host runs specific software which captures the information regarding the exploit of the worm and distributes a Self Certifying Alert (SCA) to warn other hosts regarding the spread of the worm. The end hosts can use the information in the SCA to identify if it is vulnerable to the worm and apply a host based filter to prevent the worm. Egele et al [10] proposed dynamic spyware analysis tool which makes use of the data flow and control flow to monitor the flow of the sensitive data when it is processed by the web browser and the browser help objects. The tool can analyze the behaviour of the browser help objects and generates a detail report on what data is being sent and to which location it is sent. Blade architecture [11] deals with the drive by download malware by capturing the user interactions and permits only the executables that are authorised by the user. Any download that are not authorised by the user or not supported by the browsers are considered as suspicious and isolated into the security zone. However in the current environment it is not an easy task for the users to distinguishing between benign and malicious downloads. Robertson et al [12] proposed anomaly based intrusion detection for detecting the attacks with minimal training data set. If suspicious events matched with the minimal training data set, then the behaviour is compared with the similar events in the global dataset with considerable training. Although several VMM-based security techniques have been proposed, none of the techniques considers the specific characteristics of the virtual domains in networked systems

to deal with different types of attacks, which is the focus of this paper. Cabuk et al [13] proposed techniques for implementing virtual domains using techniques such as Ethernet encapsulation, VLAN tagging and VPN. However if the virtual machines that belong to the same network are hosted on the same physical machine, then this does not use any of the physical networking devices. Techniques such as sHype [14] and Shamon [15] have been proposed to enforce mandatory access control in virtual machine monitors. sHype provides a reference monitor interface inside the hypervisor to enforce information flow constraints between virtual machine partitions. Shamon is an extension of sHype to enforce mandatory access control across networked systems such as trusted virtual domains. Catuogno et al [16] proposed techniques for securing the data in the mobile storage devices. The main idea is to enforce a master device with the policy for accessing the storage devices and there is a proxy for each physical machine. The data in the flash drive is encrypted and stored in the master device. When the flash drive is inserted into any of the virtual machine in the virtual domain then the proxy in the virtual machine validates with the master device if the flash drive is legitimate. The data in the mobile device is encrypted and can only be retrieved by the virtual machines that belong to same domain. Any changes to the data are secured using lazy encryption where only modified data is encrypted. Compared to these techniques [13-16], the focus our work is on intrusion detection techniques in virtual domains. III.

OUR APPROACH

In this section we consider generic architecture for virtual domains in the cloud. We consider an attacker model and discuss some of the design choices for developing the security techniques to deal with the attacks. Then we propose security architecture to deal with the attacks in virtual domains in the cloud. We will present the operation of our model and detail description of its components. A. Architecture Overview Consider the case where each customer hosts several virtual machines in a cloud environment. Hence the virtual machines that belong to single customer can be hosted on different hypervisors. Also in some cases, for efficient usage of resources, the cloud service provider can host different customer virtual machines on a single hypervisor. For example, as shown in Figure 1 two different customers are hosting three virtual machines on the cloud service provider infrastructure. In this case the cloud service provider can host all the customer virtual machines on three physical servers. Although the VMM1 and VMM3 are hosting virtual machines that belong to single customer, VMM2 is hosting virtual machines that belong to different customers. In such cases, there are specific security requirements. The customers want to ensure that information can be shared among their virtual machines and at the same time ensure that their virtual machines are secure from other customer virtual machines. Also, the

C2 Admin

C1 Admin

Customer 1 VM11

VM12

communication channels between virtual machines in the virtual domains have been addressed in other papers such as [13, 15]. We do not consider privacy issues in this paper.

Customer 2 VM13

VM21

VM22

VM23

VMM1

VMM2

VMM3

Hardware

Hardware

Hardware

Cloud Service Provider Figure 1: Virtual Domains in Cloud

customers may want to easily manage their virtual machines and enforce domain wide security policies on their virtual machines. This is analogous to traditional LAN environments protected by security gateways such as firewalls. Our architecture involves interconnected virtual machines that are hosted on different physical machines, put together to form virtual domain under a security policy. For example, a customer who is hosting multiple virtual machines in the cloud needs to enforce his domain specific security policies on all the virtual machines. For example, Figure 1 shows how the virtual machines that belong to same customer can be grouped into a single domain. The C1 and C2 admin machines determine the domain wide security policies that need to be enforced on their virtual machines within the virtual domain and security policies for communication of virtual machines between different virtual domains. Our aim in this paper is to develop an intrusion detection architecture that would enable efficient detection and prevention of different types of attacks in such virtual domains in the cloud environments and the isolation of the malicious entities generating these attacks. In this paper, we will assume that the VMM which are used for hosting the customer virtual machines are trusted and secure. Incidentally, this is a common trust assumption that is made with many VMM or Hypervisor based systems. This assumption is based on the premise that compared to operating systems, VMMs are intended to be small in size and in principle can be verified (or designed) to be secure. If this assumption is not made, the vulnerabilities in the VMM can be exploited by the attacker to attack any or all of the virtual machines that are running on the physical machine. We also assume that the cloud service provider co-operates with the customers to implement the security policies in the hypervisor. Note that in this paper, our focus is only on the intrusion detection aspects. Security aspects such as secure

B. Attacker Model In this section, we consider the attacker model for the virtual domains. Some of the cloud service providers [17] offer free and unlimited communication between the virtual machines that belong to same customer. However, if such communication is not monitored for security attacks, it can lead to different types of attacks in the cloud. For example, the attacker can generate attacks on all the virtual machines within a virtual domain by compromising a single virtual machine in the virtual domain. Let us consider a simple scenario with customer virtual machines that are running on different hypervisors are grouped together into a virtual domain. Figure 2 shows the two virtual domains that can be formed from the Figure 1 case scenario. In this case, there is free communication between virtual machines that belong to same customer. Hence, consider the case where the attacker has compromised VM13 of Customer1 domain by exploiting zero day vulnerability. Now the attacker can generate attacks on all the virtual machines (VM11, VM12) within the Customer1 virtual domain. Also, since VM13 and VM21 virtual machines which belong to different customer domains are implemented on the same physical server, there is possibility for the attacks to spread to other customer virtual domains. Hence there is need for techniques to detect and prevent such attacks. VM11 HBST

VM13

VM12

HBST VMM1 & IDS

HBST VMM2 & IDS Hardware

Hardware

VM21

VM22

VM23

HBST VMM2 & IDS

HBST HBST VMM3 & IDS

Hardware

Hardware

Figure 2. Attack Scenario

As shown in Figure 2, a single compromised virtual machine (VM13) client can flood all other virtual machines within the virtual domain. One of the common behaviour with any of the attacks such as slammer, conficker and torpig is that they result in congestion of the shared networks. For example, Slammer worm floods the network with malicious UDP packet, conficker and torpig perform

bruteforce attack to obtain administrative access to other machines and storm worm will result in bulk email messages. Also, it has to be noted that if the communication of virtual machines within the domain is not monitored by the cloud service provider, they are the primary target for the attacker to spread the malware since the hit rate is considerably high with a rare chance of detection. Hit rate is used as a measure of finding vulnerable hosts for the spread of malware. In the case of virtual domains, there is more possibility for the virtual machines to have similar applications and operating systems since they belong to single customer or virtual domain. The detection chances are rare since the cloud service provider has permitted unlimited and free communication between the customer’s virtual machines. In case of zero day attack, the detection process becomes more complicated if the compromised host floods other virtual machines with spoofed source address. Although there are some security tools on each virtual machine there are several challenges with the traditional security tools to deal with the emerging attacks. The emerging attacks such as conficker and torpig have the capabilities to disable the security tools and any security features such as auto updates that are running in the vulnerable machines. C. Design Choices In this section we will consider some of the design choices for developing the security architecture to deal with the attacks in virtual domains. Although the customers use host based security tools in their virtual machines to detect the attacks, there are some limitations with the traditional security tools to deal with the attacks in the current environment. The host based tools have good visibility of internal state of the monitored system and can detect the attacks more efficiently. However, as shown in Figure 2, since the tools are implemented on the monitored system itself, they are vulnerable to attacks by the attacker. Hence there is additional need for the cloud service provider to monitor the customer virtual machine interactions to detect attacks. Although it is possible for the cloud service provider to monitor the communication of the virtual machines that are implemented on different physical servers using external tools, it is not efficient to monitor the interaction of virtual machines that are implemented on the same physical server using external monitoring tools. Hence there is need for VMM based monitoring for such VM communications. For example, it is efficient to monitor the interactions between VM11–VM12 using security tool placed in VMM1 and monitor interactions between VM13-VM21 using security tool placed in VMM2. Also in some cases of external monitoring, it may be extremely difficult to determine the attacking source once the attack traffic is placed on the physical medium. Hence the attacks have to be prevented even before the attack traffic is placed on the medium for such cases. For example, consider the case where a customer virtual machine is compromised and flooding with spoofed identity. Hence if

such attacks are not monitored by the cloud service provider, it becomes extremely difficult to determine the attacking source after the attack traffic is placed on the physical medium. If the interactions of the customer virtual machine are not monitored by the cloud service provider, there is high chance for attacks. Hence a system is needed that detects any unauthorized modification forced by an attacker and able to run continually with minimal human supervision. We are talking about “Minimal Human Supervision” because there might be several hundreds of virtual machines, which might be dynamically created and destroyed on demand process. Thus machine-learning paradigms are one of the best ways to automate this detection process. Recently there is considerable research interest to develop VMM-based security tools to deal with the attacks. Since we have considered that the customer’s virtual machines are hosted on VMM in the cloud service provider infrastructure, we believe that the VMM-based security tools can also be used to efficiently deal with the attacks in virtual domains. However we need to ensure that the proposed security tools should take into consideration different factors such as specific features of the virtual machines and specific features of the virtual domains to deal with the attacks. Also, the proposed technique should not impose significant overhead on the cloud infrastructure. If the traffic generated by the customer virtual machines is not monitored by the cloud service provider, this can lead to different types of attacks on the cloud service provider infrastructure and also on other customer virtual machines. Hence we assume that there is co-operation between the customers and cloud service providers to deal with the attacks. D. System Operation In this section, we will present the operation of our model. As shown in the Figure 3, Policy Enforcement Component and Behaviour Capture Engine are the important components of our model. The components can be integrated into the VMM that is hosting the customer virtual machines. All the interactions of the customer virtual machine are monitored by the cloud service provider. Let us consider the operation of our model. The customer virtual machine traffic is received by the Policy Enforcement Component. It enforces the security policies that are specific to the virtual machines and the domain wide security policies. If the VM traffic is not violating any of the VM specific policies or virtual domain policies then the traffic is logged and forwarded to the destination. However if the customer virtual machine traffic is violating any of the VM security policies or virtual domain security policies, then the traffic is dropped and an alert is generated to the corresponding virtual domain administrator. In Figure 3, the VM_Pol refers to the security policies that are specific to each virtual machine. These policies are developed by considering the operating system, applications running in the virtual machine and the resources allocated to the virtual machine. The VD_Pol

refers to the policies that need to be enforced for communication of virtual machines between different virtual domains. Cloud Service Provider Behaviour Capture Engine

I/P

Policy Enforcement Component

Pass

O/P

VM_Pol( ), VD_Pol( )

Fail Drop

Figure 3. Security Architecture for Virtual Domains

The Behaviour Capture Engine is used to analyse the interactions of the virtual machines and develop baseline security policies for the customer virtual machine. Our current model only supports offline analysis of the customer virtual machine behaviour. We are using artificial neural network based learning algorithms for capturing the behaviour of the customer virtual machines. Our model supports signature based and anomaly based detecting and prevention of the attacks. We are using artificial neural network based learning based algorithms for analyzing the behaviour of customer virtual machines and detecting attacks. We have identified security policies that can be used for detecting emerging attacks such as conficker and torpig. For example, the following security policies were developed by analyzing the customer virtual machines with Windows XP operating system and one of the commercial host based security tool in the virtual machine. The Windows XP systems have a default configuration to check for updates at 3:00 AM and the security tool in the virtual machine checks for updated attack signatures every 10 minutes. Hence form these behaviours we were able to determine security policies that need to be enforced in the policy enforcement component to deal with the emerging attacks. If the customer virtual machine fails to check for the updates at expected intervals then the customer virtual machine is considered as suspicious. In the following sections, we will present a detail discussion on the components of our model. E. Policy Enforcement The Policy Enforcement Component enforces the security policies to ensure secure operation of the virtual domain. For example, this module can prevent if the virtual machine is trying to dominate the usage of resources within the virtual domain. The policy enforcement component uses

different techniques to validate the VM traffic. For example, it validates the VM traffic against known attack signatures, anomaly based detection policies and virtual domain policies. The evaluation process of the policy enforcement works as follows: If any of the packet(s) from the virtual machines are matching with a known attack signature, then the packet(s) are dropped. If the traffic does not match with any of the attacks signatures then it is randomly validated against the statistical policies in the anomaly detection module. If the VM traffic is found to be legitimate by the signature and anomaly based detection component, then the traffic is validated against the virtual domain security policies and passed to the destination. The relevant virtual domain policies which are to be enforced on the VM traffic are determined from the destination address of the VM traffic. If the VM traffic does not satisfy the virtual domain security policies, then the packets are dropped and an alert is generated to its administrator. Let us consider the operation of the sub components of the policy enforcement component. The Policy Enforcement Component is used to enforce signature based and anomaly based intrusion detection policies on the communication between the virtual machines. The VM_Pol has security policies that are specific to the virtual machines. For example, it can have signatures for different applications running in the virtual machine. The VD_Pol has the policies has the security policies that need to enforced for communication of the virtual machines between different virtual domains. The customer or virtual domain administrators determine what type of interactions is permitted between the virtual domains. VM_Pol has several sub policies to validate the VM traffic and detect attacks. Let us consider how attacks can be detected with some of the sample policies in the VM_Pol. The Fl_Val validates the traffic that is originating from the virtual machine. Only the traffic that is considered as legitimate is forwarded to the destination. If the traffic is found to be suspicious, then the traffic is dropped and the virtual machine is isolated from other virtual machines in the virtual domain and an alert is generated to the relevant virtual domain administrator. For example, the Fl_Val ensures that any traffic originating from VM has correct Mac address and IP address. If the source IP address or MAC address of the VM traffic is spoofed then the traffic is dropped and the virtual machine is isolated from virtual domain. Hence attacks can be detected at the source VMM that is hosting the compromised customer virtual machine. Note that since the source address of all the packets is validated by the Fl_Val module, it is not possible for the virtual machines to generate attack traffic with spoofed source address. However at this stage, it is still possible to generate attack traffic with correct source address and we will see later how this is detected and prevented in our architecture. The FL_Val logs the traffic in the Trans_Str before forwarding the traffic to the destination. Trans_Str maintains details of the VM transactions. It is important to

maintain such logs both at the source VMM and destination VMM since the current VMM’s support live migration of virtual machines between different servers. Hence it is important to maintain a log of the traffic generated and received by each customer virtual machine. In case of attack, the logs can be analysed to determine the virtual machine that is generating the attack traffic. However the time interval for storage of logs is dependent on the resources available at the VMM. The Trans_Ctrl has details of the VMM that are hosting other virtual machines within the virtual domain. It is used for the dynamic controlling of the VM transactions. The security policies are dependent on the resources available at the destination end. For example, if the destination virtual machine is experiencing congestion, the destination Trans_Ctrl can send a request to the source Trans_Ctrl to reduce the sending rate from the hosted VM. In some cases, if the virtual machine is being restarted, it may not be able to respond to the client requests. With our model, the destination Trans_Ctrl can inform the source Trans_Ctrl or the client that the virtual machine is being restarted. Similarly in some cases, the transactions may have to be suspended during the migration of the virtual machine. The VD_Pol enforces the domain wide security policies. If the communication of the virtual machine is destined to virtual machines in different virtual domain, then inter virtual domain security policies are enforced by this component. The current model only supports permit and deny policies between the virtual domains. If the communication is permitted according to the virtual domain policy then the traffic is forwarded to the destination virtual machine. If the traffic is not permitted according to the virtual domain policies, then the transaction is terminated and an alert is generated to the virtual domain administrator. The virtual domain administrator can then update their virtual domain policies or decide to perform further analysis on his virtual machine. F. Behaviour Capture Engine The Behaviour Capture Engine is used to analyse the virtual machine interactions and identify the suitable security policies that need to be enforced on the customer virtual machines. For example, if the virtual machine is running Windows XP operating system, then the virtual machines have a default check for the auto-updates at 3:00 AM. Similarly, the host based security tools that are used for monitoring and detecting the attacks in the customer virtual machines can have different behaviour. For example, the Sophos security tool checks for new attack signatures for every 10 minutes. The Behaviour Capture Engine has three main stages, as depicted in Figure 4. These stages are i) scanning, which consists of capturing packets from customer virtual machine and normalization of the connection records; ii) training and testing network which consists of configuring the neural network; iii) reporting, which involves reporting the behaviour of the virtual machines. More details of these stages are presented in the following subsections:

Figure 4. VM behaviour analysis using ANN

Scanning Module: The Scanning module is used to give an interface to networks. It provides a way to scan the virtual network interface of the VMs and capture the network packet. It returns a connections records similar kind of KDD record set, which contain list of feature value. Initially, neural network is trained on the KDD data and later on the connection records of the VM. After getting the connection record from the virtual network interfaces of customer virtual machines, we extract the feature can be used to train the neural network. This task is further divided into two sub-tasks called the feature selection and normalization. Feature Selection: This is the first step after scanning. Basically it selects some features from the entire feature list, based on which the neural network should be trained. In the original KDD dataset contain 41 features. Out of these 41 features, we can select some of the feature depending on the complexity of network and desired IDS system. Selecting the important properties and categorizing them are very important for IDSs. There can be a large number of properties and features that can be used in the learning process and not all of them are necessary or useful. Dealing with some or all of them may contain some challenges for IDS (such as the difficulty of getting them all, missing values or null values). Hence, it is very important to specify the appropriate method of selection to determine and categorize the features into a number of classes based on importance: High, Mid and Low. High-class features represent the most useful features. Mid-class features are of less impact on the learning process. Finally, the Low-class features are of the slightest impact on the detection process. This categorization can accelerate the whole learning and detection efficiency of the overall system. Normalization of input dataset: After selecting the features, we need to normalize the data, such that they can be feed to neural network. There are a number of ways we can normalize the data, into numerical values. Neural Network Module: Neural network module is the heart of our machine learning algorithm. This is responsible for whole learning process. Before proceeding to the

learning algorithm, it is necessary to define the structure of neural net. We use training multi-layer neural network using back propagation algorithms (BPN). The neural net basically consists of a number of layers and a number of nodes in each layer. Our neural net consists of 3 layers, namely input layer, hidden layer and output layer. The number of nodes in the input layer is equal to number of features selected for the training. The number of nodes in the hidden layer varies from 5-15, depending on the complexity of the neural network. The number of nodes in the output layer depends on the way we want to structure IDS. One output node is sufficient to tell whether a connection is malicious or not. But to determine the type of attack, more than one output node is necessary. The backpropagation algorithm is fairly simple. It basically consists of the Learning Phase and Test Phase. When the network fully trained, we can proceed with testing the network. We have created a list of test dataset, with which we test the network. Report Module: This module helps to analyse the customer virtual machine interactions and generate appropriate report. Now let us consider how different security policies can be determined by the Behaviour Capture Engine. Consider the case where the customer virtual machine, which is running Windows XP operating system and commercial host based security tools, is infected with zero day attack. For example, the attacker has targeted the customer virtual machine by exploiting zero day attack. Hence this will not be detected by the host based security tool in the virtual machine. The attacks such as conficker and torpig, the attacker disables the auto update features and any security tools in the vulnerable machine. Hence if we consider that the customer virtual machine is infected with such malware, the host based security tool in the virtual machine fails to check for updates every 10 minutes. Hence this behaviour can be used to identify and enforce some security policies in the policy enforcement engine to detect attacks. For example, the policy enforcement can raise the alarm if the host based security tool in the customer virtual machine fails to check for updates every 10 minutes or if the Windows XP does not check for updates at 3:00 AM. IV.

IMPLEMENTATION

We have implemented our model shown in Figure 3 on XEN VMM. In the following section we will present an overview of Xen and implementation details of some important components of our model. Finally we present the performance results of our model. A. Xen Overview In Xen, there is a privileged VM which is known as Domain 0 which is used for hosting and the management of the guest virtual machines. The guest virtual can be running several types of client or server applications on different operating system platforms such as Windows, UNIX//Linux. We have extended the Dom 0 to provide the functionalities of our model. To minimize the code in the hypervisor, the

designers of Xen have opted to include the device drivers in the privileged domain. In the following paragraph, we will present a brief overview of the Xen details that are related to our model and then discuss how the Dom 0 was extended to suit our model.

1: for all Packet P in ip_queue do 2: Log P 3: if P is received from "virbr0" then {Received from a Domain U} 4: Decode real interface I from P.mark 5: DomID := getDomIdByMAC(P.inMAC) {Simple look-up in XenStore} 6: DomID' := getDomIdByIf(I) {Simple look-up in XenStore} 7: if DomID != DomID' then 8: Report MAC spoofing, drop P and go to line 1 9: end if 10: if is spoofed(DomID, P.sourceIP) then 11: Report IP spoofing, drop P and go to line 1 12: end if is_spoofed(DomID, IP) Require: real DomID, packet IP Ensure: False, if IP is spoofed, True otherwise i: MAC := look up MAC address by IP in DHCP server in Domain 0{NULL if not found} ii: DomID' := look up DomID in XenStore by MAC {NULL if not found} iii: return DomID == DomID' Figure 5: Algorithm for detecting spoofing attacks

The privileged domain or Dom 0 manages the actual device by managing the complexities of the device and exports to the guest virtual machines a generic class of device driver interface for the guest virtual machines. The front end drivers are installed in the virtual machines and the back end driver is installed in the Dom 0 which has privileged access to the physical resources. The backend driver is also responsible for ensuring fair access for the usage of the devices by multiple guest virtual machines. For example, netback/netfront drivers are used for network interface cards and blkback/blkfront drivers are used for block devices such as disks. The backend driver is responsible for forwarding the guest VM traffic to the Internet and received packets to the corresponding guest virtual machines. The backend operates in bridge or router mode. The guest virtual machine which needs to transfer the data issues a generic request to the front end driver which is responsible for forwarding the request to the back end driver. The privileged domain queues the request and forwards them to the actual hardware. Domain 0 runs the device driver specific to each physical device and

Figure 6: Attack Alerts

communicates with the guest domains through asynchronous shared memory transport. XenStore is database for sharing the configuration information and used as a mechanism for controlling devices in guest domains. Xend daemon is a special process which runs as root in Dom 0 which is responsible for providing administrative interface to the hypervisor allowing the administrators to define security policies for virtual machines. In our implementation, the components of our model are placed between the front end drivers of the guest virtual machines and back end driver of the host virtual machine. B. Implementation We have implemented our model on Xen VMM. We have used Xen 3.1.2 VMM with virtual machines running different operating systems and applications. When a new instance is created, Xen hypervisor assigns its preconfigured network interfaces to it. Every interface has a uniform name consisting of a Domain ID and interface ID. For example, vif1.2 is a second network interface in Domain 1. In a nutshell, an outgoing packet from Domain U passes its sending network interface, enters ebtables filter, then virtual bridge "virbr0", iptables filter and finally a physical network interface connected to an external network, peth0. We will describe implementation of some of important modules for analyzing the VM traffic. The Fl_Val captures network packets from a kernel using iptables in connection

with libipq module (ip queue kernel module) and validates the source address of the traffic. However, one of the challenges was that since packets from VMs enter iptables filter after passing "virbr0", information about the original sending interface is lost. We have used ebtables to encode the information in packet:mark. This is done by patching Xen script which launches VMs. When a VM is launched, a patched Xen adds an ebtables rule which adds the information to every packet from the launched VM. Fl_Val then decodes the information and is able to reliably determine a sending VM regardless of the packet content (it may be spoofed). We have used scapy for generating the attack traffic with spoofed source address. Figure 5 shows the algorithm for validating the customer virtual machine traffic. Figure 6 shows the alerts when attack traffic with spoofed source address is detected by the policy enforcement component. C. Performance Evaluation We have conducted several experiments with different types of attacks, and in this section, we present some of these performance results. We have used the SPECjvm 2008 benchmarks to validate the performance of our architecture. SPECjvm 2008 benchmarks consist of 38 applications grouped into 11 categories. The benchmarks use different types of work loads with some of the well known applications such as compression, compiler, scientific, XML for validating the performance. If a category has multiple benchmarks, then the result for the benchmark is calculated using geometric mean. The composite score shows the geometric mean of all the benchmarks. Higher scores represent better performance. The score represents the number of times the benchmarks were run during the iteration time. The default analysis consists of 120 seconds of warm-up time for each benchmark and 240 seconds of iteration time. The results do not include the warm-up time. Each analysis runs for a minimum of 2 hours.

100 90 80 70 60 50 40 30 20 10 0

Base Run

Figure 7: SPEC Benchmark Results

xm l

sta rt u p su nf lo w

se ria l

sc io im ar k.l ar sc ge im ar k.s m al l

y

ga ud

m pe

de rb

cr yp to

pr es s

co m

pi ler

co m

Co m po s

ite

IDS

The SPECjvm benchmarks were installed on the Dom 0 of Xen VMM. Figure 7 shows the benchmark results performed on Dell machine with Intel i5-3210M Processor, 2.50 GHz, 8M Cache, 8GB DDR3 RAM with Xen 3.1.2 VMM and Centos 5.1 host operating system and four customer virtual machines. The virtual machines were running windows XP SP2 and Linux operating system with 1 GB RAM. In Figure 7, the base run shows the results of the machine with Xen VMM, Dom 0 and three virtual machines which belong to different virtual domains. Different types of applications such as yahoo messenger, quick time real player, Apache services, skype applications, and auto refresh websites such as news, cricinfo.com were running on the virtual machines. The applications generate different types of traffic at regular intervals for example to maintain the connection status, and to update the news. The composite result for the base run is 42.71 ops/m without invoking our model and without any performance tuning. IDS series shows the result with our model. In this case virtual machine traffic is monitored for spoofed source address, logged and validated against signature and anomaly based detection policies. We have validated the customer VM traffic for 3879923 attacks. Total of 104 file types for different applications such as rar, zip, doc, docx, html were monitored for intrusions. The composite result in this case is 41.92 ops/m. Hence compared to Series 1 results, the overhead in this case is in the order of 2%. However, the overhead can vary for different types of attacks. V.

CONCLUSION

In this paper we have considered the design choices and proposed intrusion detection architecture to deal with attacks in virtual domains. The proposed architecture takes into account security policies of virtual domains as well as the specific features of the virtual machines to detect the attacks. We have also described the implementation of our model on Xen virtualization system and analyzed the performance evaluation of our model. REFERENCES [1] J.E. Smith, Ravi Nair, “The Architecture of Virtual Machines”, IEEE Internet Computing May 2005. [2] Moore, D., Paxson, V., Savage, S., Shannon, C., Staniford, S., and Weaver, N. “Inside the Slammer worm”, IEEE Security and Privacy 1, 4, Jul. 2003. [3] Brett Stone-Gross, Marco Cova, Bob Gilbert, Richard Kemmerer, Christopher Kruegel, Giovanni Vigna, “ Analysis of a Botnet Takeover”, IEEE Security & Privacy, Sept. 2010.

[4]

[5]

[6] [7]

[8]

[9]

[10] [11]

[12]

[13]

[14]

[15]

[16]

[17]

Seungwon Shin, Guofei Gu, "Conficker and Beyond: A Large-Scale Empirical Study", Proceedings of ACSAC 2010, Texas, USA, Dec 2010. George W. Dunlap, Samuel T. King, Sukru Cinar, Murtaza A. Basrai, Peter M. Chen, “ReVirt: Enabling Intrusion Analysis through Virtual-Machine Logging and Replay”, Proceedings of OSDI, 2002. T. Garfinkel and M. Rosenblum, “A virtual machine introspection based architecture for intrusion detection”, Proceedings of NDSS, February 2003. Bryan D. Payne, Martim Carbone, Monirul Sharif, Wenke Lee “Lares: An Architecture for Secure Active Monitoring Using Virtualization”, Proceedings of the IEEE Symposium on Security and Privacy, 2008. Stephen T. Jones, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau,“VMM-based hidden process detection and identification using Lycosid”, Proc. of ACM VEE, March 2008. Manuel Costa, Jon Crowcroft, Miguel Castro, Antony Rowstron, Lidong Zhou, Lintao Zhang and Paul Barham, “Vigilante: End-to-End containment of Internet Worms”, Proceedings of the 20th ACM SOSP, Oct 2005. M. Egele, C. Kruegel, E.Kirda, H.Yin, and D.Song, “Dynamic Spyware Analysis”, Proceedings of Usenix, 2007. Long Lu, Vinod Yegneswaran, Phillip Porras, Wenke Lee, “BLADE: An Attack-Agnostic Approach for Preventing Drive-By Malware Infections”, Proceedings of ACM CCS, Oct 2010. William Robertson, Federico Maggi, Christopher Kruegel and Giovanni Vigna, "Effective Anomaly Detection with Scarce Training Data", Proceedings of NDSS 2010. Serdar Cabuk, Chris I. Dalton, HariGovind Ramasamy, and Matthias Schunter, “Towards automated provisioning of secure virtualised networks”, Proceedings of ACM CCS 2007. Reiner Sailer, Trent Jaeger, Enriquillo Valdez, Ramón Cáceres, Ronald Perez, Stefan Berger, John Griffin, Leendert van Doorn, “Building a MAC-based Security Architecture for the Xen Opensource Hypervisor”, Proceedings of 21st ACSAC, 2005. Jonathan M McCune, Stefan Berger, Ramón Cáceres, Trent Jaeger, Reiner Sailer:, “Shamon -- AndSystem for Distributed Mandatory Access Control”, 22 ACSAC, December 2006. Luigi Catuogno , Hans Löhr , Mark Manulis, Ahmadreza Sadeghi , Marcel Win, “Transparent Mobile Storage Protection in Trusted Virtual Domains”, Proceedings of Usenix Lisa 2009. Amazon Inc. Amazon Elastic Compute Cloud (Amazon EC2). http://aws.amazon.com/ec2/.