A Novel Defense Mechanism against SYN Flooding

0 downloads 0 Views 156KB Size Report
Abstract- SYN flooding exploits the TCP three-way handshake process by sending many connection requests with spoofed source. IP addresses to the victim.
A Novel Defense Mechanism against SYN Flooding Attacks in IP Networks Mohamad Chouman, Haidar Safa, and Hassan Artail American University of Beirut Beirut, Lebanon {mks07, hs33, hartail}@aub.edu.lb Abstract- SYN flooding exploits the TCP three-way handshake process by sending many connection requests with spoofed source IP addresses to the victim. This keeps the victim from handling legitimate requests by causing it to populate its backlog queue with forged TCP connections. In this paper we propose a novel defense mechanism that makes use of the edge routers of the spoofed IP addresses networks. These edge routers determine whether the incoming SYN-ACK segment is valid or not by maintaining a matching table of the outgoing SYNs and incoming SYN-ACKs and also by using the ARP protocol. If the incoming SYN-ACK segment is not valid, the edge router resets the connection at the victim’s machine freeing up an entry in the victim’s backlog queue and enabling it to accept other legitimate incoming connection requests. The proposed mechanism introduces also a collaborative model to encourage various networks to protect each other. Implementation and test trials have shown the efficiency of the proposed mechanism.

between the two endpoints when the source returns an ACK acknowledging the SYN-ACK. The connection continues until one of the endpoints issues a FIN/RST segment or times out. SYN flooding attacks target the TCP three-way handshake process by flooding the victim with connection requests using spoofed source addresses [4], [5], [6], [7]. Each request causes the victim host to reserve space in the backlog queue and to wait for the third ACK, which will never arrive, as shown in Figure 1. Because every TCP connection establishment requires an allocation of significant memory resources, there is a limit of the number of concurrent TCP half-open connections. Consequently, the attacks paralyze the server when the arrival rate of forged connection requests exceeds the rate at which resources are being freed.

Index Terms— Denial-of-Service, IP spoofing, SYN flooding. Source

I.

INTRODUCTION

SYN_SENT

In a world where more services are relying on internet technology, security is an essential. Compromises in the security of servers can affect the quality and integrity of the service being offered. Security breaches include Intrusion, Information Theft, Modification, and Denial of Service (DoS). DOS attacks are the most conspicuous attacks since they keep legitimate users from accessing the service that they are entitled to by consuming the server's resources such as bandwidth, CPU cycles, data structures, etc. Consequently, providers and consumers suffer because the technology these services are relying upon has been attacked [1]. A DoS attack on a system succeeds by means of exploiting a weakness that leads to the server being overwhelmed. For example, a simple DoS attack can exploit a system’s insufficient bandwidth by simply flooding it with packets whose IP addresses are unreachable. A popular and clever DoS attack is SYN flooding which exploits the normal behavior of TCP protocol [2]. The power of this type of attacks lies on that the Internet connection oriented services depend on TCP. In TCP, the three-way handshake process is used to establish communication and synchronize the sequence numbers between any two endpoints [3]. In this process the source initiates a connection by sending SYN TCP segment to the destination as shown in Figure 1. The destination replies by sending a SYN/ACK TCP segment that acknowledges the initial SYN. The connection is established

SYN I

Attacker SYN_SENT

Victim SYN I

SYN_RCVD

SYN J ACK I+1 ESTABLISHED

SYN J ACK I+1

ACK J+1 ESTABLISHED

Figure 1. Normal TCP connection (left) and half-open TCP connection (right)

Several solutions have been proposed to detect, countermeasure or prevent SYN flooding attacks [1], [5], [8], [9], [10], [11], [12]. In these defense mechanisms measures are taken at the edge of the victim’s network (i.e. firewall) or inside it. In this paper we propose a novel SYN flooding defense mechanism that makes use of the edge routers of the spoofed IP addresses’ networks in order to defend the victim by making these routers help the victim to free up entries in its backlog queue. The remainder of this paper is organized as follows. Section 2 describes and discusses some of the existing defense mechanisms. Section 3 introduces the new proposed defense mechanism. Implementation, discussion, and analysis are presented in Section 4. Conclusions are drawn in Section 5. II. RELATED WORK Several firewall-based solutions and products were proposed [4], [5], [8], [9], [12]. In one of these solutions, the firewall acts on behalf of the host like a proxy [9]. When a SYN arrives, the firewall sends a SYN/ACK to the source. If the

CCECE 2005- CCGEI 2005, Saskatoon, May/mai 2005 IEEE

CCECE / CCGEI, Saskatoon, May 2005 0-7803-5558-2/05/$20.00 ©2005 0-7803-8886-0/05/$20.00 ©2005 IEEE

Destination

2145

final ACK does not arrive, as would be in case of an attack, the firewall resets the connection and the destination does not receive any packets. On the other hand, if the connection is legitimate and the firewall receives the third ACK, it forms a new connection with the destination on behalf of the source. The main drawback of this approach is the delay in the legitimate connection establishment since the host should wait for the firewall to establish the connection before itself establishing the connection. In a second solution, the firewall acts as a gateway and monitors the traffic passing between the source and destination [9]. When it sees a SYN/ACK being sent from the destination to the source, it responds by sending an ACK to the destination, thus reallocating the resources and moving the connection out of the queue. If it is an attack, the firewall then sends a RST to the destination and the connection is dropped. The main drawback of this approach is that a large number of open connections, irrespective if they legal or not, might burden systems with limited memory and CPU power. In a third solution, the firewall acts as a passive gateway [8]. This approach respects the end-to-end agreement, thus the firewall does not construct any TCP packets on behalf of the source or destination. When it receives a SYN, it keeps record of it and allows it to pass to the destination. However, if the source has not completed the three-way handshake of the previous SYN, any other SYNs will be blocked. Using a wide range of IP addresses will cause this scheme to fail. If each IP address can cause only one half open connection, then the attacker can spoof as much IP addresses as the number of half open connections needed to bring down a host. In addition, firewall-based solutions may become inefficient since the firewall itself may become a target to TCP SYN flooding and can be disabled by a flood of 14.000 packets [1]. In [9], a firewall-based software tool called SYNKill is developed to reduce the impact of SYN flooding attacks. This tool monitors and injects network traffic to and from the machines it is protecting and operates as a state machine. It monitors incoming packets and classifies them into null, bad, good and new states. For example, SynKill sends an RST packet to the destination whenever the IP address moves to a bad state, thus releasing the allocated resources. In this system, it is possible for IPs classified as bad to reach the good state. In [4], a mechanism for detecting SYN flooding attacks was proposed. This mechanism considers a sharp rise in difference between the number of SYNs and FINs/RSTs, within a certain time frame, as an indication of SYN flooding attacks since, under normal TCP, each SYN corresponds to a FIN or RST. Cumulative Sum algorithm is used to keep track of the average number of FINs/RSTs. Upon detecting an attack, one of the solutions discussed above can be triggered. This mechanism fails if the attacker injects into the network a FIN segment for each SYN segment. The system only counts SYN and FIN segments without keeping track of whom they are destined to. Therefore, the FIN packets can be injected with random IP destinations without compromising the attack.

A mechanism similar to the one presented in [4] was proposed in [5]. This mechanism, called SYN-Dog, considers the number of SYN-SYN/ACK pairs passing through the edge router of a network, in order to detect if the network is being used to launch a SYN flood attack. Similar to [4], SYN-Dog can fail by injecting SYN/ACK packet into the network. III. PROPOSED MECHANISM In attempt to solve SYN flooding attacks we propose a new collaborative, router-based defense mechanism. In this mechanism, the edge router used to defend the attack is neither on the attacker nor on the victim’s network. Rather, it is on the network to which the spoofed IP address belongs as shown in Figure 2. Thus, its role becomes to help the victim in freeing up entries in the backlog queue. The following is a description of our solution, followed by discussion and analysis. Attacker

Victim

Internet Attacker’s Net

Router

Victim’s Net

Spoofed host

Proposed solution can be installed at this router

Host Spoofed Net

Figure 2. Network Architecture

In a normal TCP behavior, when a router encounters an incoming TCP SYN/ACK packet, it checks whether the destination address of that packet is valid by issuing an ARP request to the destination. If the router receives an ARP response then the destination host must be online, therefore, the packet is forwarded normally. A simple version of the proposed solution consists of inspecting at the router the TCP segments. If the incoming packet is a SYN-ACK segment and the destination fails to reply to the ARP request then the solution consists of building a RST segment and sends it to the victim (i.e. to the source IP address and source port number in the incoming SYN-ACK). The connection is then forcibly terminated, freeing up an entry in the victim’s backlog queue and enabling it to accept other legitimate incoming connection requests. Using ARP is advantageous since ARP requests are usually not blocked by firewalls (unlike other messages including UDP, TCP and ICMP segments). In addition, if ARP is blocked, then the host will be unreachable. In order to better understand the proposed solution, Figure 3 shows how the proposed mechanism in its simplest version replies to a SYN flood attack. We consider an attack that would have three parties involved: the attacker, the victim, and

2146

the spoofed host. The attacker sends a SYN segment with a spoofed IP address as source address. The victim replies with a SYN/ACK destined to the spoofed host. When the packet reaches the spoofed host network, the router checks if the spoofed host is online. If it is, then the spoofed host sends a RST segment to the victim since it is not expecting a SYNACK without having requested a connection. If the spoofed host is unreachable or offline, the router drops the SYN-ACK segment and replies to victim with a RST segment, causing it to free up its resources. Attacker

SYN

Victim Victim’s Net

destination IP, source port and destination port. Each time a host belonging to the spoofed network sends out a SYN segment, an entry is added to the matching table. When a SYN-ACK segment is about to enter the network, the proposed solution checks if there is a matching SYN in the table. If there is a matching SYN, then an ARP request is issued to make sure that the host is still online and the entry is deleted. If there is no matching SYN entry for the incoming SYN-ACK, then a RST segment is sent back to the victim, i.e. to the SYN-ACK source. In addition to reducing ARP traffic, using this SYN/SYN-ACK matching table relieves the spoofed host, when it is brought online, from the heavy SYN-ACK traffic.

Queued Packet

Internet

ARP request

yes

RST

Spoofed host

SYN-ACK

Attacker’s Net

no

Make entry in the SYN/SYN-ACK matching table

Outgoing SYN Packet? yes

If online, reply by RST

Incoming SYN-ACK Packet?

Host yes Router

no Found matching entry in the table?

Spoofed Net Issue ARP request

Figure 3. Attack and simple reply.

The above simple version of the proposed solution can cause congestion and bandwidth consumption in the spoofed network. Indeed injecting an ARP request for each incoming TCP SYN-ACK segment regardless if the spoofed host is online or offline and without making sure that the SYN-ACK destination host has requested the connection would flood the spoofed network with ARP requests. To solve this problem, the proposed solution can be activated only when SYN flood attacks are detected. Attacks can be detected by using a variation of SYN-Dog proposed in [5]. In this context, the outgoing SYN segments and the incoming SYN-ACK segments can be counted. If the number of the incoming SYNACKs is greater than the number of outgoing SYNs, then at least an IP address that belongs to this network has been spoofed and is being used to launch an attack on other networks. Consequently, the proposed mechanism is activated. Another issue is that the attackers can use the proposed solution to amplify their attacks. Indeed, a typical and heavy SYN flood attack (i.e. Distributed DoS) congests the network of the spoofed IPs with ARP requests and the attack efficiency can be doubled if the attackers flood the network that installs the proposed solution with SYN/ACK packets. Therefore to reduce the ARP traffic, we enhance the proposed solution with a matching table that keeps track of the initiated sessions. The algorithm shown in Figure 4 shows the main steps of the proposed solution. The matching table consists of a session id, which identifies a connection, and the time the connection was requested. The session id is a combination of the source IP,

no Got ARP reply? yes Delete SYN entry and forward packet

Send RST segment

Figure 4. Algorithm of the proposed solution.

One may ask why the network of the spoofed address would help the victim’s network. To encourage collaboration between various networks, we introduce a collaborative model that defines the rules for the networks to protect each other. In this model, a network must register its network address with the networks from which it requests help. If the network agrees to help then the request is acknowledged. Otherwise a registration failure message is sent. A registration can be accepted only if the requesting network agrees to protect in its turn the network at which it is registering. Other compensation methods can also be defined and added. The collaborative model defines five types of UDP messages: network registration request, acknowledgment, request failure, and registration refresh. UDP was chosen in order to make the registration process not vulnerable to SYN flooding. To secure further the registration integrity process, we use message authentication codes (MAC)

2147

for integrity and authentication. Keys are exchanged using station to station (STS) key exchange algorithm. IV. IMPLEMENTATION, DISCUSSION AND ANALYSIS The proposed defense mechanism was implemented and tested in an environment that includes 3 networks. Each network is 100 Mbps and includes 15 Intel machines. Each machine has 128 MB of RAM, 333 MHZ of CPU speed and Linux Red Hat 9 kernel 2.4.20-8. We intentionally used slow machines in order to prove that the proposed mechanism is light-weighted. To test the efficiency of the proposed mechanism, several scenarios were run and data were collected accordingly. First a SYN flood attack was launched against a victim for 20 seconds. Five timed connection attempts to each machine in the victim’s network were recorded. The results are shown in Table I. The victim's resources (RAM and CPU) were not depleted during the time of the attack. Therefore, the connection trials 1, 2, 3, 4 and 5 must have timed out because the system's backlog queue was full with half-open connections and not because of lack of computational resources. Table II shows the results of the trials when the proposed mechanism is activated. The proposed solution was able to neutralize the attack and all trials were completed successfully. Table I. Summary of five connection attempts during the attack. Trial 1

Trial 2

Trial 3

Trial 4

3m10s

3m10s

3m10s

3m10s

3m10s

Victim's ~20.00% CPU usage

~20.00%

~20.00%

~20.00%

~20.00%

Victim's ~10.00% RAM Usage

~10.00%

~10.00%

~10.00%

~10.00%

We have proposed a new defense mechanism against SYN flood attacks in IP networks. This mechanism allows the network of the spoofed address to help the victims. The proposed mechanism is accompanied with a collaborative model to encourage the various networks to collaborate in order to protect each other. The paper discussed and analyzed the simplicity and advantages basic of the proposed mechanism. Implementation and test trials show that, our solution retained 99.9% of the connection requests. REFERENCES [1]

Table II. Summary of five connection attempts when the proposed mechanism is used. Trial 1 Trial 2 Trial 3 Trial 4 Trial 5 Connection Established Established Established Established Established result Time Elapsed

V. CONCLUSION

Trial 5

Connection Timed out Timed out Timed out Timed out Timed out Result Time Elapsed

also be installed as an agent in order not to hinder or interfere with the router's main functionality. One router/agent can contribute to protect its own network and help defending many others. From our trials, we observe that the proposed mechanism was able to retain 99.9% of the connection requests. This solution uses the TCP protocol to reverse the actions of the attacker and restore order in the victim's backlog queue. If a sufficient number of routers have the proposed solution installed, we can find two benefits. First, a wide range of IP addresses cannot be spoofed thus hindering the attacks. Therefore the solution can be expanded to protect other types of DoS attacks. Second, it becomes a distributed defense to DoS and DDoS attacks. Finally, the proposed solution allows the router to respond on behalf of the unreachable victim, resuming TCP's normal behavior.

< 1 second < 1 second < 1 second < 1second < 1 second

Router’s CPU usage

~23.00%

~23.00%

~23.00%

~23.00%

~23.00%

Router’s RAM usage

~8.00%

~8.00%

~8.00%

~8.00%

~8.00%

The proposed mechanism has several advantages. It is platform independent. It does not require any modification in the protocol stack, or any special hardware. It can be installed on only one system and is able to protect a group of computers and help defending other networks. The proposed solution can

R.K.C. Chang, “Defending against Flooding-Based Distributed Denialof-Service Attacks: a Tutorial”, IEEE Communications Magazine, vol. 40, no. 10, 2002, pp. 42-51. [2] D. Moore, G. Voelker, S. Savage, “Inferring Internet Denial of Service Activity,” Proceedings of the 2001 USENIX Security Symposium, Washington D.C., August 2001. [3] W. Stevens, TCP/IP Illustrated, Volume 1, Addison-Wesley Publishing Company, 1994. [4] H. Wang, D. Zhang, and K.G. Shin, “Detecting SYN Flooding Attacks”, Proceedings of the Twenty- First Annual Joint Conference of the IEEE Computer and Communications Societies, 2002. [5] H. Wang, D. Zhang, and K.G. Shin, “SYN-dog: Sniffing SYN Flooding Sources”, Proceedings of the 22nd International Conference on Distributed Computing Systems, 2002. [6] B-T. Wang and H. Schulzrinne, “Analysis of Denial-of-Service Attacks on Denial-of-Service Defensive Measures”, IEEE Global Telecommunications Conference, 2003, GLOBECOM’03, vol.3, December 2003. [7] C. Douligeris, A. Mitrokotsa, “DDoS attacks and defense mechanisms: classification and state-of-the-art”, Computer Networks, no. 44, 2004, pp. 643-666. [8] N. A. Noureldien and I. M. Osman, “A Stateful Inspection Module Architecture”, Proceedings of TENCON, 2000. [9] C. H. Schuba, I. V. Krsul, M. G. Khan, E. H. Spafford, A. Sundaram and D. Zamboni, “Analysis of a Denial of Service Attack on TCP”, Proceedings of the IEEE Symposium on Security and Privacy, 1997. [10] U.K. Tupakula and V. Varadharajan, “A Practical Method to Counteract Denial-of-Service Attacks”, Proceedings of the Twenty-sixth Australasian Computer Science Conference, 2003. [11] C. Jin, H. Wang, K.G. Shin, “Hop-Count Filtering: An Effective Defense Against Spoofed DDOS Traffic”, Proceedings of the 10th ACM Conference on Computer and Communications Security, 2003. [12] Check Point Software Technologies Ltd. SynDefender: http://www.checkpoint.com/products/Firewall-1.

2148