On the Verification of Intransitive Noninterference in Mulitlevel Security

0 downloads 0 Views 539KB Size Report
INI can be used to characterize and solve several important security problems in multilevel security systems. In a previous work, we have estab- lished the notion ...
948

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 35, NO. 5, OCTOBER 2005

On the Verification of Intransitive Noninterference in Mulitlevel Security Nejib Ben Hadj-Alouane, Stéphane Lafrance, Feng Lin, John Mullins, and Mohamed Moez Yeddes

Abstract—We propose an algorithmic approach to the problem of verification of the property of intransitive noninterference (INI), using tools and concepts of discrete event systems (DES). INI can be used to characterize and solve several important security problems in multilevel security systems. In a previous work, we have estab-observability, which precisely captures the lished the notion of property of INI. We have also developed an algorithm for checking -observability by indirectly checking -observability for systems with at most three security levels. In this paper, we generalize the results for systems with any finite number of security levels by developing a direct method for checking -observability, based on an insightful observation that the function is a left congruence in terms of relations on formal languages. To demonstrate the applicability of our approach, we propose a formal method to detect denial of service vulnerabilities in security protocols based on INI. This method is illustrated using the TCP/IP protocol. The work extends the theory of supervisory control of DES to a new application domain. Index Terms—Denial of service, formal verification, information flow, interference, intransitive noninterference (INI), observability, purge, security policies.

I. INTRODUCTION

T

HE THEORY of supervisory control of discrete event systems (DES) was introduced over twenty years ago [13], [10]. Since then, the properties of controllability [13] and observability [10] have been used as tools to characterize and solve many problems with diverse application domains. In this paper, we generalize the notion of observability of DES in order to characterize and solve some open problems in the field of security of computer systems and protocols and, hence, extend the application domains of the theory of supervisory control of DES. Manuscript received September 16, 2005; revised March 1, 2005. This research was supported in part by the National Science Foundation under Grant INT-0213651, in part by the National Institute of Health under Grant 1 R21 EB001529-01A1, and in part by the Natural Sciences and Engineering Research Council of Canada under an Individual Discovery Grant. This paper was recommended by Associate Editor M. Zhou. N. Ben Hadj-Alouane and M. M. Yeddes are with the CRISTAL Laboratory, Department of Applied Computer Sciences, National School of Information Sciences, University of Manouba, Manouba 2010,Tunisia (e-mail: [email protected]; [email protected]). S. Lafrance and J. Mullins are with the Design and Realization of Complex Systems (CRAC) Laboratory, Department of Computer Engineering, École Polytechnique de Montréal, Montréal, QC H3T 2B1, Canada (e-mail: [email protected]; [email protected]). F. Lin is with Department of Electrical and Computer Engineering, Wayne State University, Detroit, MI 48202 USA, and also with the School of Electronics and Information Engineering, Tongji University, Shanghai, China (e-mail: [email protected]). Digital Object Identifier 10.1109/TSMCB.2005.847749

For this purpose, the first task is to define the notion of information flow in DES. This notion plays a central role for security specification. It roughly intends to quantify the correlation between unobservable or private strings and the way they are observed by a public observer. Also, unlike in the supervisory control of DES, where the observation of a string can be captured by a simple projection to purge unobservable events, information flow is much more complex in our problem. To illustrate this, let us consider a simple example of a cryptosystem with two security levels: a private or high level ( ) and a public or low level ( ). Suppose we attach a security level to each event say, to and to , then, in the automaton , there is an information flow from to because any public observer of the event has the possibility to know whether occurred or not. The reason is that there is some causal dependency from the private event action to the public event . Such a dependency creates an insecure channel called covert channel. Noninterference requires that no such causal dependency from to exsatisfies noninterference. ists. The automaton In practice, however, many security problems go beyond the scope of simple noninterference. In particular, confidentiality in multilevel security systems, where the relation over the set of security levels capturing allowed information flows, is not transitive. Let us consider now as a more intricate example, a cryptosystem with three security levels: a private or high level ( ), a public or low level ( ), and a downgrading level ( ). In this system, information is not allowed to flow from the high level to the low level, unless it has been previously downgraded (encrypted) through the cryptosystem. As in the previous example, suppose we attach a security level to each event say, to , to and to , then, in this case, the information flow in the automaton is allowed. Intransitive noninterference (INI) captures the allowed information flow from the high level to the low level through the downgrading level. In general, the properties of noninterference and intransitive noninterference are much more complex than what are shown by the above two examples. They must be specified by using purge functions to capture information inferred by a public observer. Since their introduction [15], these properties have been widely used to formulate various security problems [1], [3]–[7], [9], [14]–[16]. In the above two simple examples, it is intuitively easy to check if noninterference or intransitive noninterference is satisfied. However, for large systems that contains hundreds of states and events and many security levels, it is not an easy task to check noninterference or intransitive noninterference. In fact, prior to our work, no algorithm based on a necessary and sufficient condition for checking INI has been obtained.

1083-4419/$20.00 © 2005 IEEE

BEN HADJ-ALOUANE et al.: ON THE VERIFICATION OF INTRANSITIVE NONINTERFERENCE

In a previous work [2], we have developed an algorithmic approach to the verification of intransitive noninterference. We first precisely characterized INI through a new definition of observability with respect to the purge function ( -observability). Then we converted -observability into observability with respect to some projection ( -observability), as used in supervisory control problems. The approach is indirect because we do not check -observability directly. This approach is simple but can only be used for systems with at most three security levels and cannot be generalized for systems with more than three security levels. In this paper, we develop a direct approach for checking -observability. Our approach works for systems with any finite number of security levels, where admissible information flow is specified by a so called security policy, i.e., a security lattice [15] describing admissible information flow among the security levels. Based on an insightful observation that -purge is a left congruence in terms of relations on formal languages, we borrow some techniques from supervisory control of DES and work on strings in reverse to solve an otherwise difficult problem of verifying -observability and, hence, INI. Intuitively, the left congruence just reflects the fact that we should only delete those private events that are not followed by some downgrading events. In the second part of this paper, we apply our theoretical results to the problem of checking robustness to denial of service (DoS) attacks. In recent years, several Internet sites have been subjected to DoS attacks. One of the most famous DoS attacks is the SYN flooding attack [18] on the TCP/IP protocol. Since 1996, this resource exhaustion attack has been launched at several occasions by intruders having the capabilities to initiate, with little effort, a large number of protocol runs. DoS attacks have been perpetrated on Internet e-commerce sites, including Yahoo, Ebay, and E*trade in February 2000 and Microsoft in January 2001. In network DoS attacks on protocols which establish authenticated communication channels, the identity of the intruder is generally unknown because authentication has not yet been completed. One way to prevent such attacks is the use of a sequence of authentication mechanisms, arranged in order of increasing cost (to both parties) and increasing security. With this approach, an intruder must be willing to complete the earlier stages of the protocol before he can force a system to spend resources running the later stages of the protocol. Meadows recently proposed a formal cost-based framework for the analysis of denial of service [11]. The results show how to formalize this principle to make cryptographic protocols more resistant to denial of service by comparing the cost to the defender against the cost to the attacker. Although an intruder is capable to break some of the (weak) authentication steps of a protocol, it will cost him a dissuading effort. Lafrance and Mullins [9] introduced a formal method based on information flow in process algebra for detecting violation of the design principle formalized by Meadows. This paper describes this method in the framework of DES. The basic idea behind the method is to prove that no intruder can use the protocol to interfere with costly actions of the defender, in order to cause resource exhaustion DoS. Using the INI property to capture this interference, we, therefore, obtain a complete proof technique that allows us to detect whenever an

949

intruder may use a protocol to send, with little effort, a fake message in order to provoke costly actions like resource allocation, decryption or signature verification. Such a flaw could be exploited by an intruder in order to launch a DoS attack by wasting the defender’s valuable resources. More specifically, we define a robustness against DoS, a property that verifies whether the behavior of a principal (e.g., server) differs when we introduce an intruder within the protocol. Roughly speaking, if the principal behaves differently when it is being attacked, then we conclude that the protocol design is not compliant with the Meadow’s security design principle. The paper is organized as follows. Section II presents the necessary background on automata, languages, and multilevel security, and characterizes intransitive INI in terms of -observability with respect to a purge function (within the context of a given system automata describing the interactions of all security levels). Section III develops the algorithm for the verification of INI. In Section IV, we propose a framework for the specification of security protocols in terms of DES, along with a formal method based on INI for the detection of denial of service vulnerabilities. We also offer a complete application of our method to the analysis of the transmission control protocol (TCP). II. BASIC CONCEPTS AND BACKGROUND This section is divided into two subsections. The first gives the needed background on DES and languages. The second introduces the necessary concepts from security; in particular, it defines the notion of a purge function, INI, and -observability. A. Automata and Languages In this paper, we model a multilevel security system by an automaton (also called generator or state machine)

where denotes the set of events and the state space, the the initial state, and transition function, the set of marked states. The (partial) transition function describes the system dynamics: given states and event , if the execution of from state takes the system to state . Note that is undefined whenever the event cannot be executed from the state . The behavior of the security system is described by the language generated by and the language marked by . The language generated by is defined as is defined Each trace in represents a possible execution of the is a subset of traces in system. The language marked by that also end in a marked state

In this paper, we will also use nondeterministic automata with -transitions ( denotes the empty string) and several initial states

950

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 35, NO. 5, OCTOBER 2005

where transition function; and language generated by

is the nondeterministic is the set of initial states. The is defined as follows:

An equivalence relation gruence, if

is defined that is, initial states of follows:

For example, give an automata fine an equivalence relation as

if, and only if, is possible from one of the . The language marked by is defined as

Unfortunately, for a left-congruence , we cannot always consuch that is equal struct an automata . To overcome this difficulty, we consider strings in reto verse. Let

and

Automata and languages play an important role in our approach. Therefore, let us briefly recall some results in automata and language theory. For a deterministic or a nondeterministic automaton

the reversed string of Clearly, we have

Definition 1: For an equivalence relation , define a “reas follows: For all versed” equivalence relation

we denote the set of states that are reachable from some initial . The states and co-reachable to some marked states as operator removes the states that are not in

where

denotes the transition function . Clearly,

The following lemma describes the relation between right congruence and left congruence and will be used to derive our main results. is a left congruence, then is a right Lemma 1: If congruence. is a left congruence, that is Proof: Assume that

restricted to (but, in general,

). Another operator that will be used in our approach is the following parallel composition of two deterministic automata and , which describes the joint behavior of and and is defined as

where

It is clear from the above definition that if and .

if if if

, de-

Then, is a right congruence. , we can construct an In fact, for any right congruence, such that is equal to . automata For the problem studied in this paper, the intransitive noninterference property is introduced through a purge function which is not right congruence. Hence, we need to introduce left congruence as follows. An equivalence relation is called a left congruence if

that is, if, and only if, can take the automaton from one of its initial states to one of its marked states. It is well known that a nondeterministic automaton can always be converted to a deterministic automaton that marks the same language (this is obviously preserved for the case of the automata with multiple initial states, that we are defining here). For the ) as the set of all sake of convenience, we often view (or possible transitions for some

is called a right con-

Then, for all

. , then

B. Intransitive Noninterference in Multilevel Systems To formally define INI, let us consider a set of security domains, a set of events partitioned over these domains, de-

BEN HADJ-ALOUANE et al.: ON THE VERIFICATION OF INTRANSITIVE NONINTERFERENCE

scribed by the mapping , and an interference . The domains represent the security levels relation, or channels for which we will define noninterference requireis used to describe the event partiments. The mapping , the set tion: For every specifies the events associated with . The relation defines a security lattice: For domains , , the intended meaning of is to allow the domain to interfere with the domain . We assume that our system, i.e., the combined behavior asso, ciated with all the domains, is modeled by a language generated by a finite automata

To formally define INI, a function on traces called intransitive purge, denoted , is introduced by Rushby [15]. is defined given in the equation shown at using captures the set of the bottom of the page. Intuitively, domains which are allowed to interfere throughout the execution of a trace. is then defined as follows: and if otherwise . Intuitively, is a string reduction function such that, consists of the substring obtained from by removing every event belonging to any domain that should not interfere with the domain . INI can then be defined as follows, which is essentially the definition of Rushby [15]. Definition 2 (Intransitive Noninterference): A language satisfies INI if

It was proved in a previous paper [2] that the problem of verifying INI in a multilevel system can be reduced to the problem of checking -observability. and . Definition 3 ( -Observability): Let is -observable w.r.t. A prefix-closed sublanguage if

Theorem 1: A language satisfies INI if, and only if, -observable w.r.t , for all .

is

951

III. CHECKING

-OBSERVABILITY

In this section, we develop a direct approach to verifying INI by checking -observability. Note that the verification of INI in accordance to the given security lattice, reduces to checking -observability with respect to each domain in this lattice. To by . this end, let us fix a domain and denote ; therefore, For the problem studied in this paper, -observability can be rewritten as

Given , , and , the first phase of our approach, before checking -observability, consists of conthat marks the purged language of structing an automata defined as follows:

Since -induced equivalence is left congruent, the construcis rather involved: First, we need to construct an tion of . Second, we need to automaton for the reversed langauge introduce, in this automaton, information from the security latto obtain an automaton for . Last, tice and then apply in order to obtain we need to reverse the automaton for . The second phase of our approach consists of checking -observability using . First, we establish a corresponand states in ; this defines a dence between states in notion of equivalence on the states of . Second, we check the consistency of -equivalent states of , with respect to -observability. We start the presentation of the above approach by defining the following equivalence relation on traces:

It is easy to see, from the definition of , that is is a right congruence. a left congruence. By Lemma 1, Therefore, to construct an automaton for , we need to work on reversed languages. Step1: Construct a nondeterministic automaton marking by reversing

where . , and the only Note that any state of is an initial state of marked state of is . is the reversed lanLemma 2: The language marked by . gauge of : Proof: Follows by construction.

and if otherwise.

952

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 35, NO. 5, OCTOBER 2005

Step 2: Expand the state space of by adding the domain function to obtain information related to the

, Proof: According to the definition of , to obtain we proceed by erasing events in that should not interfere with the part of the trace considered from its end to its beginning. . Formally This is exactly what happens in

The transition function is defined recursively. Start with all already in the the initial states, and define, for any state , for all range of

by construction of by Lemma

where if otherwise.

,

marks and, hence, recognizes It should be noted that all the reverse postfixes of all the traces in . More importantly, the second component of any state in consists of the list of domains that are allowed to interfere with in any reverse postfix of any trace in recognized by this state. . Moreover, for all Lemma 3: , there exists such that , where . , it is clear that a path Proof: From the construction of

Step 4: Construct a nondeterministic automaton that marks by reversing all the transitions in

where . Step 5: from

to obtain an automaton that is reachable and co-reachable to

Since the operator preserves the marked language, is an automaton for . and are the Lemma 5: The languages marked by -purged langauge: . and . Proof: Follows by the construction of Step 6: Convert into a deterministic automaton

is defined if, and only if

is defined, where if otherwise and with . Step 3: Construct a nondeterministic automaton with -transitions that marks

The transition function is given by

The new automaton replaces some of the transitions of by -transitions, based on the domain information associated with every state. is the reversed lanLemma 4: The language marked by : . gauge of

where , , and is defined as: For , . Lemma 6: The language marked by is the -purged . langauge: Proof: From the automaton theory, . After constructing , we can now obtain a correspondence , captured by the function between states in and states in defined as follows:

We say that is related to if . The following lemma states that: (1) given any trace in leading to a state in , then is related to the state in led to by ; and (2) for each pair of states in related to a marked state in , there exists a pair of strings in purging the same way (i.e., ) and leading, in . respectively, to the states

BEN HADJ-ALOUANE et al.: ON THE VERIFICATION OF INTRANSITIVE NONINTERFERENCE

Lemma 7: The function 1) 2)

has the following properties: ;

Proof: To prove Property 1, let we have

and

953

Therefore

,

Now, to check if satisfies INI, we present the following theorem. if, Theorem 2: Language is -observable w.r.t. and only if is defined is defined Proof: We first show that if is not -observable, then is not the condition in the theorem is not true. Assume that -observable, that is Let and . Since and , . . Let . By Lemma 7, Furthermore, implies is defined; and implies is not defined. Therefore

Therefore is defined To prove Property 2, let and , . Take from to and label it by . Since , any path in , by the construction of , there exist two paths both , one from to for some labeled in , and the other from to for some , we have

is not defined

that is, the condition in the theorem is not true. For the other direction, we show that if the condition in the is not -observable. Assume that theorem is not true, then the condition is not true, that is, is defined By Lemma 7, there exists , is defined implies . implies Therefore

is not defined such that

; and

is not defined

that is, is not -observable. Our verification approach is given by the following algorithm, which summarizes the steps and procedures described above. Algorithm 1. Given , , and with . We verify satisfies INI, by sequentially that applying the following two procedures for . every domain • PROCEDURE A Construct the automaton as follows. marking , according – Construct to Step 1 given above. by adding domain infor– Construct , according to Step 2 given mation to above.

954

Fig. 1. Automaton

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 35, NO. 5, OCTOBER 2005

G.

– Construct marking , by into changing some transition in -transitions, according to Step 3 given above. marking , by re– Construct , according to Step 4 given versing above. by trimming , ac– Construct cording to Step 5 given above. as a deterministic ver– Construct , according to Step 6 given sion of above. is -observ• PROCEDURE B Check that as follows. able w.r.t. , as de– Construct the function fined above. This step essentially labels of all the components of the form . any states in the state space of – According to Theorem 2, check the conevent, sistency, with respect to any that are in labeled of any states of . components of any one state in

Fig. 2. Automaton

G

.

Fig. 3. Automaton

G

.

Fig. 4. Automaton

G

.

The following example gives an application of the above algorithm. Example 1: Consider the three domain lattice, with . The system , is modeled by the automata given in Fig. 1, where and . Clearly in this case to verify INI, we only need to check -observability with respect to . We start by applying the steps of PROCEDURE A of Algorithm 1. – We construct from the automaton marking . This automaton is not shown since it can be easily deduced from . , shown in – We construct the automaton and adding domain Fig. 2, by expanding information to its states. Note that also marks . the nondetermin– We construct from , shown in Fig. 3, istic automaton marking . Only one transition becomes , since it enters a state on an event from a domain not in the source of the entered state.

– By reversing the automaton , we ob, shown in Fig. 4, marking . tain remains unchanged – The automaton with the application of the operator; . hence, , shown in Fig. 5, is a – The automaton . deterministic version of Next, we apply the steps of PROCEDURE B of Algorithm 1.

BEN HADJ-ALOUANE et al.: ON THE VERIFICATION OF INTRANSITIVE NONINTERFERENCE

Fig. 5.

Automaton

G

.

– In particular, note that , is the second state from the left where in Fig. 5. is possible – Now note that the event after in but not possible after . is Hence, according to the theorem 2, -observable with respect to . not In conclusion of this example, we can say that, according to Theorem 1, the system of Fig. 1 does not satisfy INI. A Complexity Note: We note that Algorithm 1 is of exponential complexity. There are two major contributions to this complexity: 1) In Step 2 of PROCEDURE A, the state space of is expanded to include domain information; the automaton , it should be though, the complexity of this step is noted that the number of domains for most practical problems is small. 2) In Step 6 of PROCEDURE A, the automaton is converted into a deterministic automaton, with a complexity ; in fact, this also coincides with the worst-case of complexity of the entire algorithm. IV. DETECTION OF DENIAL OF SERVICE VULNERABILITIES IN SECURITY PROTOCOLS In this section, we show how to use the results of Section III to analyze security protocols. First, we specify a protocol, based on the method proposed by Lafrance and Mullins [9]. Second, we investigate the robustness of this protocol against DoS, using INI. In a DoS attack an intruder causes resource exhaustion to a defender (e.g., server) through the steps of a protocol (most commonly an authentication protocol). In such an attack, at any step of the protocol (but typically at the beginning), the intruder sends a fake message that requires little effort to forge, but causes a great waste of resources to the defender. If the defender may simultaneously process several requests (protocol runs), the intruder can then repeat this attack up to the point of exhausting all defender resources. A. Specification of Security Protocols We describe the behavior of a participant (principal) , involved in a run of the protocol , with a generator: . The event set of the principal is parti, where denotes the private events, tioned as denotes the synchronization events, i.e., the events used and by to communicate with other principals.

955

Given the specification of each principal involved in one or more runs of the protocol , the whole protocol is given as the parallel composition of all these specifications. Consider a protocol composed of two participants, Alice ( ) and Bob ( ), then can be specified with the generator: . An intruder is simply specified as another . The protocol being atprincipal, with its own generator tacked by the intruder is, therefore, specified by the generator: . Example 2 (TCP): The TCP provides a reliable connectionoriented data stream delivery service for applications. A TCP connection commonly uses memory structures to hold data related to the local end of the communication link, i.e., the TCP state, the IP address, the port number, the timers, the sequence number, the flow control status, etc. (a full description is given by Schuba et al. [18].) Before starting the transmission of data between a source principal and a destination principal , TCP uses a connection establishment protocol called the three-way handshake Message Message Message First, initiates the connection by sending a synchronization , containing a fresh sequence number along with packet the IP addresses of and . Next, acknowledges using along with a new synchronization packet . Fi. nally, acknowledges ’s using and specifications are given in Figs. 6 and 7. The important events are as follows: • ’s events : : creation of (private event); – – : sending (synchronization event); : receiving and (synchronization – event); : verification that matches , and cre– ation of (private event); : sending (synchronization event). – ’s events: • , and : allocation of structures for storing – and creation of and (private events); , and : verification that matches – (private events). A famous DoS attack on the TCP protocol is the SYN flooding attack [18]. Since 1996, this resource exhaustion attack has been launched at several occasions by intruders having the capabilities to initiate, with little effort, a large number of protocol runs. This resource exhaustion attack is possible due to the following: The intruder generates a fake IP address and creates using this fake IP address. The intruder then simply sends the to the server , who processes it as follows: first, fake is the server allocates data structures in which packet stored; second, the server creates for ; third, the ; last, the server sends and server creates a new to the IP address given by the intruder. The intruder then repeats this attacks with different fake IP addresses and different , but without completing any protocol run. This

956

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 35, NO. 5, OCTOBER 2005

Fig. 6.

Specification of principal A.

Fig. 7.

Specification of principal B .

Fig. 8. Specification of the intruder E .

attack is feasible because generating fake IP addresses and SYN packets require little effort! Resource exhaustion occurs because the server allocates expensive data structures upon receiving a SYN packet. The specification of an intruder , engaging and achieving a SYN flooding attack is given in Fig. 8. The following events are used: • ’s events: : creation of , using own IP address, ’s IP ad– dress and some random number , hence, behaving honestly (private event); : sending correct (synchronization event); – : receiving and (synchronization – event); : verification that matches and creation – (private event); of : sending correct (synchronization event); – – •: creation of using fake IP address (private event); : sending fake , hence, launching attack (syn– chronization event). The synchronization events and within ’s automaton (Fig. 7) are related to the attack launched by , and their meaning can easily deduced from the explanations already given. specifying the SYN The automaton flooding attack on the TCP protocol is illustrated in Fig. 9. B. Specification of Robustness Against Denial of Service For every principal (that is not taken as an intruder), we is composed of two assume that its set of private events types, regular and costly, hence, . The costly events are defined in terms of the principal’s memory or CPU capacities. For an intruder , we consider a subset of its as representing ’s admissible attacks. events

Fig. 9. Specification of the SYN flooding attack P .

Consider the following set of domains, . Note that represents the regular that are not admissible, and represents the events of synchronization events of all principals other . Now, we introduce robustness against DoS, based on [9], [11]. We require that no enemy process may interfere on the server’s costly events. This concept of interference is crucial in order to establish a causal dependency between the intruder’s behaviors and the defender’s behaviors : the occurrence of an high-cost actions must be a consequence of the intruder’s actions, and not of some other protocol run, in order to conclude that there is indeed a DoS attack. is roDefinition 4 (Robustness Against DoS): Protocol bust against denial of service if, for every intruder, the genersatisfies INI with the following interference relation, ator . In the following example, we apply Definition 4 to investigate the robustness of the TCP protocol against the SYN flood attack. Example 3 (Investigating DoS For TCP): In the case of the specification of the TCP protocol given in Example 2, with the we have , following event partitions among the domains: , , , , , and

BEN HADJ-ALOUANE et al.: ON THE VERIFICATION OF INTRANSITIVE NONINTERFERENCE

. In this case, the relation, , used in Definiand . tion 4, requires that Consider , the specification of the SYN flooding attack shown in Fig. 9. Note that it is sufficient to and , since they are the only check INI for domains domains that have noninterference requirements. We proceed using Algorithm 1, and find that: 1) is -ob; but 2) is not -observservable with respect to able with respect to . In particular, our analysis reand are the cause for veals that the traces failing to satisfy -observability with respect to . and Indeed, , with , but . V. CONCLUSION We studied a class of security problems in computer systems and protocols that can be characterized by the property of INI. Our objective is to obtain an efficient method to verify a necessary and sufficient condition for INI and, thus, solve an open problem in the security area. Our approach is algorithmic: we developed an algorithm based on automata representing the system/protocol behavior that can be used to derive a definite answer as to whether or not INI is satisfied. Our success in solving this otherwise difficult problem is partly due to the fact that we borrowed techniques from the theory of DES. In particular, we extended the notion of observability in DES to define a new -observability. It turns out that -observability is precisely the property we need to verify INI. We also applied our algorithm to the important problem of formalizing and checking denial of service attacks in protocols. REFERENCES [1] D. E. Bell and L. J. LaPadula, “Secure computer system: Unified Eexposition and multics interpretation,” Mitre Corporation, Bedford, MA, Tech. Rep. MTR-2997, 1976. [2] N. Ben Hadj-Alouane, S. Lafrance, F. Lin, J. Mullins, and M. Yeddes, “Characterizing intransitive noninterference for 3-domain security policies with observability,” IEEE Trans. Autom. Control, to be published. [3] R. Focardi and R. Gorrieri, “A classification of security properties for process algebras,” J. Comput. Security, vol. 3, no. 1, pp. 5–33, 1994/1995. [4] R. Focardi, R. Gorrieri, and F. Martinelli, “Message authentication through non interference,” Algebraic Methodol. Software Technoll, pp. 258–272, 2000. [5] , “Secrecy in security protocols as non interference,” presented at the DERA/RHUL Workshop on Secure Architectures and Information Flow, 2000. [6] J. A. Goguen and J. Meseguer, “Security policies and security models,” in Proc. IEEE Symp. Research in Security and Privacy, Apr. 1982, pp. 11–20. [7] J. T. Haigh and W. D. Young, “Extending the noninterference version of mls for sat,” IEEE Trans. Softw. Eng., vol. 13, no. 2, pp. 141–150, Feb. 1987. [8] J. E. Hopcroft, R. Motwani, and J. D. Ullman, Introduction to Automata Theory, Languages, and Computation, 2nd ed. Reading, MA: Addison-Wesley, 2001. [9] S. Lafrance and J. Mullins, “An information flow method to detect denial of service vulnerabilities,” J. Univ. Comput. Sci., vol. 9, no. 11, pp. 1350–1369, 2003. [10] F. Lin and W. M. Wonham, “On observability of discrete-event systems,” Inf. Sci., vol. 44, pp. 173–198, 1988. [11] C. Meadows, “A cost-based framework for analysis of denial of service networks,” J. Comput. Security, vol. 9, no. 1/2, pp. 143–164, 2001. [12] J. Mullins, “Nondeterministic admissible interference,” J. Univ. Comput. Sci., vol. 6, no. 11, pp. 1054–1070, 2000.

957

[13] P. J. Ramadge and W. M. Wonham, “Supervisory control of a class of discrete-event processes,” SIAM J. Control Optim., vol. 25, no. 1, pp. 206–230, 1987. [14] A. W. Roscoe and M. H. Goldsmith, “What is intransitive noninterference,” presented at the 12th IEEE Computer Security Foundations Workshop, 1999. [15] J. Rushby, “Noninterference, transitivity and channel-control Security policies,” SRI International, Menlo Park, CA, Tech. Rep. CSL-92-02, 1992. [16] P. Y. A. Ryan and S. A. Schneider, “Process algebra and noninterference,” presented at the CSFW-12, Mordano, Italy, Jun. 1999. [17] S. Schneider, “Verifying authentication protocols in CSP,” IEEE Trans. Softw. Eng., vol. 24, no. 8, pp. 447–479, Aug. 1998. [18] C. L. Schuba, I. V. Krsul, M. G. Kuhn, E. H. Spafford, A. Sundaram, and D. Zamboni, “Analysis of a denial of service attack on TCP,” in Proc. IEEE Symp. Security Privacy, May 1997, pp. 208–223.

Nejib Ben Hadj-Alouane was born in Mahdia, Tunisia, in 1963. He received the B.S. degree in computer engineering from Syracuse University, Syracuse, NY, in 1984, and the M.S.E. and Ph.D. degrees in computer information and control engineering from the University of Michigan, Ann Arbor, in 1986 and 1994, respectivley. From 1994 to 1996, he held the position of R&D Associate at the DOW Chemical Company, Midland, MI, working on fault-tolerant networks for global plant control systems. From 1996 to 1997, he was a Postdoctorate Fellow at the University of Michigan Intelligent Transportation Systems Laboratory. From 1997 to 1999, he held the position of Information Systems Director, for a group of companies, in Tunisia. Since 1999, he has been an Assistant Professor at the National School of Information Sciences, University of Manouba, Manouba, Tunisia. His research activities focus on discrete-event and hybrid systems, security in computer networks and systems, and issues in real-time and embedded systems.

Stéphane Lafrance was born in Piedmont, QC, Canada, in 1975. He received the B.Sc. degree in mathematics from the Université de Montréal, Montréal, QC, Canada, in 1997, and the M.Sc. degree in mathematics from the Université du Québec à Montréal in 1999. He is currently pursuing the Ph.D. degree at the École Polytechnique de Montréal. His current research interests focus on the specification and validation of cryptographic protocols using process algebras and equivalence-checking proof methods.

Feng Lin received the B.Eng. degree in electrical engineering from Shanghai Jiao-Tong University, Shanghai, China, in 1982, and the M.A.Sc. and Ph.D. degrees in electrical engineering from the University of Toronto, Toronto, ON, Canada, in 1984 and 1988, respectively. From 1987 to 1988, he was a Postdoctoral Fellow at Harvard University, Cambridge, MA. Since 1988, he has been with the Department of Electrical and Computer Engineering, Wayne State University, Detroit, MI, where he is currently a Professor. His research interests include discrete-event systems, hybrid systems, robust control, and image processing. He was a consultant for GM, Ford, Hitachi, and other auto companies. Dr. Lin received a George Axelby Outstanding Paper Award from the IEEE Control Systems Society for a paper he coauthored with S. L. Chung and S. Lafortune. He is also a recipient of a Research Initiation Award from the National Science Foundation, an Outstanding Teaching Award from Wayne State University, a Faculty Research Award from ANR Pipeline Company, and a Research Award from Ford. He was an Associate Editor of IEEE TRANSACTIONS ON AUTOMATIC CONTROL.

958

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 35, NO. 5, OCTOBER 2005

John Mullins received the M.Sc. degree in theoretical computer from the University of Montréal, Montréal, QC, Canada, and the Ph.D. degree in telecommunication software from the Institut National de la Recherche Scientifique, Montreal, QC, Canada, in 1992. Since 1999, he has been a Professor with the Department of Computer Engineering, École Polytechnique de Montréal, where he currently heads the Design and Realization of Complex Systems (CRAC) Laboratory. He is also an Associate Member of the Combinatorics and Mathematical Computer Sciences Laboratory (LaCIM), a research center associated with the Mathematics Department, Computer Sciences Department, and the Canada Research Chair in Algebra, Combinatorics, and Mathematical Computer Sciences, Université du Québec à Montréal. From 1992 to 1995, he has been an Invited Professor at the School of Information Technology and Engineering (SITE), University of Ottawa, Ottawa, ON, Canada, and, then, from 1995 to 1999, he pursued his career in Europe as an Invited Professor or Researcher in many prestigious research institutes, including the INRIA, Lorraine, France; École Normale Supérieure de Lyon, Lyon, France; École Centrale de Nantes, Nantes, France; the Royal Institute of Technology (KTH), Stockholm, Sweden; and the Swedish Institute in Computer Science (SICS), Kista, Sweden. His research interests include applications of mathematical logic to the analysis of concurrency and security: models and calculi for concurrent systems (including object-oriented and mobile systems) and security systems (e.g., security controllers and security protocols), modal and temporal logics with fixed points, and applications to system verification and description.

Mohamed Moez Yeddes was born in Korba, Tunisia, in 1971. He received the degree from the Computer Engineering Faculty of the University of Tunisia, Tunis, in 1996, and the DEA diploma and Ph.D. degree from the Institut National Polytechnique de Grenoble, Grenoble, France, in 1997 and 2000, respectively. He is currently an Assistant Professor at the National School of Information Sciences, University of Manouba, Manouba, Tunisia. His current research interests focus on the modeling and control of hybrid and timed discrete-event systems using Petri nets and automata. Recently, he has become concerned with the applications discrete-event systems tools to the area of security.