Secure Communication for Mobile Agents in an ... - CiteSeerX

Secure Communication for Mobile Agents in an Adversarial Environment Sourabh Bhattacharya

Tamer Bas¸ar

Coordinated Science Lab University of Illinois Urbana, IL 61801 USA Email: [email protected]

Electrical and Computer Engineering University of Illinois Urbana, IL 61801 USA Email: [email protected]

Abstract—In this work, a mobility based perspective of jamming among autonomous vehicles is presented, based on our work in [32], [33], [34] and [31]. The framework of differential game theory is extended from the classical two-player scenario to multiplayer games involving two teams. Necessary conditions satisfied by the optimal trajectories are derived under assumptions of uniqueness and existence of the optimal trajectories. Finally, results are presented for several simulation scenarios involving different numbers of vehicles and their dynamics.

Keywords: Differential Game Theory, Pursuit-Evasion Games, Jamming, Security. I. I NTRODUCTION The decentralized nature of wireless ad hoc networks makes them vulnerable to security threats. A prominent example of such threats is jamming: a malicious attack whose objective is to disrupt the communication of the victim network intentionally, causing interference or collision at the receiver side. Jamming attack is a well-studied and active area of research in wireless networks. In late 2009, there have been reports of predator drones being hacked [36], [37] resulting in intruders gaining access to classified data being transmitted from an aircraft. Unauthorized intrusion of such kind has started a race between the engineers and the hackers; therefore, we have been witnessing a surge of new smart systems aiming to secure modern instrumentation and software from unwanted exogenous attacks. Jamming is a malicious attack whose objective is to disrupt the communication of the victim network intentionally causing interference or collision at the receiver side. Jamming attack is a well-studied and an active area of research in wireless networks. Many defense strategies have been proposed by researchers against jamming in wireless networks. In [29], Wu et al. propose two strategies to evade jamming. The first strategy, channel surfing, is a form of spectral evasion that involves legitimate wireless devices changing the channel that they are operating on. The second strategy, spatial retreats, is a form of spatial evasion whereby legitimate devices move away from the jammer. In [27], Wood et al. present a distributed protocol to map jammed region so that the network can avoid routing traffic through it. The solution proposed by Cagalj et al. [5] uses different worm holes (wired worm This work was supported in part by a grant from ARO MURI and AFOSR.

holes, frequency-hopping pairs, and uncoordinated channel hopping) that lead out of the jammed region to report the alarm to the network operator. In [28], Wood et al. investigate how to deliberately avoid jamming in IEEE 802.15.4 based wireless networks. In [6], Lin Chen et al. propose a strategy to introduce a special node in the network called the anti-jammer to drain the jammer’s energy. To achieve its goal, the antijammer configures the probability of transmitting bait packets to attract the jammer to transmit. For a static jammer and mobile nodes, the optimal strategy for the nodes is to retreat away from the jammer after detecting jamming. Constraints in the motion of the agents as well as of the jammers might lead to situations in which the strategy of simple retreat ceases to be optimal. The scenarios presented in this paper address the aforementioned problem. The overarching goal of our work is to analyze jamming in multi-agent systems from the perspective of pursuit-evasion games so as to relate the resilience in the communication network to their mobility models, in the presence of a mobile adversary in the communication channel. Pursuit-evasion games are a special class of zero-sum games [1] that model conflict scenarios between mobile players. Such dynamic games governed by differential equations can be analyzed using tools from differential game theory [12] [10]. In [32], an elaborate reference to prior work in two-player, as well as, multi-player pursuitevasion games, relevant to this paper, is presented. In this paper, we present a general framework and associated techniques, developed in [32], [33], [34] and [31], to address different scenarios of vehicular jamming in combat scenarios. We also shed some light on future problems relevant to jamming in mobile networks from the perspective of differential game theory. In Section II, we introduce the communication and the dynamic model of the agents. In Section III, we present a formal statement of the problem under consideration. In Section IV, we present a derivation of the necessary conditions that must be satisfied for optimality. In Section V, we present our results for various scenarios and provide a qualitative analysis of our results. In Section VI, we present our current research and also the future problems that need to be addressed.

II. C OMMUNICATION AND M OBILITY MODELS In this section, at first, we present the communication model between a receiver and a transmitter in the presence of a jammer. Next, we present the dynamic models that govern the motion of the nodes as well as of the jammers. A. Jammer and Communication Model Consider a mobile node (receiver) receiving messages from another mobile node (transmitter) at some frequency. Both communicating nodes are assumed to be lying on a plane. Consider a third node that is attempting to jam the communication channel in between the transmitter and the receiver by sending a high power noise at the same frequency. This kind of jamming is referred to as trivial jamming. Two other types of jamming are : 1) Periodic jamming: This refers to a periodic noise pulse being generated by the jammer irrespective of the packets that are put on the network. 2) Intelligent jamming: In this mode of jamming a jammer is put in a promiscuous mode to destroy primarily the control packets. A variety of metrics can be used to compare the effectiveness of various jamming attacks. Some of these metrics are energy efficiency, low probability of detection, and strong denial of service [16] [15]. In this paper, we use the ratio of the jamming-power to the signal-power (JSR) as the metric. From [19], we have the following models for the JSR (ξ) at the receiver’s antenna. 1) Rn model TR PJ GJR GRJ n log10 ( D DJR ) 10 ξ= T PT GT R GRT

2) Ground Reflection Propagation 2 4 PJT GJR GRJ hJ DT R ξ= PT GT R GRT hT DJR 3) Nicholson ξ=

TR PJT GJR GRJ 4 log10 ( D DJR ) 10 PT GT R GRT

where PJT is the power of the jammer transmitting antenna, PT is the power of the transmitter, GT R is the antenna gain from transmitter to receiver, GRT is the antenna gain from receiver to transmitter, GJR is the antenna gain from jammer to receiver, GRJ is the antenna gain from receiver to jammer, hJ is the height of the jammer antenna above the ground, hT is the height of the transmitter antenna above the ground, DT R is the Euclidean distance between the transmitter and the receiver, and DJR is the Euclidean distance between the jammer and the transmitter. All the above models are based on the propagation loss depending on the distance of the jammer and the transmitter from the receiver. In all the above models TR JSR is dependent on the ratio D DJR . For digital signals, the jammer’s goal is to raise the ratio to a level such that the bit error rate [20] is above a certain threshold. For analog voice communication, the goal is to

reduce the articulation performance so that the signals are difficult to understand. Hence we assume that the communication channel between a receiver and a transmitter is considered to be jammed in the presence of a jammer if ξ ≥ ξtr where ξtr is a threshold determined by many factors including application scenario and communication hardware. If all the parameters except the mutual distances between the jammer, transmitter and receiver are kept constant, we can conclude the following TR from all the above models: If the ratio D DJR ≥ η then the communication channel between a transmitter and a receiver is considered to be jammed. Here η is a function of ξ, PJT , PT , GT R , GRT , GJR , GRJ and DT R . Hence if the transmitter is not within a disc of radius ηDJR centered around the receiver, then the communication channel is considered to be jammed. We call this disc as the perception range. The perception range for any node depends on the distance between the jammer and the node. For effective communication between two nodes, each node should be able to transmit as well as receive messages from the other node. Hence two nodes can communicate if they lie in each other’s perception range. We will adopt the above jamming and communication model, for the rest of the paper. B. Dynamics of the nodes We assume that there are m agents in the network in the presence of a jammer. Let the dynamics associated with the ith agent be given by the following equation: x˙ i = fi (xi , ui )

(1)

ni

where, xi ∈ R , ui ∈ Ui ' {φ : [0, t] → Ai | φ(·) is measurable}, where Ai ⊂ Rpi . fi : Rn × Ai → R is uniformly continuous, bounded and Lipschitz continuous in xi for fixed ui . Consequently, given a fixed ui (·) and initial point, there exists a unique trajectory solving Equation (1) [38]. Let the state 0 of node i be denoted as xi ∈ Xi ⊂ Rni . Let Xi denote the state-space of the jammer. We assume that the jammer has the following dynamics associated with itself: 0 x˙0 i = fi (x0i , u0i )

n0i

(2)

where x0i ∈ R , u0i ∈ U 0 i ' {φ : [0, t] → A0i | 0 0 φ(·) is measurable}, where A0i ⊂ Rpi . fi0 : Rni × A0i → R is uniformly continuous, bounded and Lipschitz continuous in x0i for fixed u0i . L ni L n0 i Let X0 = X01 ×· · ·×Xn ×X01 ×· · ·×X0m ⊂ i R L × iR represent the entire state of the system, where, represents the Cartesian product of the Euclidean spaces Rni . We define the workspace [35] as the ambient space in which the agents operate. Since we are interested in vehicular networks, the ambient space of the nodes is either R2 or R3 . Since all the agents reside in the same ambient space, we use Ω to denote the workspace for all agents. Let ¯ xi denote the coordinates of the ith agent in Ω. We assume that Ω is equipped with a distance metric ρ : Ω × Ω → R. As a simple example to highlight the difference between the state space and the workspace, consider the following second

order agent that moves on a straight line, with u as the control input. x˙ 1

=

x2

x˙ 2

=

u

The state space for the system can be represented by the vector x = [x1 x2 ]T . The state space is two dimensional but the agent can only move on a straight line and hence the workspace is Ω = R1 . For any state x, x ¯ = x1 . For two states x∗ , x∗∗ ∈ Ω, ρ(¯ x∗ , x ¯∗∗ ) =| x∗1 − x∗∗ |. 1 In the next section, we present a differential game formulation for the problem of maintaining connectivity among the agents in the presence of the jammer. III. P ROBLEM F ORMULATION In this section, the problem of maintaining network connectivity among the agents is formally stated. Due to the presence of a jammer in the vicinity, the connectivity of the underlying communication network depends on the position of the agents relative to the jammer. The network is modeled as a graph. Since the agents and the jammer are assumed to be mobile, the connectivity of the network evolves in time rendering the graph to be dynamic. Moreover, the topology of the graph depends on the state of the nodes. Therefore, the framework of state-dependent graphs [40] is considered. A state-dependent graph is a mapping, gc , from the state space X, to the set of all labeled graphs on n vertices, G(n), i.e., gc : X → G(n) It is assumed that the order of these graphs at all times is n, since the number of agents remains constant. Let E(gc (x)) denote the edge-set of the graph under consideration. Let d = ρ(¯ xi , ¯ xj ), where ¯ xi and ¯ xj are the coordinates of the nodes i and j in the workspace Ω equipped with a distance metric ρ. Let Br [p] = {y ∈ Ω | ρ(y, p) ≤ r}. We define Sij ⊂ Xi × Xj , to be the following set

jammers is to disconnect the communication network. Since the underlying communication network can be modeled as a graph, various notions of connectivity [39] can be used to specify the objective of the team of jammers. In this work, the objective of the team of jammers is to partition the nodes of the graph into two or more components. Disconnection refers to a situation in which there are two agents in the network and no path exists in the communication network for exchanging messages. Based on the objective of the jammers, we can formulate the following zero-sum differential game between the jammers and the nodes in the network. Problem: Consider a play in which the agents are initially connected. The objective of the jammers is to minimize the time in which it disconnects the communication network by jamming the communication channel between agents. The objective of agents is to maximize the time for which communication network remains connected. The game terminates at the first instant at which the communication network gets disconnected. Compute the optimal motion strategies for each agent. In the next section, we present the necessary conditions that are satisfied by the optimal strategies of the agents. IV. O PTIMAL S TRATEGIES In this section, we introduce the concept of optimal strategies for the vehicles. Given the control histories of the vehicles, {u1i (·)}ni=1 , {u2i (·)}m i=1 , the outcome of the game is denoted by π : X × X0 × Uin+m → R and is defined as the time of termination of the game: π(x0 , {u1i (·)}ni=1 , {u2i (·)}m i=1 ) = tf

(5)

where tf denotes the time of termination of the game when the players play ({u1i (·)}ni=1 , {u2i (·)}m i=1 ) starting from an initial point x0 ∈ X × X0 . The game terminates when the communication network gets disconnected. The objective of the jammers is to minimize the termination time and the Sij = {(xi , xj ) | ¯ x0k ∈ / Bηd [¯ xi ] ∪ Bηd [¯ xj ], ∀k ∈ [1, · · · , m]} (3) objective of the agents is to maximize it. Since the objective function of the team of agents is in Nodes i and j belong to the edge set of gc (x) when the conflict with that of the team of jammers the problem can following condition is satisfied: be formulated as a multi-player zero sum team game. A play n 2∗ m ij ∈ E(gc (x)) if and only if (xi , xj ) ∈ Sij (4) ({u1∗ i (·)}i=1 , {ui (·)}i=1 ) is said to be the optimal for the players if it satisfies the following conditions: The collection of edge states is denoted as n m {u1∗ π[x0 , {u1i (·)}ni=1 , {u2∗ i }i=1 = arg max i (·)}i=1 ] S = {Sij }i,j∈[1,··· ,n],i6=j with Sij ⊂ Xi × Xj {u1i }n i=1 From [40], the state dependent graph is defined as follows:

Definition: Given the set system S, the map gc :X→ Gm with an image consisting of graphs of order m, having an edge between vertex i and j iff (xi , xj ) ∈ Sij , is defined as a state-dependent graph with respect to S. Consider a situation in which the network is initially connected in the presence of a jammers. The objective of the

m n 2 m {u2∗ π[x0 , {u1∗ i }i=1 = arg min i (·)}i=1 , {ui (·)}i=1 ] 2 m {ui }i=1

The value of a game, denoted by the function J : X → R, can then be defined as follows: n 2∗ m J(x) = π[x0 , {u1∗ i (·)}i=1 , {ui (·)}i=1 ]

(6)

The value of the game is unique at a point X in the state-space. An important property satisfied by optimal strategies is the Nash equilibrium property. A set of strategies

1

2

({b ui }ni=1 , {b ui }m i=1 ) is said to be in Nash equilibrium if no unilateral deviation in strategy by a player can lead to a better outcome for that player. This can also be expressed mathematically by the following condition: 1

2

1

2

b−j ], π[x0 , {b ui }ni=1 , {b ui }m ui }ni=1 , u2j , u i=1 ] ≤ π[x0 , {b 2 b1−j , {b π[x0 , u1j , u ui }ni=1 ]

π(x(t), {u1i (·)}ni=1 , {u2i (·)}m i=1 ) = J(x(t)) + h(1 + nJ m X X 0 J(xi )j [fi (xi , u1i )]j + J(x0i )j [fi (x0i , u2i )]j + α(h))

ni n X X

∀j ∈ [1, m]

i=1 j=1

1 2 π[x0 , {b ui }ni=1 , {b ui }m i=1 ],

i=1 j=1

∀j ∈ [1, n] First, let us consider the controls of the team of jammers. From In the above expressions u1−j represents the controls of the Nash property, we can conclude that if u1 = u1∗ , ∀j ∈ j j all the agents except agent j. Similarly, u2−j represents the [1, · · · , n] and u2 = u2∗ then u2 = u2∗ minimizes the left −i −i i i controls of all the jammers except the jth jammer. From the hand side of the above equation. Therefore, we can conclude above definition, we can conclude that there is no motivation the following: for a player to deviate from its equilibrium strategy. In general, 1) The optimal control satisfies the following condition: there may be multiple sets of strategies for the players that are in Nash equlibrium. Assuming the existence of a value, as u2∗ J(x0 )j [fi0 (x0i , u2i )]j (7) i (t) = arg min 2 u i defined in (6) and the existence of a unique Nash equilibrium, we can conclude that the Nash equilibrium concept of personIn a similar manner the controls of the agents satisfy the by-person optimality, is a necessary condition that needs to following condition: be satisfied by the set of optimal strategies for the players, u1∗ (8) J(xi )j [fi (xi , u1i )]j and furthermore, computing the set of strategies that are in i (t) = arg max u1i Nash equilibrium also provides the set of optimal strategies. In the following analysis, we assume that the aforementioned 2) In the case, when ui = u∗i for all players in (7) we conditions are satisfied by the optimal strategies. obtain the following relation: The following theorem provides a relation between the π(x(t), {σi1∗ }ni=1 , {σi2∗ }m optimal strategy of each player and the gradient of the value i=1 ) = J(x(t)) + h(1 + {z } | function, ∇J. J(x(t)) Theorem 1: Assuming that J(x) is a smooth function of x, X nJ ni m X n X X n 2∗ m the optimal strategies ({u1∗ J(x0i )j [fi0 (x0i , u2∗ J(xi )j [fi (xi , u∗i )]j + i }i=1 , {ui }i=1 ) satisfy the followi )]j + α(h)) ing condition: i=1 j=1 i=1 j=1 u1∗ i u2∗ i

≤

where, limh→0 α(h) = 0 and J(xi )j represents the derivative of J with respect to the jth element of xi .

=

m arg max ∇J · f (x, {u1i }ni=1 , {u2∗ i }i=1 ),

=

n 2 m f (x, {u1∗ i }i=1 , {ui }i=1 ),

ui

arg min ∇J · u2

i ∈ [1, n] i = [1, m]

⇒ h(1 +

ni n X X

J(xi )j [fi (xi , u∗i )]j +

[fi0 (x0i , u2∗ i )]j + α(h)) = 0 Taking limh→0 on both sides leads to the following relation: 1+

ni n X X


i=1 j=1

π(x(t), {u1i (·)}ni=1 , {u2i (·)}m i=1 )

Z

t+h

=

dt + J(x(t + h)) t

Using Taylor series approximation of J we obtain the following relation: J(x(t + h)) − J(x(t)) =

= =

n X ∂J δi + |δ|o(|δ|) ∂xi i=1

nJ m X X

J(x0i )j

i=1 j=1

[fi0 (x0i , u2∗ i )]j = 0

(9)

(7), (8) and (9) extend the Isaacs’ conditions that provide the optimal controls for two-player zero-sum differential games to the case of multi-player zero-sum team differential games. The Hamiltonian of the system is given by the following expression:

J(x(t) + f (x(t), {u1i (t)}ni=1 , {u2i (t)}m i=1 )h + h(h)) − J(x(t)) where (h) is a vector with each entry belonging to o(h). Let δ = f (x, u)h + h(h). Then

J(xJ )j

i=1 j=1

i=1 j=1

Proof: The proof essentially follows the two-player version as provided in [10]. Let us consider the game from a point x in the state space at time t. The outcome functional is provided by the following expression:

nJ m X X

n 2∗ m H(x, ∇J, {u1∗ i }i=1 , {ui }i=1 ) = 1 +

ni n X X


i=1 j=1 nJ m X X

J(x0i )j [fi0 (x0i , u2∗ i )]j

i=1 j=1

which is the left side of (9). Hence, (9) can equivalently be nJ ni m X n X X X h[ J(xi )j [fi (xi , u1i )]j + J(x0i )j [fi0 (x0i , u2i )]j expressed as: i=1 j=1

+α(h)]

i=1 j=1 n 2∗ m H(x, ∇J, {u1∗ i }i=1 , {ui }i=1 ) = 0

(10)

In addition to the above conditions, the value function also satisfies the retrogressive path equation which is given in the following theorem. Theorem 2: The value function follows the following partial differential equation (PDE) along the optimal trajectory, namely the retrogressive path equation(RPE) ˚ = ∂H (∇J) (11) ∂x ,where˚denotes derivative with respect to retrograde time. The proof of the above theorem is an extension to the two-player scenario presented in [10]. In the next section, we characterize the terminal manifold. V. T ERMINAL C ONDITIONS In Section III, we proposed a mapping from the state of the system to a graph G that encodes the underlying connectivity of the network of UAVs. The connectivity of G can be investigated by analyzing the Laplacian of the graph that is defined as follows: • Laplacian of a graph (L(G)) : It is an n × n matrix with entries given as follows: −1 if an edge exists between i and j 1) aij = P0m if no edge exists between i and j 2) aii = − k=1,k6=i aik The second-smallest eigenvalue of L(G), denoted as λ2 (L(G)), is called the Fiedler value. It is also called the algebraic connectivity of G. It has emerged as an important parameter in many systems problems defined over networks. In [42], [41], [43], it has also been shown to be a measure of the stability and robustness of the networked dynamic system. Since this work deals with connectivity maintenance in the presence of malicious intruders, λ2 (L(G)) arises as a natural parameter of interest for both players. The following theorem relates the connectivity of G to its Fiedler value. Lemma 1: [39]: A graph G is connected if and only if λ2 (L(G)) > 0. Therefore, all disconnected graphs on m vertices belong to the following set: ˜ = {G|λ2 (L(G)) = 0} G Let F (λ, G) = det(L(G)−λIn ) where, In denotes the identity matrix of dimension n × n. Theorem 3: ˜ = {G | Fλ (λ, G)|λ=0 = 0} G where Fλ denotes derivative with respect to λ. Proof: ⇒ The smallest eigenvalue of the Laplacian of any graph is zero, i.e., λ1 (L(G)) = 0 [39]. Let G be a disconnected graph. From Lemma 1, we can conclude that λ2 (L(G)) = 0. Therefore, λ = 0 is a repeated root of the equation F (λ, G) = 0 ⇒ Fλ (λ, G) |λ=0 = 0. ⇐ F (0, G) = 0 since λ = 0 is an eigenvalue of the Laplacian matrix. In addition, Fλ (λ, G) |λ=0 = 0. Therefore, λ = 0 is a multiple root of F (λ, G) with algebraic multiplicity greater than 1. Hence λ2 (L(G)) = 0 ⇒ G is disconnected.

In [31], we presented the following expression for F (λ, G) in terms of the minors of L(G). Fλ (λ, G) |λ=0

= −

n X

Mii

i=1

,where Mii is the minor of corresponding to the diagonal element in the ith row. Substituting the above relationship in Theorem 3 leads to the following equation for the terminal manifold in terms of the variables {aij }ni,j=1 : n X

Mii = 0

(12)

i=1

Note that the variables {aij }ni,j=1 can be related to the state of the agents and the jammers. Therefore, the above equation can be expressed in terms of the state variables. In the next section, we present different scenarios involving variations in the number of players, as well as their dynamics. VI. R ESULTS In [32], we investigate a scenario in which an aerial jammer intrudes the communication channel in a multiple UAV formation. The UAVs are assumed to be flying at a constant altitude and their kinematics are modeled as a unicycle. The analysis of the previous sections can be applied to this problem for the special case of n = 2 and m = 1. Figure 1 summarizes the entire control algorithm. The controller of each agent takes as input the state variables and runs the RPE to compute the control. This control is then fed into the plant of the respective UAV. The plant updates the state variables based on the kinematic equations governing the UAV. Finally the sensors feed back the state variables into the controllers. In this case the sensors measure the position and the orientation of each UAV. PLANT x1

φ1

y1

x1= −1+σ2*y1 +cos φ 1 y = −σ2*x1 +sinφ 1

σ* 1

1

Jx

σ1* 1

j

φ1= −σ*2+σ1*

Jy

1

1

Jx = −σ2*Jy , Jy1 = −σ2*Jx

σ*1 = sign (Jφ )

1

Jφ 1= Jx sin φ1−Jy

1

1

1

cosφ 1

CONTROLLER UAV1 SENSOR SENSOR

JAMMER

σ* 2

σ* 2

σ2*= −sign ( Jx y1 −Jy x −Jφ −Jφ−Jy xj+ Jxjyj) 1

yj

1

j j

CONTROLLER

PLANT xj

1 1

φj

x j = −1+σ2*yj +cos φ j yj =−σ2*xj +sinφ j φ j = −σ*2+σ j*

Figure 1.

Jx j

σ j* σ*j

σ*j =−sign ( Jφ ) 1

Jy j

Jx j= −σ2*Jy , Jy j= −σ2*Jx j j Jφ = Jxjsin φj−Jyj cos φ j

The Control Loop for the System

Figure 2 shows the trajectories of the players along with their optimal controls for a specific terminal condition. The position of the players corresponding to the termination situation is shown by a small circle in the plots showing the

trajectories of the players. The trajectories of the players are shown for a small time interval just before termination. From the expression of the optimal controls in [32], we can infer that the controls of the players are bang-bang. This is also verified from the simulation results. From the nature of the controls and kinematics of the system, we can infer that the optimal paths comprise arcs of circles and straight line trajectories as motion primitives. Arcs of circles are generated when the UAV keeps its angular velocity saturated at one extremum for a nonzero interval of time. Straight line segments are obtained due to rapid switching between the extremum value of the controls (chattering). An instance of such a behavior is exhibited by UAV2 in Figure 2.

Trajectories of the UAVs

Initial and Final Connectivity among the UAVs

150 50 100 0 −50

50

0

−100 50 0 −50 100

−150 50

0

−50

Figure 3.

−50

−200 −100

−100

50

−50 0 −150 −100 −250 −200

Trajectories of the UAVs and Jammer

Trajectories of the Players

Initial and Final Connectivity among the UAVs

2

10

1 σ1

y

Trajectory 15

5

50

0

0

−1

−5 −5

−2

0

20 0 −20 150 100

−50

0

5 x (a)

10

15

2

2

1

1

50

0

5 t(sec) (b)

10

−100

150 100

0

−150

50 0

0 −50

−200

σ2

σj

−50

0

−1

−2

−2

5 t(sec) (c)

10

−200

0

−1 0

−100

−100 −250

−100

Figure 4. 0

5 t(sec) (d)

Trajectories of the UAVs and Jammer

10

Figure 2. Figure shows the trjectories of the players in a small time interval just before termination. The value of η = 1. The player in red is the jammer. The players in green and blue are UAV1 and UAV2 , respectively. Figure (b) shows the control of the UAV1 . Figure (c) shows the control of the jammer. Figure (c) shows the control of the UAV2 .

In [31], we analyze the previous problem for general m and n. Figure 3 shows a simulation of the trajectories of three UAVs and one jammer from a terminal state. The figure on the left shows the trajectories of the UAVs. The jammer traces the path shown on the extreme right. The figure on the right shows the connectivity of the UAVs. The network of UAVs is initally connected. At termination, the jammers disconnect the network by isolating one of the UAVs. Figure 4 shows a simulation of the trajectories of four UAVs and two jammers from a terminal state. The figure on the left shows the trajectories of the UAVs. The two UAVs on the extreme right represent the jammers. The figure on the right shows the connectivity of the UAVs. The network of UAVs is initally connected. At termination, the jammers disconnect the network into two disjoint components. In [34], we extend our previous work to address the jamming problem in a mobile network containing heterogeneous vehicles. In combat scenarios, teams of vehicles are deployed having different communication and motion constraints. Since we are interested in real scenarios, we choose the vehicles as well as the jammer to resemble the dynamics of terrestrial or aerial vehicles. We use the motion models of UAVs

(Unmanned Air Vehicles) and AGVs (Autonomous Ground Vehicles) to model the dynamics of the players. By neglecting the detailed description of the real system that might render the complete solution to be numerical in nature, the dynamical models are simplified to a level so as to capture the essential kinematic constraints of the system. We assume the following motion models for the vehicles: 1) UAV: We use the five state model [44] for the UAV that takes into account the course angles, the flight path angles and the height of the UAV from the ground during its flight. The dynamic equations are given below: x˙

= W cos ψ cos θ,

y˙ = W sin ψ cos θ g g cos θ z˙ = W sin θ, ψ˙ = η tan φ, θ˙ = (η − 1) W W where Wi represents velocity, ψi the heading angle, θi the pitch angle, g the gravitational acceleration, φi the roll angle and ηi the load factor of the UAVi . W , φ and η, satisfying the constraints |W | ≤ Wmax , |φ| ≤ φmax and 0 ≤ η ≤ ηmax , are the controls of the UAV. The configuration space of the UAV is X ' R3 × S 1 × S 1 . 2) AGV: From [45], we model the AGV as a car-like robot with 5-dimensional configuration space using the following dynamic equations: x˙ θ˙

= v cos θ cos ζ, = v sin ζ,

y˙ = v sin θ cos ζ v˙ = u1 , ζ˙ = u2

where, u1 and u2 , satisfying | u1 |≤ u1max and | u2 |≤ u2max , denote, respectively the linear and

Figure 5. UAV jamming a team of AGV’s. Figure (a) shows the snapshots for the vehicles along their optimal trajectory. Figures (b), (c) and (d) show the optimal controls of Vehicle 1, Vehicle 2 and the Jammer. γ = 0.4, u1max = 0.1, u2max = 0.015, Wmax = 11, ηmax = 1.008, φmax = 0.1

angular accelerations of the vehicles. We also consider the fact that the car has a bound on the steering angle, i.e. | ζ |≤ ζmax . The state space of the systems X ⊂ R3 × S 1 × S 1 . Figure 5 shows the simulation results for the case when a UAV Jamming a team of AGVs. VII. C ONCLUSION AND F UTURE W ORK In this work, we presented a mobility based perspective of jamming among autonomous vehicles. The framework of differential game theory was extended from the classical twoplayer zero sum pursuit-evasion game to the multi-player team game scenario involving two teams. Necessary conditions satisfied by the optimal trajectories were derived under assumptions of uniqueness and existence of the optimal trajectories. Finally, results were presented for several simulation scenarios involving different numbers of vehicles and their dynamics. Our current focus has been to extend the analysis for the case when two teams jam each others communication channel. [46], [47] analyze the aforementioned scenarios. We have formulated the jamming problem as a resource allocation problem by incorporating limitations in power and energy available to each agent, which is a major concern in real situations. Some of our future work are as follows: 1) Computation of Singular Surfaces: In this work, we have computed the trajectories based on the necessary conditions of optimality imposed by the Isaacs’ conditions. In order to complete the construction of the optimal trajectories of the agents, we have to identify the singular surfaces in the state space [1]. 2) Decentralized Strategies: Due to the high computational complexity of evaluating the optimal strategies, one might resort to sub-optimal solutions that can be computed efficiently. In [50] [49], the authors consider a decentralized approach to a team jamming problem.

A multi-pursuer multi-evader game is decomposed into several one-pursuer-one-evader and multi-pursuer oneevader games. Each team is corrupts the measurements of the opposite team. The cost function is a penalty that measures the energy cost or computation cost. The authors propose an algorithm that leads to a computationally efficient sub-optimal strategy for each team. An intersting future research direction is to extend the aforementioned techniques to the jamming problem considered in this paper. 3) Noisy models: In real systems, noise corrupts the actuation signal and the sensor measurements. The problem of noise regulation has been successfully addressed for a special class of systems by treating the noise as an adversary [48]. Unfortunately, there has been hardly any progress regarding pursuit-evasion games with players having noisy observation and actuation models. The nonlinearities present in the dynamic model of the players render the problems inutile from the perspective of mathematical analysis. One of our future efforts is to explore this direction using numerical techniques. An alternate approach to this problem is to use iterative strategies as suggested in [51]. R EFERENCES [1] T. Bas¸ar and G. J. Olsder. Dynamic Noncooperative Game Theory, 2nd Ed. SIAM Series in Classics in Applied Mathematics, Philadelphia, 1999. [2] S. Bhattacharya and T. M. Bas¸ar. Game-theoretic analysis of an aerial jamming attack on a UAV communication network. Technical report, Univeristy of Illinois at Urbana-Champaign, 2010. [3] J. V. Breakwell and P. Hagedorn. Further properties of non-zereo sum differential games. In Journal of Optimization Theory and Applications, volume 3, pages 207–219, 1969. [4] J. V. Breakwell and P. Hagedorn. Point capture of two evaders in succession. In Journal of Optimization Theory and Applications, volume 27, pages 89–97, January 1979. [5] M. Cagalj, S. Capcun, and J. P. Hubaux. Wormhole-based anti-jamming techniques in sensor networks. In IEEE transactions on Mobile Computing, volume 6, pages 100–114, January 2007. [6] L. Chen. On selfish and malicious behaviours in wireless networks - a non-cooperative game theoretic approach. PhD thesis, Ecole Nationale Superieure des Telecommunications, Paris, October 2008. [7] X. C. Ding, A. Rahmani, and M. Egerstedt. Optimal multi-uav convoy protection. In Conference on Robot Communication and Configuration, volume 9, pages 1–6, April 2009. [8] D.M.Stipanović, S.Shankaran, and C. Tomlin. Strategies for agents in multi-player pursuit-evasion games. In Eleventh International Symposium on Dynamic Games and Applications, 2008. [9] P. Hagedorn and J. V. Breakwell. A differential game of approach with two pursuers and one evader. In Journal of Optimization Theory and Applications, volume 18, pages 15–29, January 1976. [10] R. Isaacs. Differential Games. Wiley, New York, 1965. [11] A. Y. Levchenkov and A. G. Pashkov. Differential game of optimal approach of two inertial pursuers to a noninertial evader. In Journal of Optimization Theory and Applications, volume 65, pages 501–518, June 1990. [12] J. Lewin. Differential Games: Theory and Methods for Solving Game Problems with Singular Surfaces. Springer-Verlag, London, 1994. [13] A. A. Melikyan. Generalized Characteristics of First Order PDEs: Applications in Optimal Control and Differential Games. Applications of Mathematics, 2000. [14] I. M. Mitchell and C. J. Tomlin. Overapproximating reachable sets by hamilton-jacobi projections. In Journal of Scientific Computing, volume 19, pages 323–346, 2003. [15] G. Noubir and G. Lin. Low power denial of service attacks in data wireless lans and countermeasures. 2003.

[16] P. Papadimitratos and Z. J. Haas. Secure routing for mobile ad hoc networks. January 2002. [17] A. G. Pashkov and S. D. Terekhov. A differential game of approach with two pursuers and one evader. In Journal of Optimization Theory and Applications, volume 55, pages 303–311, July 1987. [18] M. Pavone, K. Savla, and E. Frazzoli. Sharing the load: Mobile robotic networks in dynamic environments. IEEE Robotics and Automation Magazine, 16:52–61, 2009. [19] R. A. Poisel. Modern Communication Jamming Principles and Techniques. Artech, Massachussets, 2004. [20] J. J. Proakis and M. Salehi. Digital Communications. McGraw-Hill, 2007. [21] T. Samad, J. S. Bay, and D. Godbole. Network-centric systems for military operations in urban terrain: The role of UAVs. Proceedings of the IEEE, 95(1):92–107, 2007. [22] S. Shankaran, D. Stipanović, and C. Tomlin. Collision avoidance strategies for a three player game. In International Society of Dynamic Games, Wroclaw, Poland, July 2008. [23] D. M. Stipanović, I. Hwang, and C. J. Tomlin. Computation of an overapproximation of the backward reachable set using subsystem level set functions. In Dynamics of Continuous, Discrete and Impulsive systems, Series A: Mathematical Analysis, volume 11, pages 399–411, 2004. [24] D. M. Stipanović, A. A. Melikyan, and N. V. Hovakimyan. Some sufficient conditions for multi-player pursuit evasion games with continuous and discrete observaitions. In Annals of the International Society of Dynamic Games, volume 10, pages 133–145, 2009. [25] J. Tisdale, Z. Kim, and J. Hedrick. Autonomous UAV path planning and estimation. IEEE Robotics and Automation Magazine, 16:35–42, 2009. [26] E. M. Vaisbord and V. I. Zhukovskiy. Introduction to multi-player differential games and their applications. Gordon and Breach, New York, 1988. [27] A. D. Wood, J. A. Stankovic, and S. H. Son. Jam: a jammed-area mapping service for sensor networks. In Proceedings of 24th IEEE RealTime Systems Symposium (RTSS 03), pages 286–297, December 2003. [28] A. D. Wood, J. A. Stankovic, and G. Zhou. Deejam: Defeating energyefficient jamming in IEEE 802.15.4 based wireless networks. In 4th Annual IEEE Conference on Sensor, Mesh and Ad Hoc Communications and Networks (SECON 07), pages 60–69, 2007. [29] X. Wu, T. Wood, W. Trappe, and Y. Zhang. Channel surfing and spatial retreats: defenses against wireless denial of service. In 3rd ACM workshop on Wireless security (WiSe 04), pages 80–89, 2004. [30] V. I. Zhukovskiy and M. E. Salukvadze. The Vector Valued Maxmin, volume 193. Academic Press, San Diego, 1994. [31] S. Bhattacharya and T. Bas¸ar. Differential game-theoretic approach for spatial jamming attack in a UAV communication network. In 14th International Symposium on Dynamic Games and Applications, Banff, CA, June 2010. [32] S. Bhattacharya and T. Bas¸ar. Game-theoretic analysis of an aerial jamming attack on a UAV communication network. In Proceedings of the American Control Conference, pages 818–823, Baltimore, MD, June 2010. [33] S. Bhattacharya and T. Bas¸ar. Graph-theoretic approach to connectivity maintenance in mobile networks in the presence of a jammer. In Proceedings of the IEEE Conference on Decision and Control, Atlanta, GA, Dec. 2010. [34] S. Bhattacharya and T. Bas¸ar. Optimal strategies to evade jamming in heterogeneous mobile networks. In Proceedings of the Workshop on Search and Pursuit-Evasion, Anchorage, AK, 2010. [35] H. Choset, K. Lynch, S. Hutchinson, G. Kantor, W. Burgard, L. Kavraki, and S. Thrun. Principles of Robot Motion: Theory, Algorithms, and Implementations (Intelligent robotics and autonomous agents). The MIT Press, Cambridge, MA, 2005. [36] S. Gorman, Y. J. Dreazen, and A. Cole. Insurgents hack U.S. drones, December 2009. http://online.wsj.com/article/SB126102247889095011.html. [37] D. McCullagh. Predator drones hacked in Iraq operations, December 2009. Available at http://news.cnet.com/301-1009_3-10417247-83.html. [38] V. I. Arnold. Geometric Method in the Theory of Ordinary Differential Equations. Springer-Verlag, New York, 1983. [39] N. Biggs. Algebraic Graph Theory. Cambridge University Press, Cambridge, U.K., 1993.

[40] M. Mesbahi. On state-dependent dynamic graphs and their controllability properties. In IEEE Transactions on Automatic Control, volume 50, pages 387–392, 2005. [41] R. Olfati-Saber and R. M. Murray. Consensus problems in networks of agents with switching topology and time delay. In IEEE Transactions on Automatic Control, pages 1520–1533, 2004. [42] J. A. Fax and R. M. Murray. Information flow and cooperative control of vehicle formations. In IEEE Transactions on Automatic Control, volume 9, pages 1465–1474, 2004. [43] A. J. H. Tanner and G. Pappas. Flocking in fixed and switching networks. In IEEE Transactions on Automatic Control, volume 5, pages 863–868, May 2007. [44] P. B. Sujit and R. Beard. Multiple UAV path planning using anytime algorithms. In Proceedings of American Control Conference, pages 2978– 2983, St. Louis, Missouri, June 2009. [45] J. P. Laumond, S. Sekhavat, and F. Lamiraux. Guidelines in Nonholonomic motion planning for Mobile Robots. Springer, 1998. [46] S. Bhattacharya, A. Khanafer, and T. Bas¸ar. Power allocation in team jamming games in wireless ad hoc networks. In Proceedings of the 4th International ICST Workshop on Game Theory in Communication Networks (Gamecomm), 2011. http://arxiv4.library.cornell.edu/abs/1101.6030v1. [47] A. Khanafer, S. Bhattacharya, and T. Bas¸ar. Adaptive resource allocation in jamming teams using game theory*. In Proceedings of the 7th International Workshop on Resource Allocation and Cooperation in Wireless Networks(RAWNET), 2011. http://arxiv4.library.cornell.edu/abs/1101.6030v1. [48] T. Bas¸ar and P. Bernhard, H-infinity Optimal Control and Related Minimax Design Problems: A Dynamic Game Approach. Boston, MA: Birkhauser, August 1995. [49] M. Wei, G. Chen, J. B. Cruz, L. Hyanes, K. Pham and E. Blasch. Multipursuer multi-evader pursuit-evasion games with jamming confrontation In Aerospace Computing, Information, and Communication, volume 4, pages 693–706, March 2007. [50] D. Li, J. B. Cruz, G. Chen, C. Kwan and M-H. Chang. A heirarchical approach to multi-player pursuit-evasion differential games In Proceedings of the IEEE CDC-ECC, pages 5674 - 5679, Seville, Spain, Dec. 2005. [51] D. Li, J. B. Cruz and C. Schumacher. Stochastic multi-player pursuitevasion differential games International Journal of Robust and Nonlinear Control, volume 18, pages 218-247, May 2007.