Assessing the impact of deterrence on aviation checked baggage ...

Int. J. Risk Assessment and Management, Vol. 5, No. 1, 2005

Assessing the impact of deterrence on aviation checked baggage screening strategies Sheldon H. Jacobson* and Tamana Karnani Simulation and Optimisation Laboratory, Department of Mechanical and Industrial Engineering, 1206 West Green Street (mc-244), University of Illinois, Urbana, Illinois 61801-2906, USA E-mail: [email protected] E-mail: [email protected] *Corresponding author

John E. Kobza Department of Industrial Engineering, Texas Tech University, Box 43061, Lubbock, TX 79409-3061, USA E-mail: [email protected] Abstract: This paper analyses checked baggage screening strategies that incorporate the effects of deterrence on explosive detection systems (EDSs) deployed at airports. Cost models for these strategies are presented that incorporate the cost of purchasing, operating, and maintaining an EDS, the number of checked bags available to be screened, and the numbers of selectees and non-selectees checked bags actually screened over a one-year period. The model also includes the effect of deterrence on the level of threat at an airport. The cost models provide a quantitative tool to assess the strategy of 100% screening of all checked bags, as set forth by the USA Aviation and Transportation Security Act. Comparing the expected direct cost per expected prevented attack to the expected cost of an aviation terrorist incident provides an indication of the cost effectiveness of 100% checked bag screening. Keywords: aviation security; checked baggage screening; cost models; deterrence. Reference to this paper should be made as follows: Jacobson, S.H., Karnani, T. and Kobza, J.E. (2005) ‘Assessing the impact of deterrence on aviation checked baggage screening strategies’, Int. J. Risk Assessment and Management, Vol. 5, No. 1, pp.1–15. Biographical notes: Sheldon H. Jacobson is a Professor, Willett Faculty Scholar, and Director of the Simulation and Optimisation Laboratory in the Department of Mechanical and Industrial Engineering at the University of Illinois. He has a BSc and MSc (in Mathematics) from McGill University, and a MS and PhD (in Operations Research) from Cornell University. He (with John Kobza) was awarded the 2002 Aviation Security Research Award by Aviation Security International, and the 2003 Best Paper Award in IIE Transactions Focused Issues on Operations Engineering. He was also awarded a 2003 Guggenheim Fellowship for his research in the area of aviation security.

Copyright © 2005 Inderscience Enterprises Ltd.

1

2

S.H. Jacobson, T. Karnani and J.E. Kobza Tamana Karnani is an industrial engineer specialising in lean manufacturing and six sigma at Guidant Corporation in St. Paul, Minnesota. She has a BS and MS (in Industrial Engineering) from the University of Illinois at Urbana-Champaign. She was awarded Second Place in the 2004 IIE Graduate Research (Master's Thesis) Award Competition. She has conducted research in the application of operations modelling and analysis to address issues in the areas of paediatric immunisation economics and aviation security system design. John E. Kobza is an associate professor in the Department of Industrial Engineering at Texas Tech University. He has a BSEE from Washington State University, a MSEE from Clemson University, and a PhD in Industrial and Systems Engineering from Virginia Tech. His research interests include modelling, analysing and designing systems involving uncertainty and risk, such as security systems, manufacturing systems and communication networks. He is a member of Sigma Xi, Alpha Pi Mu, INFORMS, IIE, and IEEE.

1

Introduction

In the wake of the terrorist attacks on September 11, 2001, the USA Congress enacted the Aviation and Transportation Security Act (ATSA). To comply with ATSA, the Transportation Security Administration (TSA) developed a strategy to acquire and deploy government certified explosive detection systems (EDSs) in airports throughout the nation. The TSA estimated that over 2,500 EDS devices would be needed to ensure one hundred percent screening of all checked bags, at a purchase and deployment cost of over $2.5B (Mead, 2002a). EDSs are considered as essential components of aviation security strategies designed to prevent and deter terrorist threats against the national airspace system. Therefore, the TSA needs to evaluate the cost effectiveness, maturity, and efficiency of current and next-generation equipment to maximise security with the available funds (Mead, 2002b). Virta et al. (2003) address cost-benefit trade off questions concerning the use of EDSs. Since 1998, the FAA has used a computer-aided passenger pre-screening system (CAPPS) to determine those passengers (and their corresponding checked bags) that cannot be cleared from posing a potential security risk (Mead, 2002a; Mead, 2002b). In particular, ‘selectees’ are the passengers that cannot be cleared by CAPPS, while ‘non-selectees’ are the ones who can be cleared by CAPPS (Virta et al., 2003). Based on security devices and procedures in place prior to September 11, 2001, Virta et al. (2003) develop and analyse a cost model that quantifies the expected annual cost associated with screening different combinations of selectee and non-selectee checked bags. They conclude that as excess EDS baggage screening capacity is used to screen non-selectee checked bags, the expected annual cost increases. They also report that the marginal increase in security per security dollar spent, is significantly lower when non-selectee checked bags are screened, than when only selectee checked bags are screened. Their analysis suggests that the added value of screening non-selectee checked bags is minimal, and that the most cost-effective approach is to focus security resources on selectee checked bags.

Assessing the impact of deterrence

3

This paper extends the cost model (Virta et al., 2003) to analyse the use of EDSs within the framework of the Aviation and Transportation Security Act, as well as to incorporate the impact of deterrence. Deterrence, as defined here, is the effect that screening a greater proportion of checked bags has, on the level of threat. If successful, deterrence reduces the level of threat within the system. This paper analyses and compares scenarios for screening different combinations of selectee and non-selectee checked bags, and evaluates various cost measures. These scenarios are designed to capture relationships between the cost and effectiveness of EDS screening, from the perspective of the TSA and the airports. Scenario A focuses on only screening selectee checked bags, hence non-selectee checked bags are not screened. Scenario B focuses on screening all checked bags (selectee and non-selectee). These two scenarios correspond to the baggage screening policies before and after September 11, 2001. They also represent two extreme situations that provide a boundary of analysis for intermediate scenarios that may be implemented in the future. This paper is organised as follows: Section 2 provides a description of the data needed to describe the cost models, including values and ranges for the cost model parameters. Note that since some of the data required in the model is security sensitive, it cannot be reported in the public domain. Therefore, fictitious values have been provided that protect the sensitivity of the information while preserving the general form and structure of the model and its application. Section 3 presents the cost models, while Section 4 reports the analysis and results for different scenarios with these models. Section 5 provides concluding comments.

2

Data needs

Cost models have been applied to a variety of areas including power generation and transmission (Yuandong and Hobbs, 1998), maintenance and reliability (Hongzhou and Pham, 1996; Veers, 1996), transportation (Carey and Kwiecinski, 1995), software (Pham, 1996), and inventory models (Chiu, 1995). Models applied to computer security systems are the most similar to those proposed here, in that they include the costs of operations and decision outcomes (Murray and Farrell, 1993; Ekenberg, Oberoi and Ocri, 2002; Wenke et al., 2002). The data needed to support the cost model used in this analysis is represented as parameters that can be classified into four groups: •

probability parameters

•

cost parameters

•

time parameters

•

volume parameters.

In these descriptions, the letters A and NA denote ‘Alarm’ and ‘No Alarm’, respectively, while the letters T and NT denote ‘Threat’ and ‘No Threat’, respectively. 1

Probability Parameters PFA = PA|NT = probability that an EDS falsely indicates a threat (false alarm) in a checked bag

4

S.H. Jacobson, T. Karnani and J.E. Kobza PTC = PNA|NT = 1– PFA = probability of a true clear for a checked bag PTA = PA|T = probability that an EDS correctly detects a threat (true alarm) in a checked bag PFC = PNA|T = 1– PTA = probability of a false clear for a checked bag PT = probability that a checked bag contains a threat.

The first four probability parameters are random variables obtained based on the testing and evaluation of a baggage screening security device. Such testing is required before a device can gain federal approval. Based on the perceived threat level, the probability that a checked bag contains a threat is assessed by personnel within the TSA. This value is considered highly sensitive and may change based on national and/or international situations or intelligence information. 2

Cost Parameters CFA = cost of a false alarm = cost of falsely indicating a threat in a checked bag CFC = cost of a false clear = cost of not detecting a threat in a checked bag CTA = cost of a true alarm = cost of correctly detecting a threat in a checked bag CTC = cost of a true clear = cost of correctly indicating a non-threat in a checked bag CF = purchase price and installation cost (fixed) of an EDS CO = annual maintenance costs (operational) for an EDS, including annual lease expenses. This cost is independent of the number of checked bags screened CI = cost of operating an EDS, per checked bag inspected.

The cost parameters are random variables. Expected values for the first four cost parameters were obtained based on information collected and analysed by the TSA, as well as from Butler and Poole (2002). Expected values for the next two cost parameters are available from the TSA and the baggage screening security device manufacturer. Note that in most airports, baggage screening security devices are not operated twenty-four hours a day. Therefore, ample non-operational time is available for maintenance so breakdowns rarely occur. The expected value for the last cost parameter was obtained from the TSA based on salaries paid to the federal employees, hired to operate the baggage screening security devices. Note that the maintenance and operational cost per year for an EDS, CO, is assumed to be a fixed cost, independent of the number of checked bags screened, while CI is the cost of screening each checked bag. These costs represent the costs borne by the TSA (government) and the airports, not the passengers flying. The cost of flight delays due to baggage screening that results in missed connections, and the cost associated with the time for early check-in requirements, which are very difficult to measure, are not included in this analysis. 3

Time Parameters N1 = number of years of useful life for an EDS before technical obsolescence N2 = number of years of useful life for an EDS before it wears out due to being in operation.


5

These two time parameters are deterministic, obtained based on information available from the baggage screening security device manufacturers and the experience of TSA personnel. By definition, both N1 and N2 do not depend on the number of checked bags screened, but rather, depend on the speed of technological advances (for N1) and the time that an EDS is in operation (for N2). 4

Volume Parameters SC = number of checked bags an EDS can screen before wearing out due to being used SCAP = number of checked bags an EDS can process per year (i.e., the baggage screening capacity) S1A = number of selectee checked bags received per year at the airport S2A = number of non-selectee checked bags received per year at the airport S1 = number of selectee checked bags actually screened per year by an EDS at the airport S2 = number of non-selectee checked bags actually screened per year by an EDS at the airport.

The first two volume parameters are deterministic and are available from the baggage screening security device manufacturer. The last four volume parameters are random variables. Their expected values were obtained based on discussions with the TSA, and are a function of the airport under study. The model parameters, their types, and their expected values are reported in Table 1. The values for PT, CFC, CFA, and CI are the only parameters that have been modified from the analysis reported (Virta et al., 2003). Note that because of the sensitive nature of the data, all the values reported in Table 1 have been adjusted from the true values that are available from the TSA. The expected cost of a true alarm, CTA, is given as $1M, while the expected cost of a false alarm, CFA, is given as $9. The cost of a true alarm includes the cost of closing an airport terminal, as well as calling in bomb squads and other law enforcement officers. The cost of a false alarm may include such costs. However, in most such instances, false alarms are often quickly resolved through inexpensive additional screening procedures, which explain the wide disparity between the expected costs of a true alarm and a false alarm.

3

Cost models

This section describes the cost models used to analyse the cost-effectiveness of 100% checked baggage screening. The costs depend on PT, the probability that a checked bag contains a threat. However, as additional checked bags are screened, deterrence should reduce PT. To account for this effect, PT is quantified through a multiplier that depends on δ = (S1 + S2) / (S1A + S2A), the fraction of all checked bags that are screened, where 0 ≤ δ ≤ 1. The resulting probability of a threat incorporating the effect of deterrence is modelled as PTDET = PT (1 – ρδη),

(1)

6

S.H. Jacobson, T. Karnani and J.E. Kobza

where η and ρ are deterrence parameters. In particular, η is a deterrence exponent that captures how quickly (i.e., the speed) deterrence reduces the threat level, while ρ is a deterrence multiplier that captures the maximum possible impact of deterrence. Without loss of generality, assume that η > 0 and 0 ≤ ρ ≤ 1, since this ensures that deterrence will never increase the probability of a threat. Table 1

Cost model parameters

Parameters

Expected Value

Parameter Type

PFA

0.30

Random Variable

PFC

0.05

Random Variable

β

155

PT

Constant –9

5.0005 × 10

Random Variable

CFC

$1.4B

Random Variable

CTA

$1M

Random Variable

CTC

$0

Random Variable

CF

$1M

Random Variable

CFA

$9

Random Variable

CO

$125,000

Random Variable

CI

$0.525

Random Variable

N1

10 years

Constant

N2

10 years

Constant

S2A

2,500,000 Bags

Random Variable

SC

10,000,000 Bags

Constant

SCAP

400,000 Bags

Constant

The actual values for δ and ρ may be very difficult to obtain, though intelligence information within the TSA can be used to provide reasonable ranges for their values (i.e., such values are a function of intelligence information that is not readily available in the public domain). Since this information is security sensitive and cannot be publicly disseminated, a wide range of values for δ and ρ will be considered, with appropriate interpretations provided they do not compromise the sensitivity of such data. The form of (1) provides a general expression that captures a wide variety of deterrence effects on the probability of a threat. The deterrence parameter ρ quantifies the maximum reduction in PT, while η determines the marginal rate at which PT is reduced as δ increases. For example, ρ = 1 corresponds to the case when deterrence can eliminate 100% of the risk in the system, as measured by the probability of a threat, while when ρ < 1, deterrence eliminates 100ρ% of the risk in the system. From (1), the greatest impact of deterrence on the PT occurs for values of ρ close to one. Note that the special case when ρ is close to zero corresponds to no deterrence effect on PT (i.e., PT is unchanged for all values of δ); this corresponds to the analysis reported in Virta et al. (2003). Since it is not clear what level of deterrence can be achieved with 100% checked baggage screening, a range of values for ρ between zero (corresponding to 0% deterrence) and one (corresponding to 100% deterrence), in steps of 0.10 (corresponding to 10% deterrence increments), will be considered.


7

Figure 1 shows the effect of η on the probability of a threat using (1). If η = 1, then there is a linear effect of δ reducing the probability of a threat due to deterrence. If η > 1, then this effect is convex in δ, while if η < 1, this effect is concave in δ. Note that when all checked bags are screened (i.e., δ = 1), the deterrence expression is no longer a function of η (i.e., PTDET = PT (1 – ρ)). However, when only selectee checked bags are screened (or all selectee checked bags and some fraction of non-selectee checked bags), δ < 1 and the value for η directly impacts the analysis. Note that since individuals from terrorist organisations are likely to feel targeted, they are likely to perceive themselves as part of the selectee population. Since selectee bags are screened before non-selectee bags, this implies η < 1. Therefore, unless otherwise noted, we assume η < 1. Figure 1

Effect of η on the probability of threat

Given a value for the CAPPS multiplier, β, the values for PT|SDET and PT|NSDET (where S and NS denote ‘Selectee’ and ‘Non-Selectee’, respectively) can be computed. The CAPPS multiplier, β, quantifies the difference in risk between the selectee and non-selectee populations. In particular, PT|S = βPT|NS (i.e., a selectee is β times more likely to be a threat than a non-selectee). Using this relationship and (1), PT|SDET = PTDET [1 + ((β–1)S2A) / (βS1A + S2A)],

(2)

PT|NSDET = PTDET [1 − ((β–1)S1A) / (βS1A + S2A)].

(3)

Without loss of generality, assume that β > 1, since CAPPS is designed to screen out passengers who are less likely to be threats to the system (i.e., non-selectee passengers). To determine the effective lifetime of an EDS, three factors must be considered. First, regardless of usage, an EDS will become technologically obsolete and require replacement in N1 years. Second, regardless of the number of checked bags processed, an

8


EDS will wear out in N2 years due to bearing erosion and other factors resulting from it, being turned on for any length of time (i.e., an EDS wears out from being operational). Third, regardless of the length of time an EDS is in use, it can only process SC checked bags before total replacement of the unit is required (i.e., an EDS wears out from being used to screen checked bags). The number of checked bags an EDS can process before wearing out is estimated to be SC = 10,000,000 bags. The number of checked bags that an EDS can screen in a given year is estimated to be SCAP = 400,000 checked bags, which is obtained (approximately) by multiplying six peak hours per day of operation (assuming a hub and spoke system where flight schedules are designed such that flights depart during peak hours) by 185 checked bags per peak hour (Butler and Poole, 2002) by 360 days. The number of EDSs needed to handle the number of checked bags at an airport is M = (S1A + S2A) / SCAP or to handle only selectee checked bags at an airport is M = S1A / SCAP. Using this information, the effective lifetime of an EDS is Neff = min{N1, N2, M*SC / (S1+S2)}.

(4)

The cost model represents the annual cost of operating EDSs. It includes both direct and indirect costs. Direct costs include the annual purchasing, maintaining, and operating costs of the baggage screening security device, as well as direct costs associated with processing the number of true clears, addressing true alarms (including the closing of an airport terminal), and resolving false alarms (including, in extreme instances, the possible need to call in law enforcement officers and bomb squads). Indirect costs include the costs associated with false clears or not screening checked bags that result in a threat incident. Since such incidents are typically rare events and the time between them is long, this cost may only need to be paid once every ten or twenty years. However, the cost model takes this into account by evenly spreading its cost over each year. The cost model is as follows: Cost = M (CF / Neff) + M CO + CI (S1 + S2) + CFAPFA [(1 – PT|SDET)S1 + (1 – PT|NSDET)S2] + CFCPFC[PT|SDET S1 + PT|NSDET S2] + CTA(1 – PFC) [PT|SDET S1 + PT|NSDET S2] + CTC(1 – PFA)[(1 – PT|SDET)S1 + (1 – PT|NSDET)S2] + CFC[PT|SDET (S1A – S1) + PT|NSDET (S2A – S2)],

(5)

where the, •

first component represents the annual (direct) cost of purchasing the EDSs

•

second component represents the annual (direct) cost of operating / maintaining the EDSs

•

third component represents the annual (direct) inspection cost for checked bags screened by the EDSs

•

fourth component represents the annual (direct) cost of false alarms

•

fifth component represents the annual (indirect) cost of false clears

•

sixth component represents the annual (direct) cost of true alarms

•

seventh component represents the annual (direct) cost of true clears

•

eighth component represents the annual (indirect) cost associated with not using an EDS to screen either selectee or non-selectee checked bags (resulting in the equivalent of a false clear cost that potentially could have been prevented by screening).


9

The cost model (5) measures the annual cost of using the EDSs to screen S1 selectee checked bags and S2 non-selectee checked bags. Note that if S2 is set to zero, then (5) reduces to the case of screening only selectee checked bags. The expected annual cost is computed using the expected value for the parameters in the cost model. Since the cost parameters and the probability parameters in (5) are independent, the expected cost can be computed using the expectation of the parameters. The only exception to this occurs for the term M (CF / Neff). However, since this term is constant across all the scenarios, it does not affect the conclusions obtained. The cost of a false clear, CFC, is both large and difficult to estimate. Therefore, a second cost model is formulated that only considers direct costs (i.e., the resulting model uses all but the fifth and eighth terms in (5)). The resulting direct cost model represents the annual direct cost of operating EDSs, CostD = M (CF / Neff) + M CO + CI (S1 + S2) + CFAPFA [(1 – PT|SDET)S1 + (1 – PT|NSDET)S2] + CTA(1 – PFC) [PT|SDET S1 + PT|NSDET S2] + CTC(1 – PFA)[(1 – PT|SDET)S1 + (1 – PT|NSDET)S2].

(6)

A third measure that can be used to quantify the effectiveness of a 100% checked baggage screening strategy is the expected number of successful attacks. The fourth and seventh terms in (5) divided by CFC correspond to the number of successful attacks, SA = PFC[PT|SDET S1 + PT|NSDET S2] + [PT|SDET (S1A – S1)+PT|NSDET (S2A – S2)].

(7)

Note that as ρ decreases from one, the value for SA will also increase since a lower level of deterrence results in more attacks on the system. All these measures will be used to assess the cost and benefit associated with a 100% checked baggage screening strategy. The cost model was implemented in an Excel spreadsheet. Sensitivity analysis was performed using the software package @Risk. The parameters in Table 1 were varied uniformly over a range of +/– 10% of the value given in Table 1, except for CTC, which was varied uniformly from 0 to $10. In addition, S1 was varied uniformly from 137,250 to 167,750, ρ was varied uniformly from 0.8 to 1.0, and η was varied uniformly from 1 to 100. The expected number of successful attacks and the expected direct cost for Scenarios A and B were monitored as outputs. The model was executed 100,000 times and the correlation coefficient between each input and output was computed by @Risk. Table 2 presents the coefficients with a magnitude greater than 0.1. Several observations can be made from this analysis. First, the expected direct cost responds similarly for both the scenarios. Moreover, the expected direct cost is most sensitive to the cost of a true clear, the number of passengers screened, the cost of a false alarm and the probability of a false alarm. Recall that direct costs exclude those associated with detected or undetected threats. Therefore, the direct costs reflect the operational costs associated with screening baggage and resolving false alarms. The impact of threats on the system is reflected in the expected number of successful attacks. For both scenarios, this measure is highly correlated with the threat probability and the number of non-selectee passengers. Scenario B is more sensitive to the probability of a false clear, since every bag (and thus, every threat) is screened, with a mistake resulting in a successful attack. For Scenario A, only the selectee bags are screened, so not all threats are screened. Scenario A is much more sensitive to the value of β, which quantifies the system’s ability to concentrate threats in the selectee

10


population. Therefore, if β decreases, more threats appear in the non-selectee population, which results in a greater number of successful attacks. Table 2

Correlation coefficients

Scenario A: Expected Direct Cost Parameter CTC

Correlation Coefficient

Scenario B: Expected Direct Cost Parameter

0.973

Correlation Coefficient

Scenario A: Expected Number of Successful Attacks Parameter

CTC

0.977

S2A

Correlation Coefficient 0.755

Scenario B: Expected Number of Successful Attacks Parameter ρ

Correlation Coefficient –0.978

S1

0.203

S2A

0.192

PT

0.496

PT

0.105

CFA

0.075

CFA

0.075

β

–0.291

PFC

0.105

PFA

0.034

PFA

0.033

S1

–0.262

S2A

0.100

PFC

0.159

Note that ρ is the most critical parameter for the expected number of successful attacks under Scenario B. When 100% screening is implemented, δ = 1, and ρ determines the maximum reduction in the threat probability. Although ρ and η do not appear in Table 2 for the other performance measures, they do affect the system performance, though not as significantly as the other parameters studied.

4

Cost model analysis

The cost models and successful attack measure introduced and discussed in Section 3 provide tools for comparing the relative effectiveness of different baggage screening strategies. They can also be used to measure the effect of deterrence on the expected annual cost. By distinguishing between selectee and non-selectee checked bags, two different extreme baggage screening scenarios are studied. Scenario A: Use EDSs to screen only selectee checked bags. Scenario B: Use EDSs to screen all checked bags (selectee and non-selectee). Scenario A reflects the pre-September 11, 2001 baggage screening strategy, while Scenario B reflects a post-September 11, 2001 analysis. Both these scenarios incorporate sufficient EDS baggage screening capacity to screen all checked bags that require screening. The expected number of successful attacks as a function of the threat reduction due to deterrence is reported and analysed for Scenarios A and B. To compare these scenarios, the difference in expected annual direct cost divided by the difference in expected number of successful attacks, as a function of the threat reduction due to deterrence, is also reported and analysed.

4.1 Scenarios A and B To compare the cost and benefit of screening only selectee checked bags (Scenario A) versus screening all checked bags (Scenario B), the impact of deterrence on the expected number of successful attacks was computed for levels of deterrence


11

ρ = j/10, j = 1,2,…,10, which correspond to a 100ρ% reduction in the probability of threat due to deterrence. Assume that the total number of checked bags available to be screened is 2,652,500, with the number of selectee checked bags set at its expected value S1A = 152,500 and the number of non-selectee checked bags S2A = 2,500,000. This corresponds to a selectee rate of approximately PS = 0.057. For Scenario A, assuming no deterrence, the expected direct cost (6) is approximately $1.35M, which translates into a $8.85 expected direct cost per selectee checked bag or $0.51 per checked bag, resulting in approximately 0.0185 expected successful attacks per 2.6525M checked bags (or 6.97 successful attacks per year, assuming one billion checked bags per year; this value is unrealistically high since the data being used has been modified for dissemination, and there is a baseline level of deterrence that is likely to be always present (i.e., 0% deterrence never exists)). Note that the expected direct cost does not change significantly as the level of deterrence increases up to 100%, since the direct cost model is not significantly affected by changes in the deterred threat probability value. However, the expected number of successful attacks decreases as the level of deterrence increases. Moreover, this decrease is most significant for smaller values of η. These results are reported in Figure 2 (where η appears in the figure as ETA). Figure 2

Expected number of successful attacks versus % of deterrence

For Scenario B, assuming no deterrence, the expected direct cost is approximately $21.59M, which translates into a $8.14 expected direct cost per checked bag, with approximately 0.0066 expected successful attacks per 2.6525M checked bags (or 2.49 successful attacks per year, assuming one billion checked bags per year). The lower number of expected successful attacks results from additional threats being detected in the previously unscreened non-selectee baggage. The increased costs are due to the increased false alarm costs associated with the higher screening volumes. As in Scenario A, this expected direct cost is not significantly affected by deterrence. However, as the threat deterrence effect approaches 100%, the expected number of successful attacks per 2.6M checked bags screened approaches zero (see Figure 2). Therefore, if 100%

12


deterrence can be achieved, then 100% baggage screening, at an expected direct cost of $8.09 per checked bag, may provide a reasonable checked baggage screening strategy. However, the key question is: what is being purchased for this cost? To address this question, Scenarios A and B can be compared for different levels of deterrence using the expression (8) ED = (E[CostBD | ρ] − E[CostAD | ρ]) / (E[SAA | ρ] − E[SAB | ρ]), which represents the difference between the expected direct costs for Scenarios A and B (see (6)) divided by the difference between the expected number of successful attacks for Scenarios A and B (see (7)). The numerator in (8) represents the additional direct cost of 100% screening, beyond screening only selectee bags. The denominator in (8) represents the number of previously successful attacks (when only selectee bags have been screened) that are prevented by being detected as a result of the additional screening. Therefore, (8) quantifies the difference between the two checked baggage screening strategies based on the cost of preventing attacks and the number of attacks prevented. To measure the impact of deterrence, (8) is computed for ρ = 0.0, 0.1, 0.2, …, 1.0 and η = 1.0, 0.50, 0.25, 0.10, 0.05, 0.01. These results are presented in Figure 3 (again, η appears in the figure as ETA). Note that values for η close to zero correspond to deterrence environments where screening a small fraction of checked bags results in a large deterrence effect. Different values for η have no impact on (6) and (7) for Scenario B (because δ = 1 with 100% checked baggage screening). However, they do impact (6) and (7) for Scenario A (with only selectee checked bags screened). Moreover, (8) is most sensitive to values of ρ close to one and small values of η, which corresponds to the cases when the deterrence effect is greatest. Figure 3 shows the effect of deterrence on the cost of preventing attacks by depicting the marginal cost of detecting a threat in the non-selectee population using Scenario B beyond the cost of Scenario A. If there is no deterrent effect (i.e., ρ = 0) this cost is $1.7 billion, that is, the additional cost of implementing Scenario B spread over the additional number of prevented attacks is $1.7 billion per prevented attack. If ρ = 1 and η = 1.0, deterrence can eliminate the threat probability, but only if the whole population is screened. Under Scenario A, only selectees are screened so δ = 0.0575 and little deterrent effect is realised. Under Scenario B, 100% screening reduces the threat probability to zero. Since Scenario B is more effective at preventing attack, the marginal cost is lower. However, if ρ = 1 and η is small, the deterrent effect is quickly realised. As a result, screening only selectees (Scenario A) dramatically reduces the threat probability and, thus, the number of threats in the non-selectee population. The additional cost of screening non-selectee passengers under Scenario B is now spread over a significantly fewer number of prevented threats, resulting in a much higher marginal cost ($38 billion for η = 0.01, as seen in Figure 3). Note that the values in Figure 3 can be compared to the cost of a false clear, CFC, to assess whether a 100% checked baggage screening strategy provides a cost-effective aviation security policy. The value for η, such that the expected cost is the same for both 0% and 100% deterrence, is approximately η = 0.1095. Note that for this value of η, 100% deterrence effect (ρ = 1.0) results in a 73.14% reduction in the threat level for 100% screening compared to screening only selectee checked bags.

Assessing the impact of deterrence Figure 3

5

13

Expected direct cost per expected prevented attack: Scenario B relative to Scenario A

Conclusions

This report analyses checked baggage screening strategies for EDSs deployed at airports. Cost models are presented, which incorporate the cost of purchasing, operating, and maintaining an EDS, the number of checked bags available to be screened, and the actual numbers of selectee and non-selectee checked bags screened. The models also incorporate the effect of deterrence. The results quantify the direct cost of 100% checked baggage screening, as well as the security gained through such a strategy. The key conclusion obtained is that 100% checked baggage screening may provide an effective deterrence against terrorist activities, depending on the perceived cost of a terrorist incident and the deterrence effect achieved by screening only selectee checked bags. The cost effectiveness of 100% screening of all checked bags compared to screening only checked selectee bags, depends on how quickly an increase in the number of checked bags screened reduces the threat level (i.e., the deterrence effect). The expected direct cost per expected prevented attack to the expected cost of an aviation terrorist incident provides one measure for the cost effectiveness of 100% checked bag screening. The cost models developed are scalable to any size airport. These costs are restricted to those borne by the TSA (government) and airports; they do not include the costs associated with military operations to deter threats against the air system. The model also focuses only on checked baggage and does not include costs associated with cargo and

14


passenger screening. Therefore, this analysis provides a conservative, lower bound analysis on the cost of preventing a single terrorist incident. Note that although the results of this analysis focus only on checked baggage screening, the models can also be used for checkpoint screening of passengers and screening of aircraft cargo. Moreover, other transportation security screening applications such as seaport cargo container screening, can also be analysed with these models. Work is in progress to extend these results to such domains, as well as to further analyse how these models can help identify ways to enhance security operations at a reasonable cost.

Acknowledgement This research has been supported in part by the National Science Foundation (DMI-0114046, DMI-0114499). The first author has also been supported in part by the Air Force Office of Scientific Research (FA9550-04-1-0110). The authors would also like to thank Mr. James Farrell, Dr. Anthony Fainberg, Ms. Theresa Hasty, Dr. Lyle Malotky, Mr. Michael McCormick and Dr. John J. Nestor of the Transportation Security Administration for their helpful advice and feedback on the results, data, and analysis presented in this paper. Lastly, the authors wish to thank Dr. Robert Batson and two anonymous referees for their helpful comments and feedback on an earlier version of the manuscript.

References Butler, V. and Poole Jr., R.W. (2002) ‘Rethinking checked baggage screening’, Reason Foundation, Policy Study 278, Los Angeles, CA, July 2002. Carey, M. and Kwiecinski, A. (1995) ‘Properties of expected costs and performance measures in stochastic models of scheduled transport’, European Journal of Operational Research, Vol. 83, No. 1, pp.182–199. Chiu, H. (1995) ‘An approximation to the continuous review inventory model with perishable items and lead times’, European Journal of Operational Research, Vol. 87, No. 1, pp.93–108. Ekenberg, L., Oberoi, S. and Ocri, I. (2002) ‘A cost model for managing information security hazards’, Computers & Security, Vol. 14, No. 8, pp.707–717. Hongzhou, W. and Pham, H. (1996) ‘Optimal maintenance policies for several imperfect repair models’, International Journal of Systems Science, Vol. 27, No. 6, pp.543–549. Mead, K.M. (2002a) Challenges Facing the TSA in Implementing the Aviation and Transportation Security Act, Report Number CC-2002–088, Office of Inspector General, Department of Transportation, Washington DC. Mead, K.M. (2002b) Key Issues Concerning Implementation of the Aviation and Transportation Security Act, Report Number CC-2002-098, Office of Inspector General, Department of Transportation, Washington DC. Murray, W.H. and Farrell, P. (1993) ‘Toward a model of security for a network of computers’, Computer Security Journal, Vol. 9, No. 1, pp.1–13. Pham, H. (1996) ‘A software cost model with imperfect Debugging, Random life cycle and penalty cost’, International Journal of Systems Science, Vol. 27, No. 5, pp.455–463. Veers, P.S. (1996) ‘Fatigue reliability of wind turbine fleets: the effect of uncertainty on projected costs’, Transactions of the ASME-Journal of Solar Energy Engineering, Vol. 118, No. 4, pp.222–227.


15

Virta, J.L., Jacobson, S.H. and Kobza, J.E. (2003) ‘Analyzing the cost of screening selectee and non-selectee baggage’, Risk Analysis, Vol. 23, No. 5, pp.897–908. Wenke, L., Wei, F., Miller, M., Stolfo, S. and Zadok, E. (2002) ‘Toward cost-sensitive modeling for intrusion detection and response’, Journal of Computer Security, Vol. 10, No. 1/2, pp.5–22. Yuandong, J. and Hobbs, B.F. (1998) ‘Including a DC network approximation in a multiarea probabilistic costing model’, IEEE Transactions on Power Systems, Vol. 13, No. 3, pp.1121–1127.