Cost-Benefit Analysis of Full-Body Scanners

Research paper from Mark Stewart and John Mueller:

The Transportation Security Administration (TSA) has been deploying Advanced Imaging Technologies (AIT) that are full-body scanners to inspect a passenger’s body for concealed weapons, explosives, and other prohibited items. The terrorist threat that AITs are primarily dedicated to is preventing the downing of a commercial airliner by an IED (Improvised Explosive Device) smuggled on board by a passenger. The cost of this technology will reach $1.2 billion per year by 2014. The paper develops a cost-benefit analysis of AITs for passenger screening at U.S. airports. The analysis considered threat probability, risk reduction, losses, and costs of security measures in the estimation of costs and benefits. Since there is uncertainty and variability of these parameters, three alternate probability (uncertainty) models were used to characterise risk reduction and losses. Economic losses were assumed to vary from $2-50 billion, and risk reduction from 5-10%. Monte-Carlo simulation methods were used to propagate these uncertainties in the calculation of benefits, and the minimum attack probability necessary for AITs to be cost-effective was calculated. It was found that, based on mean results, more than one attack every two years would need to originate from U.S. airports for AITs to pass a cost-benefit analysis. In other words, to be cost-effective, AITs every two years would have to disrupt more than one attack effort with body-borne explosives that otherwise would have been successful despite other security measures, terrorist incompetence and amateurishness, and the technical difficulties in setting off a bomb sufficiently destructive to down an airliner. The attack probability needs to exceed 160-330% per year to be 90% certain that AITs are cost-effective.

EDITED TO ADD (1/26): Response from one of the paper’s authors.

Tags: , , , , , , ,

Posted on January 20, 2011 at 1:39 PM • 39 Comments

Comments

Muhammad Naveed Khurshid • January 20, 2011 1:55 PM

According to paper,

Net Benefit=p(attack) ×C(loss)×ΔR−C(security).

If security cost 1.2 Billion by 2014 then according to above formula for Net benefit should be -ve or as close to zero as possible (zero is ideal, I believe).

The question is how will it be possible without proper analysis of every security threat… In eye of security officers in airport, every person who is in boundary of airport or aeroplane is a threat or can be a threat.

The cost of monitoring will be exponentially high as new security threats will arrive from time to time.

Steven Tylor • January 20, 2011 2:01 PM

“Disruption” would also include “prevention”. So if AITs scare off an attack effort that would otherwise have been executed successfully, it is just as effective.
And if AITs scare off an attack effort that would otherwise have been executed unsuccessfully but at a cost, that cost was saved as well.

But I agree, AITs suck.

Joseph R. Jones • January 20, 2011 2:04 PM

Interesting, but not really the point. If the negative outcome of a downed commercial jetliner was primarily that of property damage and cost then this would be a good way to approach the problem. However, the hard costs are irrelevant in the face of the human cost, which one has to admit is incalculable. Also, the cost of deployment goes way beyond the hard costs of buying, installing, and maintaining devices– it’s more about consumer inconvenience and preservation of rights.

More interesting would be to focus on the efficacy of these devices in deterring the classes of attacks studied relative to less invasive (and hence less objectionable) mechanisms like bomb detecting devices and animals. Cost is one thing to measure in this comparison, but it shouldn’t be at the top of the list.

Tim • January 20, 2011 2:35 PM

I would submit that human costs are irrelevant because, for the airlines, it does come down to what can they risk and still be profitable, no matter what the PR department says about how much they care.

If you’re willing to risk your life in the first place by getting on the plane then you don’t really believe the human cost of any potential loss is incalculable, otherwise you’d stay off the plane.

The more creative the TSA gets, the more creative the terrorists will get. Eventually some crazy bastard will hook up a pound of semtex to a cellphone detonator and have it surgically implanted in his stomach and ask for a seat right over the wings. How do you stop someone with that level of committment? You just can’t.

Fred P • January 20, 2011 3:08 PM

@Joseph R. Jones

Did you read the paper? They mention additional loss of life due to security measures (due to car travel vs. air travel), but they then ignore it in their cost analysis so as to tip the scales in the favor of this machines. Even doing so, the machines aren’t worth it.

kingsnake • January 20, 2011 3:12 PM

Tim, some Saudi already tried to kiester his bomb and off a Saudi prince. All he succeeded in doing was giving himself a fatal case of loose bowel syndrome. Why? Can’t cram enough explosives into a body cavity. Maybe a fat guy could, but then he’d also have more blubber to absorb the explosion of the interno-bomb, so it would seem to offset.

Greg • January 20, 2011 3:15 PM

It is worth considering the agency problem here too. Specifically, the TSA executives making the decision to use the AIT don’t make the global cost-benefit computation, but a local one about their own careers. Like with the silliness of removing shoes and throwing out water, TSA executives are worried that, if they remove the screening and there is an incident, they will be blamed. They are not making the calculation of time lost to travelers, radiation exposure to travelers, and cost to the taxpayers, but the calculation that this is the path that is safest for their careers.

Brandioch Conner • January 20, 2011 3:16 PM

@Steven Tylor
“So if AITs scare off an attack effort that would otherwise have been executed successfully, it is just as effective.”

So the terrorist with the motivation to DIE in the attack, who has built his bomb which would be sufficient to take out the aircraft and could not be stopped by other passengers or detected by any other means …

… is just going to give up because there’s a chance that the scanner might detect his bomb?

That’s a rather specific kind of bomb.

And a rather strange terrorist.

Mr. Gel-pack • January 20, 2011 4:09 PM

Disrupting the attacks into a cheap wheelie-bag claymore in the checkpoint or into bagclaim could cause as much loss of life and similar indirect, psychological and political effects with far less complication.

That’s what real terrorists do at real checkpoints.

Since there aren’t any terrorists currently diverted into the easier softer targets, the ones that TSA pretends they are protecting us from are just boogymen.

The armored cockpit door and non-cooperation stance obsoletes much of what TSA is ostensibly trying to protect us from–“Hey let me hijack the plane or I’ll light my exploding underwear!” is just silly.

Of course since the terrorists can cause us to overreact and double the security-theater-tax every time they pay some PETN-packer to light his farts, they’ll laugh and keep doing it.

Clive Robinson • January 20, 2011 4:12 PM

@ Brandioch Conner,

“That’s a rather specific kind of bomb And a rather strange terrorist.”

+1

I do believe you are developing a very fine touch of sarcasm these days 8)

nobodyspecial • January 20, 2011 4:46 PM

@Joseph R. Jones – true but presumably the human cost of a downed airliners is the same whatever reason it crashed?

Then the cost:benefit analysis is cost of TSA vs number of accidents that can be prevented by spending the same amount on air safety – more FAA inspections, weather radar, better crew conditions/training etc

Of course the loss of plane when it’s the airlines fault is limited to $138K/passenger so the cost benefit suggests it’s not worth spending anything on air safety.

kingsnake • January 20, 2011 4:51 PM

And, of course, private planes and charters don’t have any of those restrictions. Just walk out of your car, and five minutes later you are airborne. But don’t tell the bad guys. (Or the TSA …)

Thomas • January 20, 2011 5:59 PM

The cost of a successful attack is inflated by the crazy over-reaction that follows it (e.g. 2 wars, almost a decade long with no end in sight as a result of an attack that killed under 4000 people).

As long as the cost of a successful attack is so massive almost anything that mitigates the risk seems worthwhile.

Dirk Praet • January 20, 2011 6:35 PM

“In other words, to be cost-effective, AITs every two years would have to disrupt more than one attack effort with body-borne explosives that otherwise would have been successful despite other security measures, terrorist incompetence and amateurishness, and the technical difficulties in setting off a bomb sufficiently destructive to down an airliner”

I don’t think TSA and DHS are thinking in terms of clinical cost-benefit analysis only. They’ve probably already made their own with pretty much the same results. Based on the outcome of red team tests, internally they must be aware of their poor efficiency too. I believe that more than anything else these machines are being used as a deterrent, like a dweep being picked at trying to bluff his way out of a fight yelling “I know kung fu”. Depending on his social engineering skills, this may work on some folk, but eventually he’s going to get his *ss kicked.

As long as the perception of real security holds up both with terrorists and the general public, budget is really the last of their concerns. It has been proposed before to replace this theatre with better techniques such as profiling and the Israeli approach. One of the main problems with the Israeli approach however is exactly the cost-benefit ratio because of scalability issues and resource intensiveness.

IMHO the main reason we are not seeing more terrorist attacks on planes is not because of AIT efficiency but by lack of sufficiently skilled operatives and resources to study and successfully exploit US airport weaknesses.

Slarty • January 20, 2011 6:47 PM

“the human cost, which one has to admit is incalculable”

Nonsense. We have to calculate human costs every day. We’re just not very good at it. Which is why health per capita in the US costs three times what it does in any other Western nation, and thousands more Americans died as a result of taking to the roads post-9/11 than died in the actual attack.

The issue is that because politicians (supported by a sound-bite media) trip out such trite comments we all end up living in a far more dangerous world than we should.

In a democracy, you get the government you deserve…

Jason • January 20, 2011 6:51 PM

Here is the only cost/benefit analysis that matters:

“Will I, a TSA official, be fired if I buy these scanners and they’re not needed? Will I be fired if I don’t buy them and someone blows up a plane?”

Richard Steven Hack • January 20, 2011 7:28 PM

Dirk: “IMHO the main reason we are not seeing more terrorist attacks on planes is not because of AIT efficiency but by lack of sufficiently skilled operatives and resources to study and successfully exploit US airport weaknesses.”

Agreed.

However, the sort of person who is prepared to blow himself up doesn’t match with the sort of person who has the intelligence and skill to be a good terrorist. The guys who blow themselves up are “patsies”. It’s the guys who PLAN the attack one has to be aware of. Sometimes these guys are in the employ of foreign intelligence operations like Pakistan.

However (again), THOSE guys tend to be known by Western intelligence forces – so THEY can’t GET next to a Western airport without being spotted. It’s a chicken-and-egg problem for foreign, non-white, terrorists in general.

It can be overcome by using intelligence assets assigned to embassies in the US to some degree who travel through those airports (also likely known by the FBI) or by developing indigenous assets (which the FBI is preempting by doing it themselves, albeit picking morons rather than useful assets – which presumably leaves the useful assets to the real terrorists).

In the end, though, it appears that the reason we have so few terrorist incidents is because…there are so few terrorists – especially terrorists with coherent plans and the determination to pull them off.

If that’s true, the cost/benefit analysis for body scanners is way off from the actual risk.

Of course, there’s also what USED to be the supposed main threat of airplane bombers – people who want to kill themselves so their spouses get the insurance money. How many TV shows and movies used to feature that scenario? Now it’s all Muslims. Not that the insurance scenario ever made any sense even to crazy people.

RobertT • January 20, 2011 10:59 PM

Not sure why I should care about this.

From my perspective, some clever guys are onto a nice little earner here selling marginally useful products to a paranoid customer. It’ll be junked in a few years when we figure out what the real cancer risk is but in the mean time ….

Maybe it is just me, but I believe that 3/4 of the security industry sells people stuff they don’t really need, why should AIT’s be any different.

X • January 21, 2011 12:31 AM

I thought the whole point of the scanners was to build a photo database of us all for use with the backscatter units they have deployed in vans. (Think “body recognition” instead of “facial recognition”). With the right groundwork and data collection ahead of time and you could go (have gone?) from zero to 100% surveillance society in a matter of months with nobody the wiser. Not that I’m paranoid or anything. 🙂

Hidden Assumption Spotter • January 21, 2011 2:54 AM

Implicit in the cost-benefit analysis is the assumption that there is some benefit.

Israeli aviation security expert Rafi Sela says he knows how to get enough explosive past one of the scanners to bring down a 747. He only talks about how to other security officials, but realistically, does anyone think the terrorists can’t figure it out? (http://www.vancouversun.com/travel/Full+body+scanners+waste+money+Israeli+expert+says/2941610/story.html)

Speaking of cost-benefit analyses, Sela makes another interesting point. He argues that the people who decide what’s needed shouldn’t be the same people who implement the measures. If you let that happen, there’s a conflict of interest as a bureaucrat muses “My, my, is this additional screening technique that will put 2,000 new employees under me and expand my budget necessary?”.

Robert in San Diego • January 21, 2011 3:23 AM

One fun question: How many attacks are “prevented” because the bad actors are afraid they’d not get on the plane in the first place. Admittedly we’re in “tiger repellent rock” territory here, but even a terrorist mastermind with disposable proxies at their disposal is going to go with a plan that is more likely to work.

Who’d want to talk in to a situation where they’d be leading the cast in another act of “Security Theater?”

Also, the people doing the actual cost-benefit analysis evaluation in DHS and TSA are often careerists — they have incentives to weight the cost of an attack quite high compared to an outside macroeconomist.

Grumpy • January 21, 2011 3:56 AM

“IMHO the main reason we are not seeing more terrorist attacks on planes is not because of AIT efficiency but by lack of sufficiently skilled operatives and resources to study and successfully exploit US airport weaknesses.”

and

“In the end, though, it appears that the reason we have so few terrorist incidents is because…there are so few terrorists – especially terrorists with coherent plans and the determination to pull them off.”

Exactly! Making plans, getting materials for a bomb, training for and coordinating the attack is a hard problem because of what the police is doing, not because anything TSA is doing. Hence, the only ones to make it to screening are going to be those who don’t prepare well enough, have silly “bombs” (burning underwear, HE up the wazoo – oh really), or otherwise are unable to actually cause much harm.

Paeniteo • January 21, 2011 4:01 AM

@Greg: “It is worth considering the agency problem here too. Specifically, the TSA executives making the decision to use the AIT don’t make the global cost-benefit computation, but a local one about their own careers.”

Very good point. The term for it (coined by Bruce?) is cover-your-ass, or CYA.
For decision-makers, there is no incentive at all to make risk-conscious decisions. However, there is a significant penalty if the decision proves wrong. No matter how small the probability for disaster given the risky choice is, one would always choose the safe option, since there are no benefits and the cost of the safe option is paid by others (= externalities).

Dirk Praet • January 21, 2011 4:25 AM

@ Hidden Assumption Spotter

“He only talks about how to other security officials, but realistically, does anyone think the terrorists can’t figure it out?”

Now that’s the difference between a well-funded drug kartel and a loosely knit organisation of religious fundamentalists living off hand-outs. The former would throw a lot of money at acquiring an ATI and studying its weaknesses, or bribing someone with an intimate knowledge thereof on their payroll. The former will require much less time to get there than the latter. Still, it cannot be precluded that both will eventually succeed.

dd • January 21, 2011 9:11 AM

seems like the dangerous ones are the red teams. trained to know all the flaws, wait til one of them gets disgruntled.
The israeli guy will use his knowlege when israel wants some plane down. and they likly will some day. they got a lot of neighbors they like to kill for their property.

Brandioch Conner • January 21, 2011 11:50 AM

@Robert in San Diego
“One fun question: How many attacks are “prevented” because the bad actors are afraid they’d not get on the plane in the first place. Admittedly we’re in “tiger repellent rock” territory here, but even a terrorist mastermind with disposable proxies at their disposal is going to go with a plan that is more likely to work.”

Why would said “terrorist mastermind” not find a different way to get past the scanners OR find a different target such as the line waiting to go through the scanners?

Or a mall during a busy shopping day?

Or any one of a thousand other targets?

John • January 21, 2011 12:57 PM

Human costs? Incalculable? Bollocks.

Let’s say airport A includes human body scanners and airport B doesn’t. Incidents at Airport A send a certain number of people away from flying; at Airport B, the invasive scanning and ad-hoc bad handjobs send people away from flying.

For human cost, how many people chose to fly at Airport A, despite the risk of loss of life, rather than at Airport B? How many people leave Airport B because of the scanners and elect to fly at Airport A?

We can calculate here a relative value between human life risk and liberty risk. Although airport scanners aren’t a liberty risk per se (more convenience; airports have the right to deny service to anyone, on near any terms), people tend to equate invasion of their privacy by corporations and governments.

Human life isn’t all-important. Would you rather save 5 peoples’ lives, or save 5 million people from having to live a “safe,” secure life free from sudden death but on a shortened life span under a tyrannical government and constant unwarranted scrutiny?

Roger • January 21, 2011 11:16 PM

This analysis is wrong.

It suffers from at least two major errors that are annoyingly prevalent in attempts to put security risk analysis on a mathematical basis.

  1. It assumes the fundamental parameters can be varied independently.
    I have privately named this “the Bean Counter fallacy” because I hear it so often from finance departments who want to cut security staff. There are few real world systems where such an assumption is completely true, but in a security context, it is nearly always grossly wrong. This renders all such mathematical risk analyses dangerous unless some way can be found to correct for it (which can generally only be done with expensive social experiments, which no-one wants to do.) In particular, for this analysis it is clearly total poppycock to assume that the likelihood of an attack being attempted is independent of the likelihood of an attacker failing.
  2. Product of many uncertainties.
    Like the infamous Greenbank equation, where anyone can get the result that suits their prejudices, the final result here is produced by multiplying out a long string of unknown, guessed parameters. Consequently the results are highly uncertain, and depend very sensitively on the authors’ assumptions.

The authors try to mitigate this by making very long arguments to claim that all their assumptions are conservative, but they frequently fail to convince me. There are many examples of this but for brevity I offer just one major one and one smaller one.

At one point they admit that the final result depends sensitively on the loss distribution function, which is unknown. To offset this lack of knowledge, they do the calculation 3 times, with three different candidate functions. However two of their candidate functions are absurd, and the third is a common blunder: widely assumed for mathematical convenience, but known to badly underestimate the risk of very high losses. (It was one of the root causes of the global financial crisis.) In fact, all their candidate loss functions share this property that very high losses are impossible. This is hardly a conservative assumption. A better choice would have been the Weibull function.

It also has quite a large number of lesser errors. For example, the cost of maintaining the complete AIT program is currently not known, so the authors estimate it (at $1.2 billion per annum) by linearly extrapolating the cost of the first budget estimate. This is not a sound method and usually produces large overestimates; pilot programs are invariably far more expensive per unit article than full scale programs.

In particular, this maintenance budget works out to $650,000 per machine per year, when the cost of purchasing and installing a brand new machine is only $430,000 each. Either someone is robbing the US Government blind, or that $650K includes a very large fraction of “pilot program” costs that cannot be extrapolated into the full program, making the authors’ $1.2 billion p.a. estimate not just non-conservative, but ridiculously high. Based on typical service life for equipment of this class, a more realistic estimate is probably less than $200 million p.a., only 1/6 as much.

Finally, I might also note that the DHS FY2011 budget request, described in the paper, seems very strange in itself. Only a very small part of the AIT budget is for the scanners, most of it is for staff wages and training. So far that is not remarkable, it is typical for many programs.

However, the numbers themselves are bizarre. Apparently, 500 scanners require 5,355 new TSOs and managers, or just under 11 persons per machine. I have seen videos of these machines operating; it is a one person job. Allowing for three shifts to give 24 hr operation gets us to 3 persons, and allowing a typical level of management, training and service overhead doesn’t quite get us to 4 (although it is an “effective” count of 4 if we allow a higher salary for the low-level managers.) No additional secondary search or guard staff are required because the scanners are being installed at existing checkpoints where those personnel are already posted.

Similarly, it is desirable to rotate operators on a fairly fast schedule (say, every 20 minutes) to maintain alertness. However, this doesn’t require additional “head count” because because they will be rotating with other personnel already on duty; only the total change in staff actually on duty counts against the budget increase. So how the hell do we get 11 people per machine? I think someone has a thumb on the scales.

Even worse is the figure of “$95.7 million for 255 positions to fund the support and airport management costs associated with the 5,355 new TSOs and screen managers.” That’s $375,000 — a quite high level executive salary — for each of these new positions! Apparently, the DHS needs to fund the airports a lucrative executive position for each 21 TSA personnel on site (who also have their own managers already.) Frankly, that sounds a lot like a bribe.

This is the real problem with this program: it’s a gravy train. If this program was staffed only with “effective”” head count of 4 per machine (to allow 24 hour shifts + maintenance, training and low-level management), a mid-level manager (double salary) for each 4 low-level managers, plus one executive level position (9 times salary!) per 4 mid-level managers, the cost of this program would be slashed by more than 80%.

Cale • January 22, 2011 9:43 AM

Probabilities, by their mathematical definition, cannot be greater than 1. Citing a probability greater than 1 reduces the credibility of their report. This topic interests me, but I don’t think I’ll be picking up a copy of Steward and Mueller’s forthcoming book “Security and Money: Balancing the Risks, Benefits and Costs of Homeland Security”

Mr. Gel-pack • January 22, 2011 9:11 PM

The authors’ “probability of attack” is more of a rate than a probability, and the results of 330% are intended to mean 3.3 attacks per year. On the same scale as the three attacks of 9/11, Reid, and that underwear dude over the last 10 years are about a 30% annual attack rate.

With such low occurrence problems, like airplane terrorism, yearly probabilities of attack on the order of 0-50% seem reasonable, but what the paper is trying to point out is that you’d need a much higher rate of terrorist attacks than we’ve seen. AIT might make sense in certain war-zone checkpoints, but does not make sense when applied to our relatively safe 2,000,000 passenger per day airline volumes.

X • January 22, 2011 11:40 PM

Roger: if you consider my total surveillance society post above and factor in the additional costs of vehicles at numerous additional transportation choke points, the manpower and cost numbers in that budget start to look a lot more realistic. Remember that a few dozen of the vans are already deployed in a testing phase and they have also been doing test installations of scanners at Greyhound bus depots as well. I certainly do hope this is all just paranoid fantasy but for several years now all of the evidence keeps pointing in the same direction.

averros • January 24, 2011 3:49 PM

Domodedovo (Moscow) International Airport was blown up today (35 dead, 150 wounded) by a bomb detonated in arrivals area.

Fat load of good these idiotic scanners did (yes, they are deployed in Domodedovo).

dd • January 25, 2011 7:40 AM

a better solution is to put a bunch of jersey barriers in the airport and have the passengers put their luggage between two of them, anyone lifting the luggage onto the barrier could be considered a terrorist and the order given to the passengers when someone puts luggage on one, they don’t get to fly and they have to go and be stripped and groped and whatever else dick cheney/jonny woo approves of for them.

these same barriers would have kept the casualties down in the sarajevo market mortar attack that killed 68 in the mid ninties.

or a type of ontario barrier built to have a top edge that is tapered to a thin edge, so the concept of a mortar hitting it becomes even more remote, and a thin edge would keep people from putting luggage there, along with the added height.

domodedovo

etherspirit • January 25, 2011 10:00 AM

Your research is inane. You are playing the game that Michael Chertoff wants you to play by asking whether or not body scanners are necessary or not, instead of asking whether former government officials should be able to sell their products to the government they just used to work for. Did you incorporate the fact that TSA has close to a 100% failure rate in many airports despite these scanners?

Constant Watcher • January 25, 2011 11:26 PM

TSA has done some serious analysis along the lines suggested and it is possible the analysis could go toe-to-toe or better than with the paper. TSA might be able to produce a response to this paper and schneier if internal permission is granted.

Roger’s comments on the flaws of the analysis are pertinent but there is even more to his line of thinking. More later if there is opportunity.

Mark Stewart • January 26, 2011 12:09 PM

I am very pleased that our research report has generated so much discussion – thank you to those who made comments, and most seemed supportive. This is the principal aim of the report, not to say that we know the truth of the matter, but good policy needs understanding of the benefits and effectiveness of new security measures before spending vast sums of public money. Our report sets out how systems thinking and reliability analysis can help quantify risk reduction which is essential to any cost-benefit analysis. The analysis is completely transparent and most calculations can be done by hand as shown in our equations – if you do not like our numbers, then use your own and see if that changes the outcomes. We have done sensitivity analyses where we show that the numbers need to be significantly different to change the final outcome. So the outcomes are robust we believe.

One issue that we did not mention was risk transfer – while an attacker may be deterred from attacking an airliner, they may not be deterred from attacking another ‘softer’ target, such as we have seen in Moscow where the airport arrivals hall was attacked yesterday. So while risks may be reduced to airlines by having AITs, risks to society may not change one ioata. This is why funds directed to intelligence and policing will be more effective than trying to protect individual targets.

I must also point out that this study is self-funded and we provide our findings to TSA for free. If the TSA were to fund a truly comprehensive cost-benefit study then the analysis would be more detailed and would benefit from their operational insights and data, including ‘near misses’ which the public may never hear about. If the TSA believes our work to be wrong then the onus is on the TSA to prove this by stating how they would do it better and producing their own results.

Let me address a few comments raised:

  1. While human life is incalculable, we as a society need to be sure that funds we spend maximise lives saved. For this, we place a monetary value on life as a means to compare one option against another. We contend that it is immoral not to do this type of analysis before spending $1.2 billion per year on a security measure that may have little or no benefit. It is probably costing lives as there is much research that shows that similar expenditures spent on flood mitigation or tornado shelters would save up to 200 lives each and every year.
  2. Some people have trouble getting their heads around an attack probability of more than 100% per year which is not surprising. For example, 330% means a mean rate of attack of 3.3 attacks per year. Or if you want to keep probabilities to be less than 100%, then an attack probability of 27.5% per month (330/12). The fact that we are dealing with such large (and counter-intuitive) numbers is an indication of how large the threat has to be for AITs to be cost-effective.
  3. Many parameters in risk analyses are assumed independent for the simple fact that they are. I would contend that many terrorists have an overblown confidence in their ability, and so it is reasonable to assume that the likelihood of an attack is independent of the likelihood of the attacker failing. However, if someone believes this is not correct then there are other systems and reliability techniques that can explore such correlations – such as game theory which is mentioned in our report. Risk transfer, as discussed above, also comes into play here.
  4. Loss estimates due to terrorism are often over-stated, but we take an upper value that is based on losses from 9/11. The shape of probability distribution is not that important, as the effect on distribution shape on the 90th percentile is not significant as the normal or Weibull distrubutions are unbounded and their 90th percent confidence intervals not massively different. We also found that doubling losses to $100 billion doesn’t change the outcomes that much. So we recognise that some parameter values are uncertain (but they are not ‘unknown’) which is why we did a detailed sensitivity analysis that included doubling losses and doubling risk reduction – both assumptions that bias the calculations in favour of finding AITs to be cost-effective.

  5. The TSA has documented that the annualised cost of 1,000 scanners is $650 million per year – and this includes purchasing, installing, staffing, operating, supporting, upgrading, and maintainable costs. For one blogger to then claim that the cost of 1,800 scanners should only be $200 million is without foundation. The TSA cost estimates are not our own, but I do agree that some seem high – but these are official TSA figures (and 1,000 scanners is hardly a ‘pilot’ program) and we took them at face value. I suspect that the TSA is not the most efficient operation around!

Finally, a risk analysis is very much a living document. It needs scrutiny from interested parties and experts, and thorough peer review before findings are implemented. Our report is the first step in the process, and not the final step as some seem to assume! If anyone has any questions about the report I will be very happy to discuss them anytime.

Leave a comment

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.