Schneier on Security
A blog covering security and security technology.
« Friday Squid Blogging: Researching the Reproductive Habits of Giant Squids |
| Scary Knife Makes for Great Newspaper Headlines »
July 21, 2008
Cost/Benefit Analysis of Airline Security
This report, "Assessing the risks, costs and benefits of United States aviation security measures" by Mark Stewart and John Mueller, is excellent reading:
The United States Office of Management and Budget has recommended the use of cost-benefit assessment for all proposed federal regulations. Since 9/11 government agencies in Australia, United States, Canada, Europe and elsewhere have devoted much effort and expenditure to attempt to ensure that a 9/11 type attack involving hijacked aircraft is not repeated. This effort has come at considerable cost, running in excess of US$6 billion per year for the United States Transportation Security Administration (TSA) alone. In particular, significant expenditure has been dedicated to two aviation security measures aimed at preventing terrorists from hijacking and crashing an aircraft into buildings and other infrastructure: (i) Hardened cockpit doors and (ii) Federal Air Marshal Service. These two security measures cost the United States government and the airlines nearly $1 billion per year. This paper seeks to discover whether aviation security measures are cost-effective by considering their effectiveness, their cost and expected lives saved as a result of such expenditure. An assessment of the Federal Air Marshal Service suggests that the annual cost is $180 million per life saved. This is greatly in excess of the regulatory safety goal of $1-$10 million per life saved. As such, the air marshal program would seem to fail a cost-benefit analysis. In addition, the opportunity cost of these expenditures is considerable, and it is highly likely that far more lives would have been saved if the money had been invested instead in a wide range of more cost-effective risk mitigation programs. On the other hand, hardening of cockpit doors has an annual cost of only $800,000 per life saved, showing that this is a cost-effective security measure.
From the body:
Hardening cockpit doors has the highest risk reduction (16.67%) at lowest additional cost of $40 million. On the other hand, the Federal Air Marshal Service costs $900 million pa but reduces risk by only 1.67%. The Federal Air Marshal Service may be more cost-effective if it is able to show extra benefit over the cheaper measure of hardening cockpit doors. However, the Federal Air Marshal Service seems to have significantly less benefit which means that hardening cockpit doors is the more cost-effective measure.
Cost-benefit analysis is definitely the way to look at these security measures. It's hard for people to do, because it requires putting a dollar value on a human life -- something we can't possibly do with our own. But as a society, it is something we do again and again: when we raise or lower speed limits, when we ban a certain pesticide, when we enact building codes. Insurance companies do it all the time. We do it implicitly, because we can't talk about it explicitly. I think there is considerable value in talking about it.
(Note the table on page 5 of the report, which lists the cost per lives saved for a variety of safety and security measures.)
The final paper will eventually be published in the Journal of Transportation Security. I never even knew there was such a thing.
EDITED TO ADD (8/13): New York Times op-ed on the subject.
Posted on July 21, 2008 at 5:53 AM
• 25 Comments
To receive these entries once a month by e-mail, sign up for the Crypto-Gram Newsletter.
Discussing costs of saving human lives gets easier when you realize there's a cost on the other side too: spending more money on safety will mean raising taxes and consumer prices so that a certain number of people will become homeless or unable to pay for medical costs. That also costs lives.
Zoom in on the TSA's colorful 20 Layers of Security. Despite the layout, the paths are meaningless, and the whole is simply a list of twenty different budget items. The whole defense might fail without any of the 'layers' failing. This thing is as silly as nutritionists' "food pyramid", a solid with three dimensions which are never explained.
Yup, that journal is one of the /grande dames/ of industry journals. Highly respected.
Some of this data comes from a senate committee that was charged with making recommendations to industry in the wake of the September 11th attacks. Their final recommendations were rather clear-headed, and still make all sorts of sense:
"have never knew"?
Very interesting discussion, though. :)
If they REALLY did cost/benefit analysis, they would realize that public healthcare would save many more lives than junk like full-body x-ray machines at airports.
I am not able to download the full text of this report. Can someone help me point out from where I might be able to do so??
Regarding the air marshal program, already one innocent person has lost his life (Rigoberto Alpizar, December 2005 in Miami). The cost of false-positives is quite high.
I think people might be more receptive to cost-benefit analysis if you emphasize that it's important to implement the things that give the most bang for the buck _first_.
Personally, I've always thought a bounty on hijackers is the best option. $10,000.00 per passenger per dead hijacker. $100,000.00 per passenger per hijacker alive to answer questions. Mistakes and attempts at fraud or to otherwise game the system would be prosecuted as air piracy. Relatives, friends, or other associates of the hijacker(s) could not collect. If the relative, friend, or other associate is instrumental in subduing the hijacker the government will waive the death penalty.
We'll mob them anyway. The government should encourage that, while discouraging fools, idiots, and fraudsters.
Link doesn't appear to work. Or is it just me?
"The United States Office of Management and Budget has recommended the use of cost-benefit assessment for all proposed federal regulations."
Oh, that's rich! Thanks for the laugh! And what will they recommend when they find the costs outweigh the benefits of the proposals? That the proposals get reworked? That's hilarious!
And regulations that are already in place, what about a c-b analysis for them? Didn't think so. So if a proposal claims it's just amending *existing* regulations, it's exempt from having to do any c-b study, right?
"Cost-benefit analysis is definitely the way to look at these security measures."
Hmm. I wonder if it's a way to look at government entitlement programs also? Didn't think so. Not our babies. Stay away, you evil economists!
Check out a recent Colbert Report. He has an interesting outlook on the "cost of life" albeit I think tounge-in-cheek.
We have been putting a price on life for many years. It is a fundamental component of the design process for many engineering projects. Build a road or a railway and the actuarial tables tell us how many additional deaths will result. Having a state approved cost per life saved, the engineer can decide how much to spend on additional safely features.
The point is, spend the money on the areas that will save most lives. The billions spend on useless airline security could have saved thousands of lives on the roads.
" putting a dollar value on a human life...I think there is considerable value in talking about it."
I don't think so. The cost of saving a life isn't the same thing as the value of a life.
You don't need to assign a dollar value to life to compare the cost per life of various safety measures - that factor cancels out.
It's a laudable goal to take a rational approach to these things, and the report is very good and a step in the right direction. However, it attempts to provide a rational cost on irrational measures, making up arbitrary numbers in the process. The fundamental problem is that massive amounts of money and manpower have been put to work to prevent the repetition of an incident that is by its nature most likely unique.
In paragraph 4.2 (pp. 15-16) the authors quote two sources that consider the 9/11 incident an "outlier", "with deaths on this single day approximately equal to all transnational terrorist-related deaths recorded during the entire 1988-2000 period" (Sandler and Enders), or "among stratagems so uniquely surprising that their very success precludes their repetition" (Seitz). To make a rational - in the sense of not dividing by zero - cost-benefit calculation the authors make the seemingly arbitrary assumption that an incident of a similar magnitude (3000 victims) would be repeated about every 10 years. This is justified by the fact that the measures under evaluation have been taken specifically to prevent a repeat of 9/11. However valiant an attempt to enumerate the effectiveness of the measures, I think this logic is somewhat circular and only serves to obfuscate the fact that the 9/11 incident itself was not easily repeatable.
(The sources quoted in the report: Todd Sandler and Walter Enders, ‘Transnational Terrorism: An Economic Analysis’, The Economic Impact of Terrorist Attacks, H.W. Richardson, P. Gordon, and J.E. Moore II (eds.), Elgar, 2005, pp. 11-34.
Russell Seitz, ‘Weaker Than We Think: Al-Qaeda May Have Already Fired Its Best Shot’, American Conservative, December 6, 2004. See also John Mueller, ‘Harbinger or Aberration?’ National Interest, Fall 2002, pp. 45-50.)
So how far do you take this cost-benefit analysis? Do you factor in the law suits that will come out of it when people realize you could have implemented a feature to prevent a given action but didn't?
Just a thought.
Great op-ed in the New York Times today that addresses this precise topic:
Written by a civil servant from the DHS.
An excerpt that shows that at least someone at the DHS knows what they're talking about:
"To appreciate the challenge we face, it is important to remember that the Sept. 11 terrorist attack was a classic “low-probability, high-consequence” event."
and then further
"Not surprisingly, when people started to focus on bad things that might happen to us, they identified a seemingly infinite catalogue of worrisome possibilities ..."
> Do you factor in the law suits that will come out of it
high-probability of lawsuits if you do, high-probability of lawsuits if you don't.
From the table:
"Hazardous waste listing for wood-preserving chemicals ... 1990 ... EPA ... 6,785,822.0"
Did anyone else catch that? 6,785,822 is a pretty penny in millions of dollars. I doubt we've spent 6.7 trillion on that... a divide by zero error?
> It requires putting a dollar value on a human
> life -- something we can't possibly do with our
Execpt that we do that every day by choosing cheaper Japanese tin cans over Volvos, by skipping on expensive medical insurance,. by flying discount airlines, by not hiring bodyguards, etc, etc. Every one of us is quite adept at valuation our own lives.
The solution to the problem of assigning value to someone's life is very simple - let the person to make choices for himself. (However the corollary would be dismantling the nanny government with all its three-letter agencies and letting the parasites to look for honest jobs - so, no wonder, this is not a popular approach amond the tax-fed "intellectuals").
If we really want to thwart Al-Quaeda and make sure we do not reward their efforts we must do this -> NOT spend lots of money and impinge on freedoms in the US; because that is their goal.
The goal of a terrorist to cause undesirable changes in the attacked society, making that society willing to accept [whatever political goal the terrorist seeks].
On 9/11 President Bush said that freedom had been attacked. This is not true, external forces can not attack our freedom. They can attack us politically, economically or physically; both collectively and individually, but only our own GOVERNMENT can attack our FREEDOM.
Cost-benefit analysis is obviously laudable, but if the report (I haven't read it yet) sticks to a benefit of lives saved then it isn't doing it right. 9/11 had immense financial costs beyond the loss of human life. I have no idea what those costs are, but just the immediate damage to the buildings is the same order of magnitude as 3000 people times $1-10 million per person.
Cost-benefit is very difficult to utilize in a security setting because the benefit is that a bad event doesn't happen. How do you measure how many bad events have not happened? Answer--you don't unless you have a perfect simulation and have perfect intel on terrorist intent.
how do the people in the study know how many lives have been saved? Maybe it's zero, maybe it's 1 million. We CANNOT know. I haven't read the report, yet, but if they use an expected value function to estimate the lives lost from a terrorist event based on past experience, they are also wildly off. Terrorist acts occur with a particular liklihood until they don't. That is to say, past performance is no guarantee of future performance. Terrorists are strategic actors---not randomized phenomenon like car wrecks so we can't know even from past experience how many lives have been saved. That Bruce Schnier doesn't point this out is baffling to me.
>but only our own GOVERNMENT can attack our FREEDOM.
So true, as the article on sunscreen being confiscated at Yankee Stadium clearly shows.
So who has won? Us or the "terrorists". Sounds like they might ought to be called "Inconveniencists" and also the winners of this round.
Also I would like to point out that we often name our collective percieved threat as "Al-Qaeda" but in reality there are many organizations in the world that would do us harm. Al-Qaeda just seems to have gained notoriety.
So we focus on them and miss other "threats"?
This cost-benefit analysis measures only the direct disbursement cost of the government in implementing the security measures - and still finds most of the measures uneconomical.
What of the massive costs to everyone flying? What is the cost of the extra time that flyers spend waiting in line, taking off their shoes, forgetting their laptops ... an extra half hour or hour per passenger per trip over what reasonable security might take - multiplied by millions of flights per year?
Add in those costs and the measures *really* don't make sense (except probably the hardened flight cabin doors, which impose no costs on travellers).
Hi every body, Further information on airlines of USA, UK and some other countries airlines information like aircraft’s images, cabins, phone numbers, address, baggage information, check in option, rules, IATA code, history visit "Altiusdirectory.com".
Schneier.com is a personal website. Opinions expressed are not necessarily those of BT.