Schneier on Security
A blog covering security and security technology.
« Arming New York City Police with Machine Guns |
| Brazilian Logging Firms Hire Hackers to Modify Logging Limits »
December 17, 2008
Ed Felten on TSA Behavioral Screening
Now suppose that TSA head Kip Hawley came to you and asked you to submit voluntarily to a pat-down search the next time you travel. And suppose you knew, with complete certainty, that if you agreed to the search, this would magically give the TSA a 0.1% chance of stopping a deadly crime. You'd agree to the search, wouldn't you? Any reasonable person would accept the search to save (by assumption) at least 0.001 lives. This hypothetical TSA program is reasonable, even though it only has a 0.1% arrest rate. (I'm assuming here that an attack would cost only one life. Attacks that killed more people would justify searches with an even smaller arrest rate.)
So the commentators' critique is weak -- but of course this doesn't mean the TSA program should be seen as a success. The article says that the arrests the system generates are mostly for drug charges or carrying a false ID. Should a false-ID arrest be considered a success for the system? Certainly we don't want to condone the use of false ID, but I'd bet most of these people are just trying to save money by flying on a ticket in another person's name -- which hardly makes them Public Enemy Number One. Is it really worth doing hundreds of searches to catch one such person? Are those searches really the best use of TSA screeners' time? Probably not.
Right. It's not just about the hit rate. It's the cost vs. benefit: cost in taxpayer money, passenger time, TSA screener attention, fundamental liberties, etc.
Posted on December 17, 2008 at 6:38 AM
• 53 Comments
To receive these entries once a month by e-mail, sign up for the Crypto-Gram Newsletter.
I think his assumption about the point where all "reasonable" people would agree to the search is wrong. Its right around the point where I would say no (depending on the pool of people who might be affected). For example I have gone skydiving and at that time (things might be safer now) there was something on that order chance of getting myself killed. So I think his argument that taking chances with life at that magnitude is "unreasonable" is wrong.
Thought experiment: Now suppose that TSA head Kip Hawley came to you and asked you to submit voluntarily to a pat-down search the next time you travel. And suppose you knew, with complete certainty, that if you agreed to the search, this would magically give the TSA a 0.1% chance of stopping someone from flying with drugs or a fake ID.
Some years ago (I won't say exactly how many), my 16-year-old incarnation posited to my Dad that life is infinitely valuable, and any effort necessary to save a life is justified. (Yeah, yeah, I know, I'm a slow learner.) Suffice to say that he disabused me of that belief, using logic as opposed to my emotional ``justification''.
Unfortunately, the majority of the U.S. populace needs to talk to Dad.
if he told me that if they search me, they have a one in 1,000 chance of catching someone with a fake ID, i'd laugh right in his face.
Anonymous at December 17, 2008 8:35 AM: "if he told me that if they search me, they have a one in 1,000 chance of catching someone with a fake ID, i'd laugh right in his face."
That's really not the statistic. A 0.1% chance of identifying something is not justifiable, if that were the case.
That is a much different statistic than the probability that about 0.1% of the people you encounter are doing something you want to identify. Then the actual number is a portion of the population which is in the hundreds per day, if not thousands. Needless to say, that does not mean I agree with the TSA, but we need to understand that the 0.1% is not the odds they will identify something (which inverted is a 99.9% that someone will get away with something), it is the percentage of things they want to identify. Those are very different figures.
@: "most of these people are just trying to save money by flying on a ticket in another person's name -- which hardly makes them Public Enemy Number One."
Hardly public enemy number one, but worth identifying. Fake IDs are a problem, and so are people who cheat to save a buck, which gets paid for by higher prices for honest people like us. This isn't the reason for the security, but it is hardly an argument against it.
I disagree with Felten's notion of reasonableness. Why do shallow people always assume that everyone in the world shares their values and judges things the way they do, or that if they don't, they're not "reasonable"? No, it is not worth it to me to submit to constantly being treated like a suspected criminal and subjected to humiliating personal searches in order to reduce the likelihood (already small to begin with) that someone else might do something nasty. So all this talk about the ratio of arrests is really beside the point.
Let's carry this thought a little further, though. Let's say we were in Israel or Iraq where there is a constant threat of pedestrian suicide bombers. Would you consent to a pat-down search every time you walk down the street in order to reduce the likelihood of suicide attacks? Conceivably you might find yourself getting stopped and searched several times each day. (And let's assume that, like Americans, Israelis didn't believe in racial profiling, so even if you're obviously Jewish, you still get searched.) How many offenses against your liberty and personal dignity in the name of "safety" are you willing to accept? And how long will it take for a government that has gotten its citizens to accept such things to start thinking of other uses for such powers?
@Craig: "I disagree with Felten's notion of reasonableness. Why do shallow people always assume that everyone in the world shares their values and judges things the way they do, or that if they don't, they're not "reasonable"?"
I think you make a good point. I think both sides make both reasonable and unreasonable arguments, particularly their assumptions about the other. While I do think TSA practices are ridiculous at times, I also think the reactions they get drive some of their thinking. Take the shoe bomb example, which is a very low risk. No one will receive much, if any, credit for correctly saying "hey, it's not worth it, you can keep your shoes on." On the flip side, all it takes is one copy cat idiot out of millions of flyers (someone they have no reason to suspect of anything) to lose his mind and blow up his shoe on a plane, and they will be cut to ribbons for "failing to protect against a known threat", however small. Big lawsuits will follow, and a loss of business in the industry as a whole. Part of it is CYA, but part of it is also just simply being less costly (not just money) than a lawsuit with sobbing relatives of a small number of victims.
While I think things would be better if everyone were as rational as most posters on this blog, but that isn't the reality. And as long as the general public remains unreasonable in their expectations, we can expect unreasonable results. But, part of the problem is leadership's inability or unwilingness to communicate reality.
This rebuttal makes no sense, because the percentage is way too high.
Would you submit an entire nation to humiliating and expensive pat-down searches if it would save 0.1 lives per year? Is that worth it?
I say no. Maybe this is callous and evil of me, but it's simply not worth it. There are limits to how far you should go to save lives. If you end up wasting hundreds of man-years of people's time before you save a single life, it seems to me that in the end you've suffered a large net loss.
Furthermore, could all the time and money spent on pat-down searches be put elsewhere and save more than 0.1 lives per year? Almost certainly.
Making up numbers to prove your point really doesn't say much. If every airport pat-down search saved one thousandth of a life, violent crime would be completely eliminated and then much more beyond! Presumably this would be accomplished by raising past crime victims from the dead, I don't know.
1) "apparently hasn't caught any big fish yet" because there aren't any. If there were, they'd long ago have started hitting softer targets.
2) The next arrow in the TSA's quiver is full body X-Ray for every passenger.
I can't think of a better example of "unreasonable searches and seizures" than the methods and madness of the TSA.
The 4th amendment is dead.
@Michael: "Would you submit an entire nation to humiliating and expensive pat-down searches if it would save 0.1 lives per year? Is that worth it?"
In that case, I would say no. For the same reason I don't want speed limits cut in half and loath "roadside safety checks" where drivers get stopped at a road block for no reason and get inspected for license, insurance, and soberness. We should be willing to assume a bit higher amount of risk to live freer and happier lives.
I am willing to make situational exceptions. Key word, exceptions.
The trick to making people give up their rights and freedoms is by doing it very gradually. First you just ask them to remove their shoes while walking through the metal detector, then you kindly suggest they submit to pat down searches, then full-body X-Ray screening and the next thing you know - people think it's normal when you put cameras in their living room. Throw in some guilt-trips about saving lives and being patriotic and you've got yourself a Brave New World...
"...and so are people who cheat to save a buck, which gets paid for by higher prices for honest people like us."
To give you an example.
I buy a ticket, but for some reason I can't fly.
The ticket is a cheap one, therefore not refundable, no changes allowed.
I can throw it away or give it to a friend who is happy to fly instead of me for free (and cheat as the ticket is not transferable).
Can you explain to me why would you pay a higher price if he flies instead of me throwing away the ticket?
I'm certainly no lawyer - but TSA bothers me on a major civil liberty and civil rights level.
The justification for searches and seizures without a warrant or specific cause, is national security and to stop terrorism.
Where are the terrorists we've caught? I read the TSA happy-talk-blog, and they talk about covert weapons and bad IDs and drug seizures. They are proud of the tons of "contraband" they seize.
But, after years: where are the terrorists, the arrests for terrorists? If they stop a tourist or businessman with a pocket knife, keep the knife and let the person go, they have either done something utterly useless, or worse: released a terrorist.
We would not tolerate border checks on highways between states, and we do not find it legal to stop all patrons leaving a bar to see if they are drinking and driving... why is wholesale TSA screening tolerated if it does not fulfill its real purpose?
Where are the terrorists TSA stops?
As an Israeli (and, if we're going to discuss profiling, might as well state that I'm Jewish too), the answer is a definite "yes". I do submit to physical search (quick, usually a swab with a portable metal detector, but often at least being asked to let the guard have a peek inside your bag) whenever entering a bank, restaurant, mall or any other public place. And people do, in fact, agree to it, because these searches have, in the past, saved lives (usually by scaring the attacker into triggering early, but the math does add up).
In other words, if the chances of actually saving lives are high enough (and the threat concrete enough), then it makes sense. The reason the TSA's (and Felten's) logic does not stand to reason is twofold:
1. The numbers don't add up. Contrast and compare dozens of publicly known incidents where such searches actually prevented an armed/carrying a bomb terrorist from entering a crowded place with this low chance of catching something unrelated to the original purpose of the search.
It is worth noting that, due to many reasons, suicide bombers attacks have not been seen in Israel for some years now, and as the threat gets less concrete, so does my annoyance with the search increase.
2. These are not voluntary searches. If I don't agree to the search, they won't let me in.
I don't get the logic in agreeing to being searched. *I* know I'm not a terrorist, so the chances that they'll catch a terrorist if I allow them to search *me* are zero.
Pray tell then what should be done?
Get rid of the checking?
Remove metal detectors?
Or just stop flying
It might be instructive to follow the link and read Ed's post. He's not a TSA apologist. The whole point of his article was just to point out that no matter how you feel about the TSA, "low hit rate" isn't the main problem with screening ("Low Hit Rate Isn't the Problem with TSA Screening" is even the name of his piece). He was trying to promote finding stronger arguments, not cheerleading for the TSA.
We know that some cops are also drug traffickers. If we citizens were allowed to search cops at will, there is a better than a .1% chance a search would turn up a dirty cop, so why isn't this the law?
Because the cops would never submit to it, even if it were the law, and the cops have the guns and we don't.
If Kip Hawley wants to search me, I'd say fine, let's negotiate. He can start with offering me $5,000 in cash and lending me his car for a week. We'll see how much it's worth it to him to have a chance to catch a crook.
Bah, TSA-DEA, same thing anymore and about the same success rate.
@cheater: "Can you explain to me why would you pay a higher price if he flies instead of me throwing away the ticket?"
The situation you explained, no one would lose out financially. Thank you for that clarification.
I was thinking along the lines of deception in booking to get lower fares. Sorry if I wasn't clear.
In any case, someone lying about their identity to "scalp" a ticket is a problem, for many obvious reasons. Financial is just one.
Except low hit rate *is* a big problem with screening. If the hit rate is too low then the program ceases to be worth the money. There may be other critiques against the program as well, but that it's not very effective seems like an important one to me.
As he points out, most of the "hits" are drug charges and fake IDs. These aren't really hits in the sense being discussed, because they don't save lives. So the true hit-rate is much lower than 0.1%. The money could be better spent elsewhere, and it's precisely because of the very low hit rate that this is true. Imagine if 90% of the people targeted by this program were found to be carrying explosives and automatic weapons; it would be a smashing success!
1) Math check: I calculate 1266/160k = 0.79%. Still tiny, but eight times more than one-in-a-thousand.
2) Cost / Inconvenience level: According to the USATODAY article, many of the people get brief interviews, not patdowns. I object much less to the inconvenience of an interview than I do to the intrusiveness of a patdown.
3) Value / Results obtained: Neither the USATODAY article nor Felten’s commentary mentions the conviction rate, which I would find much more salient than the arrest rate. How many arrests went unconvicted due to lack of evidence or other problems? How many were due to poorly-trained agents just being pissy? How many people were inconvenienced well beyond a simple interview or patdown — and at what taxpayer cost — with no benefit to society at all? Without this number, the arrest rate means very little to me.
I ask if Israelis should submit to random pat-down searches on the street and you reply that you do, in fact, submit to someone waving a metal detector at you and looking in your bag when you enter a public building. Not exactly the same thing, is it?
"if you agreed to the search, this would magically give the tsa a 0.1% chance of stopping a deadly crime..."
total b.s. deadly crimes on airplanes aren't statistically predictable events like wins in roulette are. precisely which deadly crime would that be?
"any reasonable person would accept the search to save (by assumption) at least 0.001 lives..."
my reasonability and your reasonability are obviously not the same reasonability.
One serious flaw is we have no way to measure what the statistics would be if airport security was different. Surely the practices, even the poor ones, deter some people from doing things they would otherwise do if the measures were eliminated.
I don't think anybody disputes that these measures have *some* effectiveness. That is not the question. The question is whether the effectiveness is worth the money, and whether something more effective could be done for the same budget. Given the astonishing (although not total) ineffectiveness of current measures, it seems virtually certain that something better could be done instead.
The problem with Felten's argument, I think, is that in the absence of any real data about prevention of terrorist acts or total cost to the public, and given the not-obviously-terrorist nature of the hits the TSA does get, hit rate is pretty much the datum we have to work with.
It's possibly that such a low hit rate does have a positive cost-benefit ratio (color me skeptical) but without the rest of the data required to make that argument, I don't think the onus is on members of the public to prove that it doesn't.
And suppose you knew, with complete certainty, that if you agreed to the search, this would magically give the TSA a 0.X% chance of accidentally sending a family man to Guantanamo. You'd agree to the search, wouldn't you?
There are oh so many problems with the TSA position. Here is just one...
The TSA spends billions targeting threats on aircraft, but as we have seen in recent times, produce markets, hotels and night clubs are currently prime targets. How much does Homeland Security spend to ensure you can buy organic carrots in safety?
I agree with you. I don't think the measures are cost effective either, I'm just saying that percentages based on people who are aware of the existence of checkpoints cannot measure the percentages of people's behavior should the measures be eliminated.
But I agree, something better could (should) be done. I just don't think it should be based on the faulty assumption that only 0.x% of people are caught, considering that number will change as people are aware of new/changed/removed measures.
The very fact that the TSA feels a need to count false positives (e.g., detections of drugs or fake military uniforms) as "successes" can only indicate that terrorists or other true threats to aviation are too few and far between to justify their intrusive measures and funding levels. But they presumably also assume that many if not most people will view the false positives as evidence that the TSA is accomplishing *something*, even if it's not actually stopping threats to aviation. Convincing the public that the TSA is doing *something* apparently matters more than whether that "something" is effective at protecting aviation from real threats. Fear always trumps rational analysis. But let's hope the Obama administration encourages rational analysis and cost-effectiveness rather than promoting its own delusions of infallibility through fear as the Bush administration does.
Nice off-topic whoppers there George.
Anything rational to contribute?
"And suppose you knew, with complete certainty..."
They lost me after this point.
Might as well say dead-and-alive cats are real, therefore...
The question shouldn't even be the cost-benefit ratio of the TSA's methods, but how that cost-benefit ratio compares to spending the money on, say, healthcare.
@George "But let's hope..."
i don't think commenting on current/subsequent political administration is relevant to this post. we have all seen examples in our lifetimes where, no matter what the political climate, things happen that simply don't make sense. to say "this is happening because THEY did this and it is THEIR fault" is puerile and akin to the "nanny nanny boo boo" comeback from primary school.
>In any case, someone lying about their identity to "scalp" a ticket is a problem,
>for many obvious reasons. Financial is just one.
I'm afraid the reasons aren't so obvious to me. Other than saving the airlines money (and costing travellers money) what are the problems with not showing ID at the airport? [And saying 'because the TSA won't know who they are' isn't an answer, since this discussion is about trying to justify the TSA ID-check in the first place]
@cheater, HJohn: Airlines overbook. If your hypothetical flight is full, one more of the confirmed pax will get to fly and that is a "free" seat they do not have to give away at a later time and make up revenue for.
If it is not overbooked they can still have standby pax show up at the last minute and sell THEM your unoccupied seat (your friend, perhaps).
@ice weasel: You're not even close. The TSA is -!!!WILDLY!!!- more successful than DEA and at a tiny-tiny-tiny-tiny fraction of the cost of DEA (and that's just the financial cost and doesn't count the money spent by the states either).
At least the TSA can point out that there have been no planes hijacked on their watch [I'm not saying TSA is responsible for the lack, just that it hasn't occurred]. It would be ludicrous in the extreme to even begin to say that no drugs have been brought into the country in the same time frame. Its probably not even true that there are fewer (unlawful) drug users in the US today than 7 years ago.
@TO EVERYBODY who is wrapped around the axle on the stated "success rate": The author is not claiming that it is the success rate, I am sure he would stipulate its far lower than that. Its simply a made-up number to show that even if it were that good, success rate is not the problem with the process.
I learned a long time ago...never agree to anything voluntary with the Government (any Government, federal, state, local, whatever) which amounts to giving up a constitutional right. To do so can only harm you, even if you have never committed even the smallest of infractions your entire life. Once you give up the right, you cannot take it back for that "search" and if you have no control over the searcher then you have no way to prevent corrupt use of the search results.
Just remember, agreeing to a search can never help you. If they have the right to search you, they don't need your permission in the first place. If they don't have the right and they search you without your permission, anything they find can't be used to incriminate you.
Where does the 4th amendment say "unless we're scared"?
To get on an airplane, I'm subject to an unconstitutional search by the federal government. I don't care how many lives that saves. Liberty trumps safety.
Worse, to enter a courthouse, I'm subject to an unconstitutional search by the local government. The government is *supposed* to be scared of gun-toting citizens unhappy with their leaders. That was the entire point of the 2nd amendment.
"I'm just saying that percentages based on people who are aware of the existence of checkpoints cannot measure the percentages of people's behavior should the measures be eliminated."
That's nice, but I don't think anyone has said or is saying anything of the sort.
"I just don't think it should be based on the faulty assumption that only 0.x% of people are caught, considering that number will change as people are aware of new/changed/removed measures."
It should be clear that there is deterrent value. And indeed I don't see anyone contributing to the discussion who doesn't. So I really don't see why you're saying this. Either you're missing the point or you're building a strawman....
I wasn't accusing anyone of saying anything. I don't think I'm missing the point or building a strawman. And I apologize if it seems that I'm saying others are saying things they aren't.
So, I will attempt to be clearer. My points are summarized in two parts:
1. I think the TSA is inefficient and that they are wasting some resources. I concur with the majority here on that point.
2. I was attempting to point out, without insinuating anyone is saying the opposite, that when we make changes to the process (and changes are needed), that it would be prudent to keep in mind that the changes may very well change the equation of what is attempted.
The only thing I'll say about your "strawman" argument is I doubt anyone on this blog, myself included, is looking to provide cover for whatever the TSA does.
I appreciate the dialogue and your keeping me honest.
Interestingly enough, if airport security was being handled by the airlines you would have no constitutional rights with respect to the search. The search would be part of the agreement you made when you bought your ticket. So, if it is ever determined unconstitutional then the government will stop doing it, but require that the airlines do it instead. This would actually be worse than the situation we have now as the airlines would have ever incentive to be more onerous than the current regime with no potential checks on their power.
@JimFive: "the airlines would have ever incentive to be more onerous than the current regime with no potential checks on their power."
True enough, as far as it goes. On the *other* hand, there's a hypothetical-at-least possibility that one or more airlines will *refuse* to be "even more onerous" in order to attract passengers - "Fly with us, we're nicer about it than they are".
Hypothetical because there's precious little evidence of an airline treating passengers as anything other than cattle - it's rare enough that when it actually happens, it becomes a news story on cnn.com.
My apologies, then. I've seen a tendency in this sort of discussion to demonize the other side by arguing as though they hold a position far more extreme than they really do, and I thought you were doing that. Clearly not, you were just making explicit some important points that I think we can all agree on. Carry on!
Re: Racial profiling in Israel?
Bruce wrote in July 2005 "Whenever you design a security system with two ways through -- an easy way and a hard way -- you invite the attacker to take the easy way"
Israeli screeners have learned the hard way that some suicide bombers disguise themselves as ultra-orthodox Jews, or as soldiers. So everyone gets searched. Including Jews.
There's another reason for airlines to do fewer searches: less cost to them. So not only do you attract customers by saying "Fly with us, we're nicer about it than they are.", you also get to tell your prospective stockholders "Buy our stock, we have lower costs".
I would do the math. Since "Last year, 769 million passengers boarded U.S. airline flights" that means we are saving the lives of 769,000 people. Pretty amazing.
"Freedom isn't Free." Some Americans love to put that on bumper stickers. But do they really believe it? Are they really willing to pay a price for freedom? What if that price is an airline crash or two? Gruesome, certainly. Appalling, certainly. But Freedom isn't Free, or so the bumper stickers tell us.
The Patriots who fought the American Revolution were willing to pay for freedom with their lives. Are their descendants?
50,000 TSA employees working 2000 hrs/year = 100 million hrs
258 million air passengers/year each spending an extra 20 minutes in airports due to early-checkin security requirements and waiting in security lines = 86 million hours
1% of all air travel delays due to security induced delays at an average of 8 hours per missed/delayed/cancelled flight = 20 million hours (WAG)
total: 206 million hours of wasted time per year (Americans only)
average life expectancy: 75 years = 0.657 million hours or 0.438 million waking hours
Thus ~470 lifetimes are wasted each year.
Since 9/11/2001 that's 3418 lifetimes or more than the 2974 people killed in the 9/11 attacks.
So the TSA has killed about as many people as Bin Laden.
And the TSA is still going strong.
Note this is just counting wasted time, not the monetary value of the salaries and equipment used by the TSA.
Unless the search is going to save hundreds of lives per year, airport security is too expensive.
Thats a first time in a long long time I have seen someone using the phrase "Brave New World" correctly.
@JimFive: "the airlines would have ever incentive to be more onerous than the current regime with no potential checks on their power."
Experience would dictate otherwise. Remember, it was the airlines who were responsible for security before 9/11.
And, oddly enough, airport security worked well enough on 9/11. What didn't work was the POLICY of acquiescence.
My comment was directed toward a post-WTC world where the regulations were being enforced by a private entity required to meet standards created by the government. The main point is that it isn't unconstitutional if it is done by a private entity.
Thanks for your post. That was along the lines of what I intended to say, but I just got caught up on reading Bruce's last 2 months of posts. My number was going to be at least 50% higher, since the arrive-early time for me has always been at least an extra half-hour over what it was pre-post-9/11 security theatre. And honestly, I still think that number is too low.
Schneier.com is a personal website. Opinions expressed are not necessarily those of Co3 Systems, Inc.