Entries Tagged "air travel"

Page 9 of 46

Criminal Intent Prescreening and the Base Rate Fallacy

I’ve often written about the base rate fallacy and how it makes tests for rare events—like airplane terrorists—useless because the false positives vastly outnumber the real positives. This essay uses that argument to demonstrate why the TSA’s FAST program is useless:

First, predictive software of this kind is undermined by a simple statistical problem known as the false-positive paradox. Any system designed to spot terrorists before they commit an act of terrorism is, necessarily, looking for a needle in a haystack. As the adage would suggest, it turns out that this is an incredibly difficult thing to do. Here is why: let’s assume for a moment that 1 in 1,000,000 people is a terrorist about to commit a crime. Terrorists are actually probably much much more rare, or we would have a whole lot more acts of terrorism, given the daily throughput of the global transportation system. Now lets imagine the FAST algorithm correctly classifies 99.99 percent of observations—an incredibly high rate of accuracy for any big data-based predictive model. Even with this unbelievable level of accuracy, the system would still falsely accuse 99 people of being terrorists for every one terrorist it finds. Given that none of these people would have actually committed a terrorist act yet distinguishing the innocent false positives from the guilty might be a non-trivial, and invasive task.

Of course FAST has nowhere near a 99.99 percent accuracy rate. I imagine much of the work being done here is classified, but a writeup in Nature reported that the first round of field tests had a 70 percent accuracy rate. From the available material it is difficult to determine exactly what this number means. There are a couple of ways to interpret this, since both the write-up and the DHS documentation (all pdfs) are unclear. This might mean that the current iteration of FAST correctly classifies 70 percent of people it observes—which would produce false positives at an abysmal rate, given the rarity of terrorists in the population. The other way of interpreting this reported result is that FAST will call a terrorist a terrorist 70 percent of the time. This second option tells us nothing about the rate of false positives, but it would likely be quite high. In either case, it is likely that the false-positive paradox would be in full force for FAST, ensuring that any real terrorists identified are lost in a sea of falsely accused innocents.

It’s that final sentence in the first quoted paragraph that really points to how bad this idea is. If FAST determines you are guilty of a crime you have not yet committed, how do you exonerate yourself?

Posted on May 3, 2012 at 6:22 AM • View Comments

TSA Behavioral Detection Statistics

Interesting data from the U.S. Government Accountability Office:

But congressional auditors have questions about other efficiencies as well, like having 3,000 “behavior detection” officers assigned to question passengers. The officers sidetracked 50,000 passengers in 2010, resulting in the arrests of 300 passengers, the GAO found. None turned out to be terrorists.

Yet in the same year, behavior detection teams apparently let at least 16 individuals allegedly involved in six subsequent terror plots slip through eight different airports. GAO said the individuals moved through protected airports on at least 23 different occasions.

I don’t believe the second paragraph. We haven’t had six terror plots between 2010 and today. And even if we did, how would the auditors know? But I’m sure the first paragraph is correct: the behavioral detection program is 0% effective at preventing terrorism.

The rest of the article is pretty depressing. The TSA refuses to back down on any of its security theater measures. At the same time, its budget is being cut and more people are flying. The result: longer waiting times at security.

Posted on April 20, 2012 at 6:19 AM • View Comments

Hawley Channels His Inner Schneier

Kip Hawley wrote an essay for the Wall Street Journal on airport security. In it, he says so many sensible things that people have been forwarding it to me with comments like “did you ghostwrite this?” and “it looks like you won an argument” and “how did you convince him?”

(Sadly, the essay was published in the Journal, which means it won’t be freely available on the Internet forever. Because of that, I’m going to quote from it liberally. And if anyone finds a permanent URL for this, I’ll add it here.)

Hawley:

Any effort to rebuild TSA and get airport security right in the U.S. has to start with two basic principles:

First, the TSA’s mission is to prevent a catastrophic attack on the transportation system, not to ensure that every single passenger can avoid harm while traveling. Much of the friction in the system today results from rules that are direct responses to how we were attacked on 9/11. But it’s simply no longer the case that killing a few people on board a plane could lead to a hijacking. Never again will a terrorist be able to breach the cockpit simply with a box cutter or a knife. The cockpit doors have been reinforced, and passengers, flight crews and air marshals would intervene.

This sounds a lot like me (2005):

Exactly two things have made airline travel safer since 9/11: reinforcement of cockpit doors, and passengers who now know that they may have to fight back.

I’m less into sky marshals than he is.

Hawley:

Second, the TSA’s job is to manage risk, not to enforce regulations. Terrorists are adaptive, and we need to be adaptive, too. Regulations are always playing catch-up, because terrorists design their plots around the loopholes.

Me in 2008:

It’s this fetish-like focus on tactics that results in the security follies at airports. We ban guns and knives, and terrorists use box-cutters. We take away box-cutters and corkscrews, so they put explosives in their shoes. We screen shoes, so they use liquids. We take away liquids, and they’re going to do something else. Or they’ll ignore airplanes entirely and attack a school, church, theatre, stadium, shopping mall, airport terminal outside the security area, or any of the other places where people pack together tightly.

These are stupid games, so let’s stop playing.

He disses Trusted Traveler programs, where known people are allowed bypass some security measures:

I had hoped to advance the idea of a Registered Traveler program, but the second that you create a population of travelers who are considered “trusted,” that category of fliers moves to the top of al Qaeda’s training list, whether they are old, young, white, Asian, military, civilian, male or female. The men who bombed the London Underground in July 2005 would all have been eligible for the Registered Traveler cards we were developing at the time. No realistic amount of prescreening can alleviate this threat when al Qaeda is working to recruit “clean” agents. TSA dropped the idea on my watch—though new versions of it continue to pop up.

Me in 2004:

What the Trusted Traveler program does is create two different access paths into the airport: high security and low security. The intent is that only good guys will take the low-security path, and the bad guys will be forced to take the high-security path, but it rarely works out that way. You have to assume that the bad guys will find a way to take the low-security path.

Hawley’s essay ends with a list of recommendations for change, and they are mostly good:

What would a better system look like? If politicians gave the TSA some political cover, the agency could institute the following changes before the start of the summer travel season:

1. No more banned items: Aside from obvious weapons capable of fast, multiple killings—such as guns, toxins and explosive devices—it is time to end the TSA’s use of well-trained security officers as kindergarten teachers to millions of passengers a day. The list of banned items has created an “Easter-egg hunt” mentality at the TSA. Worse, banning certain items gives terrorists a complete list of what not to use in their next attack. Lighters are banned? The next attack will use an electric trigger.

Me in 2009:

Return passenger screening to pre-9/11 levels.

Hawley:

2. Allow all liquids: Simple checkpoint signage, a small software update and some traffic management are all that stand between you and bringing all your liquids on every U.S. flight. Really.

This is referring to a point he makes earlier in his essay:

I was initially against a ban on liquids as well, because I thought that, with proper briefing, TSA officers could stop al Qaeda’s new liquid bombs. Unfortunately, al Qaeda’s advancing skill with hydrogen-peroxide-based bombs made a total liquid ban necessary for a brief period and a restriction on the amount of liquid one could carry on a plane necessary thereafter.

Existing scanners could allow passengers to carry on any amount of liquid they want, so long as they put it in the gray bins. The scanners have yet to be used in this way because of concern for the large number of false alarms and delays that they could cause. When I left TSA in 2009, the plan was to designate “liquid lanes” where waits might be longer but passengers could board with snow globes, beauty products or booze. That plan is still sitting on someone’s desk.

I have been complaining about the liquids ban for years, but Hawley’s comment confuses me. He says that hydrogen-peroxide based bombs—these are the bombs that are too dangerous to bring on board in 4-oz. bottles, but perfectly fine in four 1-oz bottles combined after the checkpoints—can be detected with existing scanners, not with new scanners using new technology. Does anyone know what he’s talking about?

Hawley:

3. Give TSA officers more flexibility and rewards for initiative, and hold them accountable: No security agency on earth has the experience and pattern-recognition skills of TSA officers. We need to leverage that ability. TSA officers should have more discretion to interact with passengers and to work in looser teams throughout airports. And TSA’s leaders must be prepared to support initiative even when officers make mistakes. Currently, independence on the ground is more likely to lead to discipline than reward.

This is a great idea, but it’s going to cost money. Being a TSA screener is a pretty lousy job. Morale is poor: “In surveys on employee morale and job satisfaction, TSA often performs poorly compared to other government agencies. In 2010 TSA ranked 220 out of 224 government agency subcomponents for employee satisfaction.” Pay is low: “The men and women at the front lines of the battle to keep the skies safe are among the lowest paid of all federal employees, and they have one of the highest injury rates.” And there is traditionally a high turnover: 20% in 2008. The 2011 decision allowing TSA workers to unionize will help this somewhat, but for it to really work, the rules can’t be this limiting: “the paper outlining his decision precludes negotiations on security policies, pay, pensions and compensation, proficiency testing, job qualifications and discipline standards. It also will prohibit screeners from striking or engaging in work slowdowns.”

TSA workers who are smart, flexible, and show initiative will cost money, and that’ll be difficult when the TSA’s budget is being cut.

Hawley:

4. Eliminate baggage fees: Much of the pain at TSA checkpoints these days can be attributed to passengers overstuffing their carry-on luggage to avoid baggage fees. The airlines had their reasons for implementing these fees, but the result has been a checkpoint nightmare. Airlines might increase ticket prices slightly to compensate for the lost revenue, but the main impact would be that checkpoint screening for everybody will be faster and safer.

Another great idea, but I don’t see how we can do it without passing a law forbidding airlines to charge those fees. Over the past few years, airlines have drastically increased fees as a revenue source. Sneaking in extra charges allows them to advertise lower prices, and I don’t see that changing anytime soon.

5. Randomize security: Predictability is deadly. Banned-item lists, rigid protocols—if terrorists know what to expect at the airport, they have a greater chance of evading our system.

This would be a disaster. Actually, I’m surprised Hawley even mentions it, given that he wrote this a few paragraphs earlier:

One brilliant bit of streamlining from the consultants: It turned out that if the outline of two footprints was drawn on a mat in the area for using metal-detecting wands, most people stepped on the feet with no prompting and spread their legs in the most efficient stance. Every second counts when you’re processing thousands of passengers a day.

Randomization would slow checkpoints down to a crawl, as well as anger passengers. Do I have to take my shoes off or not? Does my computer go in the bin or not? (Even the weird but mostly consistent rules about laptops vs. iPads is annoying people.) Yesterday, liquids were allowed—today they’re banned. But at this airport, the TSA is confiscating anything with more than two ounces of aluminum and questioning people carrying Tom Clancy novels.

I’m not even convinced this would be a hardship for the terrorists. I’ve gotten really good at avoiding lanes with full-body scanners, and presumably the terrorists will simply assume that all security regulations are in force at all times. I’d like to see a cost-benefit analysis of this sort of thing first.

Hawley’s concluding paragraph:

In America, any successful attack—no matter how small—is likely to lead to a series of public recriminations and witch hunts. But security is a series of trade-offs. We’ve made it through the 10 years after 9/11 without another attack, something that was not a given. But no security system can be maintained over the long term without public support and cooperation. If Americans are ready to embrace risk, it is time to strike a new balance.

I agree with this. Sadly, I’m not optimistic for change anytime soon. There’s one point Hawley makes, but I don’t think he makes it strongly enough. He says:

I wanted to reduce the amount of time that officers spent searching for low-risk objects, but politics intervened at every turn. Lighters were untouchable, having been banned by an act of Congress. And despite the radically reduced risk that knives and box cutters presented in the post-9/11 world, allowing them back on board was considered too emotionally charged for the American public.

This is the fundamental political problem of airport security: it’s in nobody’s self-interest to take a stand for what might appear to be reduced security. Imagine that the TSA management announces a new rule that box cutters are now okay, and that they respond to critics by explaining that the current risks to airplanes don’t warrant prohibiting them. Even if they’re right, they’re open to attacks from political opponents that they’re not taking terrorism seriously enough. And if they’re wrong, their careers are over.

It’s even worse when it’s elected officials who have to make the decision. Which congressman is going to jeopardize his political career by standing up and saying that the cigarette lighter ban is stupid and should be repealed? It’s all political risk, and no political gain.

We have the same problem with the no-fly list: Congress mandates that the TSA match passengers against these lists. Rolling this back is politically difficult at the best of times, and impossible in today’s climate, even if the TSA decided it wanted to do so.

I am very impressed with Hawley’s essay. I do wonder where it came from. This wasn’t the same argument Hawley made when I debated him last month on the Economist website. This definitely wasn’t the same argument he made when I interviewed him in 2007, when he was still head of the TSA. But it’s great to read today.

Hopefully, someone is listening. And hopefully, our social climate will change so that these sorts of changes become politically possible.

ETA (4/16): Slashdot thread.

Posted on April 16, 2012 at 12:29 PM • View Comments

A Heathrow Airport Story about Trousers

Usually I don’t bother posting random stories about dumb or inconsistent airport security measures. But this one is particularly interesting:

“Sir, your trousers.”

“Pardon?”

“Sir, please take your trousers off.”

A pause.

“No.”

“No?”

The security official clearly was not expecting that response.

He begins to look like he doesn’t know what to do, bless him.

“You have no power to require me to do that. You also haven’t also given any good reason. I am sure any genuine security concerns you have can be addressed in other ways. You do not need to invade my privacy in this manner.”

A pause.

“I think you probably need to get your manager, don’t you?” I am trying to be helpful.

As I said in my Economist essay, “At this point, we don’t trust America’s TSA, Britain’s Department for Transport, or airport security in general.” We don’t trust that, when they tell us to do something and claim it’s essential for security, they’re tellling the truth.

Posted on April 11, 2012 at 9:57 AM • View Comments

JetBlue Captain Clayton Osbon and Resilient Security

This is the most intelligent thing I’ve read about the JetBlue incident where a pilot had a mental breakdown in the cockpit:

For decades, public safety officials and those who fund them have focused on training and equipment that has a dual-use function for any hazard that may come our way. The post-9/11 focus on terrorism, with all the gizmos that were bought in its name, was a moment of frenzy, and sometimes inconsistent with sound public policy. Over time, there was a return to security measures that were adaptable (dual or multiple use) to any threat and more sustainable in a world that has its fair share of both predictable and utterly bizarre events.

The mental condition of airline pilots is a relevant factor in their annual or bi-annual physicals. (FAA rules differ on the number of physicals required, based on the type of plane being flown.) But believing that the system is flawed because it didn’t predict the breakdown of one of 450,000 certified pilots is a myopic reaction.

In many ways, though, this kind of incident was anticipated. The system envisions pilot incapacitation—physical, mental, or possibly, as in the campy movie ”Snakes on a Plane,” a slithering foe.

That is, after all, why we have copilots.

The whole essay is worth reading.

Posted on April 5, 2012 at 6:19 AM • View Comments

Harms of Post-9/11 Airline Security

As I posted previously, I have been debating former TSA Administrator Kip Hawley on the Economist website. I didn’t bother reposting my opening statement and rebuttal, because—even though I thought I did a really good job with them—they were largely things I’ve said before. In my closing statement, I talked about specific harms post-9/11 airport security has caused. This is mostly new, so here it is, British spelling and punctuation and all.

In my previous two statements, I made two basic arguments about post-9/11 airport security. One, we are not doing the right things: the focus on airports at the expense of the broader threat is not making us safer. And two, the things we are doing are wrong: the specific security measures put in place since 9/11 do not work. Kip Hawley doesn’t argue with the specifics of my criticisms, but instead provides anecdotes and asks us to trust that airport security—and the Transportation Security Administration (TSA) in particular—knows what it’s doing.

He wants us to trust that a 400-ml bottle of liquid is dangerous, but transferring it to four 100-ml bottles magically makes it safe. He wants us to trust that the butter knives given to first-class passengers are nevertheless too dangerous to be taken through a security checkpoint. He wants us to trust the no-fly list: 21,000 people so dangerous they’re not allowed to fly, yet so innocent they can’t be arrested. He wants us to trust that the deployment of expensive full-body scanners has nothing to do with the fact that the former secretary of homeland security, Michael Chertoff, lobbies for one of the companies that makes them. He wants us to trust that there’s a reason to confiscate a cupcake (Las Vegas), a 3-inch plastic toy gun (London Gatwick), a purse with an embroidered gun on it (Norfolk, VA), a T-shirt with a picture of a gun on it (London Heathrow) and a plastic lightsaber that’s really a flashlight with a long cone on top (Dallas/Fort Worth).

At this point, we don’t trust America’s TSA, Britain’s Department for Transport, or airport security in general. We don’t believe they’re acting in the best interests of passengers. We suspect their actions are the result of politicians and government appointees making decisions based on their concerns about the security of their own careers if they don’t act tough on terror, and capitulating to public demands that “something must be done”.

In this final statement, I promised to discuss the broader societal harms of post-9/11 airport security. This loss of trust—in both airport security and counterterrorism policies in general—is the first harm. Trust is fundamental to society. There is an enormous amount written about this; high-trust societies are simply happier and more prosperous than low-trust societies. Trust is essential for both free markets and democracy. This is why open-government laws are so important; trust requires government transparency. The secret policies implemented by airport security harm society because of their very secrecy.

The humiliation, the dehumanisation and the privacy violations are also harms. That Mr Hawley dismisses these as mere “costs in convenience” demonstrates how out-of-touch the TSA is from the people it claims to be protecting. Additionally, there’s actual physical harm: the radiation from full-body scanners still not publicly tested for safety; and the mental harm suffered by both abuse survivors and children: the things screeners tell them as they touch their bodies are uncomfortably similar to what child molesters say.

In 2004, the average extra waiting time due to TSA procedures was 19.5 minutes per person. That’s a total economic loss—in –America—of $10 billion per year, more than the TSA’s entire budget. The increased automobile deaths due to people deciding to drive instead of fly is 500 per year. Both of these numbers are for America only, and by themselves demonstrate that post-9/11 airport security has done more harm than good.

The current TSA measures create an even greater harm: loss of liberty. Airports are effectively rights-free zones. Security officers have enormous power over you as a passenger. You have limited rights to refuse a search. Your possessions can be confiscated. You cannot make jokes, or wear clothing, that airport security does not approve of. You cannot travel anonymously. (Remember when we would mock Soviet-style “show me your papers” societies? That we’ve become inured to the very practice is a harm.) And if you’re on a certain secret list, you cannot fly, and you enter a Kafkaesque world where you cannot face your accuser, protest your innocence, clear your name, or even get confirmation from the government that someone, somewhere, has judged you guilty. These police powers would be illegal anywhere but in an airport, and we are all harmed—individually and collectively—by their existence.

In his first statement, Mr Hawley related a quote predicting “blood running in the aisles” if small scissors and tools were allowed on planes. That was said by Corey Caldwell, an Association of Flight Attendants spokesman, in 2005. It was not the statement of someone who is thinking rationally about airport security; it was the voice of irrational fear.

Increased fear is the final harm, and its effects are both emotional and physical. By sowing mistrust, by stripping us of our privacy—and in many cases our dignity—by taking away our rights, by subjecting us to arbitrary and irrational rules, and by constantly reminding us that this is the only thing between us and death by the hands of terrorists, the TSA and its ilk are sowing fear. And by doing so, they are playing directly into the terrorists’ hands.

The goal of terrorism is not to crash planes, or even to kill people; the goal of terrorism is to cause terror. Liquid bombs, PETN, planes as missiles: these are all tactics designed to cause terror by killing innocents. But terrorists can only do so much. They cannot take away our freedoms. They cannot reduce our liberties. They cannot, by themselves, cause that much terror. It’s our reaction to terrorism that determines whether or not their actions are ultimately successful. That we allow governments to do these things to us—to effectively do the terrorists’ job for them—is the greatest harm of all.

Return airport security checkpoints to pre-9/11 levels. Get rid of everything that isn’t needed to protect against random amateur terrorists and won’t work against professional al-Qaeda plots. Take the savings thus earned and invest them in investigation, intelligence, and emergency response: security outside the airport, security that does not require us to play guessing games about plots. Recognise that 100% safety is impossible, and also that terrorism is not an “existential threat” to our way of life. Respond to terrorism not with fear but with indomitability. Refuse to be terrorized.

EDITED TO ADD (3/20): Cory Doctorow on the exchange:

All of Hawley’s best arguments sum up to “Someone somewhere did something bad, and if he’d tried it on us, we would have caught him.” His closing clincher? They heard a bad guy was getting on a plane somewhere. The figured out which plane, stopped it from taking off and “resolved” the situation. Seeing as there were no recent reports of foiled terrorist plots, I’m guessing the “resolution” was “it turned out we made a mistake.” But Hawley’s takeaway is: “look at how fast our mistake was!”

EDITED TO ADD (4/19): German translation of the closing statement.

Posted on March 29, 2012 at 6:53 AM • View Comments

Congressional Testimony on the TSA

I was supposed to testify today about the TSA in front of the House Committee on Oversight and Government Reform. I was informally invited a couple of weeks ago, and formally invited last Tuesday:

The hearing will examine the successes and challenges associated with Advanced Imaging Technology (AIT), the Screening of Passengers by Observation Techniques (SPOT) program, the Transportation Worker Credential Card (TWIC), and other security initiatives administered by the TSA.

On Friday, at the request of the TSA, I was removed from the witness list. The excuse was that I am involved in a lawsuit against the TSA, trying to get them to suspend their full-body scanner program. But it’s pretty clear that the TSA is afraid of public testimony on the topic, and especially of being challenged in front of Congress. They want to control the story, and it’s easier for them to do that if I’m not sitting next to them pointing out all the holes in their position. Unfortunately, the committee went along with them. (They tried to pull the same thing last year and it failed—video at the 10:50 mark.)

The committee said it would try to invite me back for another hearing, but with my busy schedule, I don’t know if I will be able to make it. And it would be far less effective for me to testify without forcing the TSA to respond to my points.

I’m there in spirit, though. The title of the hearing is “TSA Oversight Part III: Effective Security or Security Theater?”

Posted on March 26, 2012 at 1:02 PM • View Comments

The Economist Debate on Airplane Security

On The Economist website, I am currently debating Kip Hawley on airplane security. On Tuesday we posted our initial statements, and today (London time) we posted our rebuttals. We have one more round to go.

I’ve set it up to talk about the myriad of harms airport security has caused: loss of trust in government, increased fear, creeping police state, loss of liberty in the “rights free zone,” and so on. Suggestions of what to say next are appreciated.

Posted on March 23, 2012 at 6:33 AM • View Comments

Australian Security Theater

I like the quote at the end of this excerpt:

Aviation officials have questioned the need for such a strong permanent police presence at airports, suggesting they were there simply “to make the government look tough on terror”.

One senior executive said in his experience, the officers were expensive window-dressing.

“When you add the body scanners, the ritual humiliation of old ladies with knitting needles and the farcical air marshals, it all adds up to billions of dollars to prevent what? A politician being called soft on terror, that’s what,” he said.

Posted on March 19, 2012 at 6:38 AM • View Comments

1 … 7 8 9 10 11 … 46

Sidebar photo of Bruce Schneier by Joe MacInnis.