Entries Tagged "behavioral detection"

Page 2 of 4

Detecting People Who Want to Do Harm

I’m dubious:

At a demonstration of the technology this week, project manager Robert P. Burns said the idea is to track a set of involuntary physiological reactions that might slip by a human observer. These occur when a person harbors malicious intent—but not when someone is late for a flight or annoyed by something else, he said, citing years of research into the psychology of deception.

The development team is investigating how effective its techniques are at flagging only people who intend to do harm. Even if it works, the technology raises a slew of questions – from privacy concerns, to the more fundamental issue of whether machines are up to a task now entrusted to humans.

I have a lot of respect for Paul Ekman’s opinion on the matter:

“I can understand why there’s an attempt being made to find a way to replace or improve on what human observers can do: the need is vast, for a country as large and porous as we are. However, I’m by no means convinced that any technology, any hardware will come close to doing what a highly trained human observer can do,'” said Ekman, who directs a company that trains government workers, including for the Transportation Security Administration, to detect suspicious behavior.

Posted on October 7, 2009 at 12:54 PM • View Comments

Lie Detector Charlatans

This is worth reading:

Five years ago I wrote a Language Log post entitled “BS conditional semantics and the Pinocchio effect” about the nonsense spouted by a lie detection company, Nemesysco. I was disturbed by the marketing literature of the company, which suggested a 98% success rate in detecting evil intent of airline passengers, and included crap like this:

The LVA uses a patented and unique technology to detect “Brain activity finger prints” using the voice as a “medium” to the brain and analyzes the complete emotional structure of your subject. Using wide range spectrum analysis and micro-changes in the speech waveform itself (not micro tremors!) we can learn about any anomaly in the brain activity, and furthermore, classify it accordingly. Stress (“fight or flight” paradigm) is only a small part of this emotional structure

The 98% figure, as I pointed out, and as Mark Liberman made even clearer in a follow up post, is meaningless. There is no type of lie detector in existence whose performance can reasonably be compared to the performance of finger printing. It is meaningless to talk about someone’s “complete emotional structure”, and there is no interesting sense in which any current technology can analyze it. It is not the case that looking at speech will provide information about “any anomaly in the brain activity”: at most it will tell you about some anomalies. Oh, the delicious irony, a lie detector company that engages in wanton deception.

So, ok, Nemesysco, as I said in my earlier post, is clearly trying to pull the wool over people’s eyes. Disturbing, yes, but it doesn’t follow from the fact that its marketing is wildly misleading that the company’s technology is of no merit. However, we now know that the company’s technology is, in fact, of no merit. How do we know? Because two phoneticians, Anders Eriksson and Francisco Lacerda, studied the company’s technology, based largely on the original patent, and and provided a thorough analysis in a 2007 article Charlatanry in forensic speech science: A problem to be taken seriously, which appeared in the International Journal of Speech Language and the Law (IJSLL), vol 14.2 2007, 169–­193, Equinox Publishing. Eriksson and Lacerda conclude, regarding the original technology on which Nemesysco’s products are based, Layered Voice Analysis (LVA), that:

Any qualified speech scientist with some computer background can see at a glance, by consulting the documents, that the methods on which the program is based have no scientific validity.

Most of the lie detector industry is based on, well, lies.

EDITED TO ADD (5/13): The paper is available here. More details here. Nemesyco’s systems are being used to bully people out of receiving government aid in the UK.

Posted on May 6, 2009 at 12:14 PM • View Comments

Ed Felten on TSA Behavioral Screening

Good comment:

Now suppose that TSA head Kip Hawley came to you and asked you to submit voluntarily to a pat-down search the next time you travel. And suppose you knew, with complete certainty, that if you agreed to the search, this would magically give the TSA a 0.1% chance of stopping a deadly crime. You’d agree to the search, wouldn’t you? Any reasonable person would accept the search to save (by assumption) at least 0.001 lives. This hypothetical TSA program is reasonable, even though it only has a 0.1% arrest rate. (I’m assuming here that an attack would cost only one life. Attacks that killed more people would justify searches with an even smaller arrest rate.)

So the commentators’ critique is weak—but of course this doesn’t mean the TSA program should be seen as a success. The article says that the arrests the system generates are mostly for drug charges or carrying a false ID. Should a false-ID arrest be considered a success for the system? Certainly we don’t want to condone the use of false ID, but I’d bet most of these people are just trying to save money by flying on a ticket in another person’s name—which hardly makes them Public Enemy Number One. Is it really worth doing hundreds of searches to catch one such person? Are those searches really the best use of TSA screeners’ time? Probably not.

Right. It’s not just about the hit rate. It’s the cost vs. benefit: cost in taxpayer money, passenger time, TSA screener attention, fundamental liberties, etc.

Posted on December 17, 2008 at 6:38 AM • View Comments

Kip Hawley Responds to My Airport Security Antics

Kip Hawley, head of the TSA, has responded to my airport security penetration testing, published in The Atlantic.

Unfortunately, there’s not really anything to his response. It’s obvious he doesn’t want to admit that they’ve been checking ID’s all this time to no purpose whatsoever, so he just emits vague generalities like a frightened squid filling the water with ink. Yes, some of the stunts in article are silly (who cares if people fly with Hezbollah T-shirts?) so that gives him an opportunity to minimize the real issues.

Watch-lists and identity checks are important and effective security measures. We identify dozens of terrorist-related individuals a week and stop No-Flys regularly with our watch-list process.

It is simply impossible that the TSA catches dozens of terrorists every week. If it were true, the administration would be trumpeting this all over the press—it would be an amazing success story in their war on terrorism. But note that Hawley doesn’t exactly say that; he calls them “terrorist-related individuals.” Which means exactly what? People so dangerous they can’t be allowed to fly for any reason, yet so innocent they can’t be arrested—even under the provisions of the Patriot Act.

And if Secretary Chertoff is telling the truth when he says that there are only 2,500 people on the no-fly list and fewer than 16,000 people on the selectee list—they’re the ones that get extra screening—and that most of them live outside the U.S., then it is just plain impossible that the TSA identifies “dozens” of these people every week. The math just doesn’t make sense.

And I also don’t believe this:

Behavior detection works and we have 2,000 trained officers at airports today. They alert us to people who may pose a threat but who may also have items that could elude other layers of physical security.

It does work, but I don’t see the TSA doing it properly. (Fly El Al if you want to see it done properly.) But what I think Hawley is doing is engaging in a little bit of psychological manipulation. Like sky marshals, the real benefit of behavior detection isn’t whether or not you do it but whether or not the bad guys believe you’re doing it. If they think you are doing behavior detection at security checkpoints, or have sky marshals on every airplane, then you don’t actually have to do it. It’s the threat that’s the deterrent, not the actual security system.

This doesn’t impress me, either:

Items carried on the person, be they a ‘beer belly’ or concealed objects in very private areas, are why we are buying over 100 whole body imagers in upcoming months and will deploy more over time. In the meantime, we use hand-held devices that detect hydrogen peroxide and other explosives compounds as well as targeted pat-downs that require private screening.

Optional security measures don’t work, because the bad guys will opt not to use them. It’s like those air-puff machines at some airports now. They’re probably great at detecting explosive residue off clothing, but every time I have seen the machines in operation, the passengers have the option whether to go through the lane with them or another lane. What possible good is that?

The closest thing to a real response from Hawley is that the terrorists might get caught stealing credit cards.

Using stolen credit cards and false documents as a way to get around watch-lists makes the point that forcing terrorists to use increasingly risky tactics has its own security value.

He’s right about that. And, truth be told, that was my sloppiest answer during the original interview. Thinking about it afterwards, it’s far more likely is that someone with a clean record and a legal credit card will buy the various plane tickets.

This is new:

Boarding pass scanners and encryption are being tested in eight airports now and more will be coming.

Ignoring for a moment that “eight airports” nonsense—unless you do it at every airport, the bad guys will choose the airport where you don’t do it to launch their attack—this is an excellent idea. The reason my attack works, the reason I can get through TSA checkpoints with a fake boarding pass, is that the TSA never confirms that the information on the boarding pass matches a legitimate reservation. If all TSA checkpoints had boarding pass scanners that connected to the airlines’ computers, this attack would not work. (Interestingly enough, I noticed exactly this system at the Dublin airport earlier this month.)

Stopping the “James Bond” terrorist is truly a team effort and I whole-heartedly agree that the best way to stop those attacks is with intelligence and law enforcement working together.

This isn’t about “Stopping the ‘James Bond’ terrorist,” it’s about stopping terrorism. And if all this focus on airports, even assuming it starts working, shifts the terrorists to other targets, we haven’t gotten a whole lot of security for our money.

FYI: I did a long interview with Kip Hawley last year. If you haven’t read it, I strongly recommend you do. I pressed him on these and many other points, and didn’t get very good answers then, either.

EDITED TO ADD (10/28): Kip Hawley responds in comments. Yes, it’s him.

EDITED TO ADD (11/17): Another article on those boarding pass verifiers.

Posted on October 23, 2008 at 6:24 AM • View Comments

Me Helping Evade Airport Security

Great article from The Atlantic:

As we stood at an airport Starbucks, Schneier spread before me a batch of fabricated boarding passes for Northwest Airlines flight 1714, scheduled to depart at 2:20 p.m. and arrive at Reagan National at 5:47 p.m. He had taken the liberty of upgrading us to first class, and had even granted me “Platinum/Elite Plus” status, which was gracious of him. This status would allow us to skip the ranks of hoi-polloi flyers and join the expedited line, which is my preference, because those knotty, teeming security lines are the most dangerous places in airports: terrorists could paralyze U.S. aviation merely by detonating a bomb at any security checkpoint, all of which are, of course, entirely unsecured. (I once asked Michael Chertoff, the secretary of Homeland Security, about this. “We actually ultimately do have a vision of trying to move the security checkpoint away from the gate, deeper into the airport itself, but there’s always going to be some place that people congregate. So if you’re asking me, is there any way to protect against a person taking a bomb into a crowded location and blowing it up, the answer is no.”)

Schneier and I walked to the security checkpoint. “Counterterrorism in the airport is a show designed to make people feel better,” he said. “Only two things have made flying safer: the reinforcement of cockpit doors, and the fact that passengers know now to resist hijackers.” This assumes, of course, that al-Qaeda will target airplanes for hijacking, or target aviation at all. “We defend against what the terrorists did last week,” Schneier said. He believes that the country would be just as safe as it is today if airport security were rolled back to pre-9/11 levels. “Spend the rest of your money on intelligence, investigations, and emergency response.”

Schneier and I joined the line with our ersatz boarding passes. “Technically we could get arrested for this,” he said, but we judged the risk to be acceptable. We handed our boarding passes and IDs to the security officer, who inspected our driver’s licenses through a loupe, one of those magnifying-glass devices jewelers use for minute examinations of fine detail. This was the moment of maximum peril, not because the boarding passes were flawed, but because the TSA now trains its officers in the science of behavior detection. The SPOT program—Screening of Passengers by Observation Techniques—was based in part on the work of a psychologist who believes that involuntary facial-muscle movements, including the most fleeting “micro-expressions,” can betray lying or criminality. The training program for behavior-detection officers is one week long. Our facial muscles did not cooperate with the SPOT program, apparently, because the officer chicken-scratched onto our boarding passes what might have been his signature, or the number 4, or the letter y. We took our shoes off and placed our laptops in bins. Schneier took from his bag a 12-ounce container labeled “saline solution.”

“It’s allowed,” he said. Medical supplies, such as saline solution for contact-lens cleaning, don’t fall under the TSA’s three-ounce rule.

“What’s allowed?” I asked. “Saline solution, or bottles labeled saline solution?”

“Bottles labeled saline solution. They won’t check what’s in it, trust me.”

They did not check. As we gathered our belongings, Schneier held up the bottle and said to the nearest security officer, “This is okay, right?” “Yep,” the officer said. “Just have to put it in the tray.”

“Maybe if you lit it on fire, he’d pay attention,” I said, risking arrest for making a joke at airport security. (Later, Schneier would carry two bottles labeled saline solution—24 ounces in total—through security. An officer asked him why he needed two bottles. “Two eyes,” he said. He was allowed to keep the bottles.)

Posted on October 16, 2008 at 4:32 PM • View Comments

More on Airplane Seat Cameras

I already blogged this once: an airplane-seat camera system that tries to detect terrorists before they leap up and do whatever they were planning on doing. Amazingly enough, the EU is “testing” this system:

Each camera tracks passengers’ facial expressions, with the footage then analysed by software to detect developing terrorist activity or potential air rage. Six wide-angle cameras are also positioned to monitor the plane’s aisles, presumably to catch anyone standing by the cockpit door with a suspiciously crusty bread roll.

But since people never sit still on planes, the software’s also designed so that footage from multiple cameras can be analysed. So, if one person continually walks from his seat to the bathroom, then several cameras can be used to track his facial movements.

The software watches for all sorts of other terrorist-like activities too, including running in the cabin, someone nervously touching their face or excessive sweating. An innocent nose scratch won’t see the F16s scrambled, but a combination of several threat indicators could trigger a red alert.

This pegs the stupid meter. All it will do is false alarm. No one has any idea what sorts of facial characteristics are unique to terrorists. And how in the world are they “testing” this system without any real terrorists? In any case, what happens when the alarm goes off? How exactly is a ten-second warning going to save people?

Sure, you can invent a terrorist tactic where a system like this, assuming it actually works, saves people—but that’s the very definition of a movie-plot threat. How about we spend this money on something that’s effective in more than just a few carefully chosen scenarios?

Posted on June 4, 2008 at 12:05 PM • View Comments

Would-Be Bomber Caught at Orlando Airport

Oddly enough, I flew into Orlando Airport on Tuesday night, hours after TSA and police caught Kevin Brown—not the baseball player—with bomb-making equipment in his checked luggage. (Yes, checked luggage. He was bringing it to Jamaica, not planning on blowing up the plane he was on.) Seems like someone trained in behavioral profiling singled him out, probably for stuff like this:

“He was rocking left to right, bouncing up and down … he was there acting crazy,” passenger Jason Doyle said.

But that was a passenger remembering Brown after the fact, so I wouldn’t put too much credence in it.

There are a bunch of articles about Brown and potential motives. Note that he is not an Islamic terrorist; he’s a U.S. Army veteran who served in Iraq:

“This is not him,” she said in a phone interview. “It has to be a mental issue for him. I know if they looked through his medical records…I’m sure they will see…”He’s not a terrorist.”

Brown married Holt’s daughter, Kamishia, 25, about three years ago. They met while serving in the Army and separated a year later. Brown wasn’t the same after returning from Iraq, her daughter told her.

“When he doesn’t take it [medication], he’s off the chain,” Holt said. “When you don’t take it and drink alcohol, it makes it worse.”

Doesn’t sound like a terrorist, but this does:

According to the affidavit, Brown admitted he had the items because he wanted to make pipe bombs in Jamaica. It also indicated he wanted to show friends how to make pipe bombs like he made while in Iraq.

Federal agents said federal agents found two vodka bottles filled with nitro-methane, a highly explosive liquid, as well as galvanized pipes, end caps with holes, BBs, a model-rocket igniter, AA batteries, a lighter and lighter fluid, plus other items used to make pipe bombs and detailed instructions and diagrams. He indicated the items were purchased in Gainesville where he lived at one time.

Ignore the hyperbole; nitromethane is a liquid fuel, not a high explosive. Here’s the whole affidavit, if you want to read it.

Even with all this news, the truth is that we just don’t know what happened. It looks like a great win for behavioral profiling (which, when done well, I think is a good idea) and the TSA. The TSA is certainly pleased. But we’ve seen apparent TSA wins before that turn out to be bogus when the details finally come out. Right now I’m cautiously pleased with the TSA’s performance, and offer them a tentative congratulations, especially for not over-reacting. I read—but can’t find the link now—that only 11 flights were delayed because of the event. The TSA claims that no flights were delayed, and also says that no security checkpoints were closed. Either way, it’s certainly something to congratulate the TSA about.

Posted on April 3, 2008 at 9:02 AM • View Comments

Airport Behavioral Profiling Leads to an Arrest

I’m generally a fan of behavioral profiling. While it sounds weird and creepy and has been likened to Orwell’s “facecrime”, there’s no doubt that—when done properly—it works at catching common criminals:

On Dec. 4, Juan Carlos Berriel-Castillo, 22, and Bernardo Carmona-Olivares, 20, were planning to fly to Maui but were instead arrested on suspicion of forgery.

They tried to pass through a Terminal 4 security checkpoint with suspicious documents, Phoenix police spokeswoman Stacie Derge said.

The pair had false permanent-resident identification, and authorities also found false Social Security cards, officials say.

While the pair were questioned about the papers, a TSA official who had received behavior-recognition training observed a third man in the area who appeared to be connected to Berriel-Castillo and Carmona-Olivares, Melendez said.

As a result, police later arrested Samuel Gonzalez, 32. A background check revealed that Gonzalez was wanted on two misdemeanor warrants.

TSA press release here.

Security is a trade-off. The question is whether the expense of the Screening Passengers by Observation Techniques (SPOT) program, given the minor criminals it catches, is worth it. (Remember, it’s supposed to catch terrorists, not people with outstanding misdemeanor warrants.) Especially with the 99% false alarm rate:

Since January 2006, behavior-detection officers have referred about 70,000 people for secondary screening, Maccario said. Of those, about 600 to 700 were arrested on a variety of charges, including possession of drugs, weapons violations and outstanding warrants.

And the other social costs, including loss of liberty, restriction of fundamental freedoms, and the creation of a thoughtcrime. Is this the sort of power we want to give a police force in a constitutional democracy, or does it feel more like a police-state sort of thing?

This “Bizarro” cartoon sums it up nicely.

Posted on January 3, 2008 at 12:49 PM • View Comments

More Behavioral Profiling

I’ve seen several articles based on this press release:

Computer and behavioral scientists at the University at Buffalo are developing automated systems that track faces, voices, bodies and other biometrics against scientifically tested behavioral indicators to provide a numerical score of the likelihood that an individual may be about to commit a terrorist act.

I am generally in favor of funding all sorts of research, no matter how outlandish—you never know when you’ll discover something really good—and I am generally in favor of this sort of behavioral assessment profiling.

But I wish reporters would approach these topics with something resembling skepticism. The false-positive rate matters far more than the false-negative rate, and I doubt something like this will be ready for fielding any time soon.

EDITED TO ADD (10/13): Another comment.

Posted on October 15, 2007 at 6:16 AM • View Comments

1 2 3 4

Sidebar photo of Bruce Schneier by Joe MacInnis.