Schneier on Security
A blog covering security and security technology.
« Friday Squid Blogging: Polymer Clay Squid Ornament |
| Protecting Cars with The Club »
June 14, 2010
Behavioral Profiling at Airports
There's a long article in Nature on the practice:
It remains unclear what the officers found anomalous about George's behaviour, and why he was detained. The TSA's parent agency, the Department of Homeland Security (DHS), has declined to comment on his case because it is the subject of a federal lawsuit that was filed on George's behalf in February by the American Civil Liberties Union. But the incident has brought renewed attention to a burgeoning controversy: is it possible to know whether people are being deceptive, or planning hostile acts, just by observing them?
Some people seem to think so. At London's Heathrow Airport, for example, the UK government is deploying behaviour-detection officers in a trial modelled in part on SPOT. And in the United States, the DHS is pursuing a programme that would use sensors to look at nonverbal behaviours, and thereby spot terrorists as they walk through a corridor. The US Department of Defense and intelligence agencies have expressed interest in similar ideas.
Yet a growing number of researchers are dubious not just about the projects themselves, but about the science on which they are based. "Simply put, people (including professional lie-catchers with extensive experience of assessing veracity) would achieve similar hit rates if they flipped a coin," noted a 2007 report from a committee of credibility-assessment experts who reviewed research on portal screening.
"No scientific evidence exists to support the detection or inference of future behaviour, including intent," declares a 2008 report prepared by the JASON defence advisory group. And the TSA had no business deploying SPOT across the nation's airports "without first validating the scientific basis for identifying suspicious passengers in an airport environment", stated a two-year review of the programme released on 20 May by the Government Accountability Office (GAO), the investigative arm of the US Congress.
Commentary from the MindHacks blog.
Also, the GAO has published a report on the U.S. DHS's SPOT program: "Aviation Security: Efforts to Validate TSA’s Passenger Screening Behavior Detection Program Underway, but Opportunities Exist to Strengthen Validation and Address Operational Challenges."
As of March 2010, TSA deployed about 3,000 BDOs at an annual cost of about $212 million; this force increased almost fifteen-fold between March 2007 and July 2009. BDOs have been selectively deployed to 161 of the 457 TSA-regulated airports in the United States at which passengers and their property are subject to TSA-mandated screening procedures.
It seems pretty clear that the program only catches criminals, and no terrorists. You'd think there would be more important things to spend $200 million a year on.
EDITED TO ADD (6/14): In the comments, a couple of people asked how this compares with the Israeli model of airport security -- concentrate on the person -- and the idea that trained officers notice if someone is acting "hinky": both things that I have written favorably about.
The difference is the experience of the detecting officer and the amount of time they spend with each person. If you read about the programs described above, they're supposed to "spot terrorists as they walk through a corridor," or possibly after a few questions. That's very different from what happens when you check into a flight an Ben Gurion Airport.
The problem with fast detection programs is that they don't work, and the problem with the Israeli security model is that it doesn't scale.
Posted on June 14, 2010 at 6:23 AM
• 70 Comments
To receive these entries once a month by e-mail, sign up for the Crypto-Gram Newsletter.
But isn't this very close to what your usual solution is? That we need guards with training that can spot people acting hinky?
There's probably some distinction between what you describe as flexible and adaptive investigation on one hand and war on the unexpected on the other that I miss. Could you please elaborate on that some time?
There's a difference between a trained guard - note, I said trained, whereas few are - who can get suspicious with a specific confrontation with a specific individual in a specific set of circumstances, and the notion of poorly trained guards playing "amateur psychiatrist" with a line of PO'd people waiting to get on a plane.
Well OF COURSE they don't catch any terrorists. Terrorists are very rare. Even if we their system was extraordinary effective, and that false positives represent only 1% of those detained, the number of people needlessly detained would outnumber terrorists something like 1,000 to 1. The question is, does this sort of think dissuade any terrorists. I suspect not, but that is the argument to be made.
Scientists using their early work to produce salable products and contracting fees so justify no longer following the scientific method?
Even our nuke scientists publish.
So in order to justify the SPOT program TSA is contracted it's own 'study'? Something of a interest conflict isn't there?
If NO terrorist has ever been apprehended does it remain 'reasonable', in a 4th ammendment sense, to search people's intent? Either through SPOT or FAST. The argument DHS seems to be forming is 'our technology isn't good enough.' No it's not. And public strip searches weren't 'reasonable' until they development of millimeter scanner; but now they are? Reasonableness isn't a function or criteria of capability.
1,000 criminals apprehended. So why isn't it reasonable to set up checkpoints along public highways or boarding trains to look, not for terrorists, but for criminals?
Me I'm for properly setup indepth random screening at around the 10% level.
However it has to be "properly random" and thus not gamable by either side.
And this has it's own problems (have a read up on que theory in manufacture and OS design).
It is likley to give a much better bang for the buck as it would have a very good chance of catching "group attacks" of more than three people.
A group of less than three going through ordinary screening will have very limited capabilities that so far the passangers have been able to deal with.
Not perfect but probably better value.
I wonder if this would work in court. Next time I get jury duty, I'm going to tell the judge that I've read up on all of the latest behavior detection technologies and I'll be able to tell if the defendant is guilty just by looking at them.
Malcolm Gladwell did a New Yorker article on how the criminal profiling done by the FBI (lionized in multiple books, inspiration for endless movies and TV shows) is basically BS and no better than random guessing.
Quote:"Brussel did not really understand the mind of the Mad Bomber. He seems to have understood only that, if you make a great number of predictions, the ones that were wrong will soon be forgotten, and the ones that turn out to be true will make you famous. The Hedunit is not a triumph of forensic analysis. It’s a party trick."
Via googling: Malcolm Gladwell profiling
@ Mary Arrrr
It pains me to say this but I feel Malcom Gladwell is spot on here. (And normally I think he is full of crap)
Its nice to see that there are still massive revenue streams available, even in a supposed recession, for parlour tricks.
Now, I just need a moral bypass so I can sell snake oil to the UK Government. I am not greedy, I wont ask for $212m. If they pay me £10m a year I will be happy and I will detect just as many terrorists and other BADPEOPLE.....
Now, how do I submit the contract....
You yourself were positive on the Israeli use of exactly this kind if thing, an informed and trained one on one check on each passenger.
This is not an example of the "Hinky" test. There's an important difference.
When it comes to screening for bad intent, The U.S. government is fond of scientifically-dubious magic bullets. Usually if favors technological magic (polygraphs, in various styles), but occasionally, as in this case, it comes up with "procedural" magic: we'll just train people to recognize the "scientific" signs of bad intent.
This is very different from leveraging the long experience of law-enforcement professionals, who over time have developed a set of sensitivities (some conscious, some less-so), as well as the ability to probe effectively through conversation and casual questioning. These tools can make such professionals unusually effective in picking out likely intent problems.
Here we're just trying to short-circuit the experience-building phase that makes such professionals so effective. We're going to use the "science" of intent-reading to simply stamp such experts out of the ground in large numbers, until we have an army of them large enough to screen the 700,000,000 passengers passing through U.S. airports every year.
It's a pipe dream, if for no other reason that the U.S. government places no requirement for proper scientific validation on any of the magic bullets that it solicits from vendors eager to sip at the public trough. So even if there were an authentic science of intent reading (which I rather doubt, though I could be wrong), the scientifically corrupted culture encouraged by the Feds' hypnotized fascination with magic technology guarantees that such science will never inform any product that they buy into.
Interesting. That said, Bruce - haven't you always been in favor of this kind of profiling - the whole "he was acting hinky" thing?
(I'm not critizicing that, mind you. I'm just curious about what your thoughts are on the whole thing - if your opinion changed, if so, why, if not, why not, and all that..)
I have to say that I agree with the other commenters: you have typically come down in favour of Behavioural Profiling before. What has changed, or what subtleties are we (or the TSA/DHS) missing in this?
The whole premise of behavioral profiling/screening is based on the assumption that people may give away their feelings of guilt or deception through "micro-expressions".
This works for morally and ethically honest people.
However, if someone has a sociopathic defect, do you think he will externalize his true intent?
If I remember correctly Bruce has in the past answered the question about the difference between hinky (the gut feeling based on long experiance) and some types of behavioural profiling based on questionable or faux science/mathmatics.
I think you will find Bruce has never rulled out sound science or mathmatics providing it can be shown to have an advantage over other techniques.
Unfortunatly nearly all the systems that are getting funding are realy "butt protection" for the TSA they can blaim the faux science when somebody drags them through the courts.
As I said above about True zrandom Sampling for indepth searching etc you have to be very carefull how you do it to avoid either side gaming it for their own advantage.
So even scientificaly sound systems can fail due to incorect or poor implementation.
It seems to me that there should be a very easy way to determine whether these systems work or not.
Test them with real criminals.
If Ekman's process is legit, he should be able to tell what crime the criminal committed that landed him in jail.
And he should be able to identify non-criminals used as controls in the group.
@nobody: "The whole premise of behavioral profiling/screening is based on the assumption that people may give away their feelings of guilt or deception through "micro-expressions". This works for morally and ethically honest people. However, if someone has a sociopathic defect, do you think he will externalize his true intent?
I think that's part of the problem with the behavioral aspect, with another component being that people can be trained to blend in and avoid looking suspicious. I do it all the time, in physical security audits I dress the part, carry a clipboard, look like I belong, and see who questions me and where I can get.
Probably the best solution is layered solutions at airports. Reasonable screening to limit the materials they can use (and by reasonable, I mean they should drop the shoe bit), and they should pay attention to people that look hinky. Neither layer is perfect, but it is tougher to bypass two layers than one.
Then, of course, the reinforced cockpit doors and passenger knowledge to right back provide another layer. Quite a few things have to go right for an attacker to pull off an attack, and he'll have to be pretty crafty with limited resources. Movie plots in this area are more suited for McGyver than real life threats.
One thing I'll feel I should point out is the possibility of lawsuits introduced by the behavioral profiling. Of course, it should be done, but not over relied upon. But it can be exploited by someone who will deliberately act "hinky" just to set up a discrimination lawsuit.
I remember a man who did this in wealthy LA (i think) neighborhoods several years ago. I heard him interviewed on TV, and this was a very intelligent and educated man. His intelligence was surprising based on his appearance...he looked mean and dangerous, deliberately by the way he dressed, groomed, and carried himself. What he would do is hang out and make some neighbors nervous...he clearly looked like he didn't belong, then he would file a racial discrimination lawsuit when the police would question him.
To be honest, I don't know how the lawsuits turned out, but I can see airport security being targetted a similar way if they over-rely on behavioral profiling. Behave in a calculated manner to get their attention, then say they are only doing it because I am [insert a characteristic here].
Just a thought.
@Mary Arrrr "criminal profiling done by the FBI ... is basically BS and no better than random guessing. "
A most effective control on social behavior is the general belief in the invulnerability of the man. We carry this model around in our head like a little police officer. We feel that someone is watching over our shoulder and will catch us when we step out of line so we don't. This is of course magical thinking that has to be periodically and randomly reinforced. So the FBI deliberately spends time perpetrating it's mythology.
America's most wanted often makes the claim that people have turned themselves in just hearing that the show profiled them. If true, power of myth.
Some people start with the idea that police are just ordinary people who can be out run or out gunned. This makes them dangerous individually. But they can be brought down individually. It's when masses of people decide that law enforcers are just ordinary people, few, badly resourced easily overcome, that we should start to sweat. This is why public leaders lying as a matter of policy is so bad. First it pisses people off giving them an emotional energy and then adopt the leaders actions as a reason to act out. And the usual people people act out on are their neighbors and when the law get's in the way of a group of concerned citizens intent on burning a witch; it can go badly.
I liked this article even though it started with a story. Stories and story telling is what we hear on the news everyday. It engages our feelings of sympathy and revulsion. Our normal response is "aw poor people ...". the problem with stories is it doesn't matter if they are true or not. We feel for the story characters caught up in the plotline.
This went farther (probably due to Nature's nature) to discuss the actual numbers involved and then the underlying science or lack of it. This is where truth began to be explained. It allows us to move beyond the "If it catches ONE terrorist it's worth it" cant to answering more interesting questions of "Well you've spent 35 million dollars and haven't caught ONE terrorist. You claim effectivness remains in the apprehension of other villians but how is the federal government responsible for the business of enforcing your local laws at the expense of everyone else in the nation?"
@Brandioch Conner "If Ekman's process is legit,"
If he was interested in legitamite research in the effectiveness of his method he could couldn't he? It seems like he's more interested in producing revenue from his brand. And if his brand really is snake oil he'll argue against the need for tests, anyone performing analysis on his "intellectual property", and fall back on the obscuration argument 'the enemy will know what we're doing'. Likely he will resist or denigrate anyone elses attempt to do so remarking on failure of results as "well the BDO wasn't effectively trained or using their training."
Obscurancy or is it occlumency?
"Sir, you can't let him in here. He'll see everything. He'll see the big board!"
Here's some behavioral detecting screening you can do:
Look for people in positions of authority who resist the release of numbers, transparency, and other measures of effectiveness. They'll be the ones in charge of questionable programs.
"General Turgidson! When you instituted the human reliability tests, you *assured* me there was *no* possibility of such a thing *ever* occurring! "
"Well, I, uh, don't think it's quite fair to condemn a whole program because of a single slip-up, sir. "
I think this article is a model for security reportage.
The TSA's entire approach to "security" seems based on the half-assed and inept implementation of the latest high-tech "solutions" to provide the appearance of deploying "advanced technology." It doesn't matter whether there's any scientific proof that the measures are effective, or whether their inept implementation of the measures negates whatever benefit might exist. The important thing is to impress the public that the government is doing everything they can (and sparing no expense) to deploy the latest technology at the checkpoints.
If it's highly visible, highly intrusive, and highly invasive of privacy, that's even better, since the TSA's motto since its inception has been "the more hassle, the more security!" Thus the latest "ennancement" of full-body scanners, where the TSA treats concerns about privacy, radiation, cost, and effectiveness simply as irrelevant obstacles for their juggernaut to bulldoze away. It doesn't have to be effective, as long as it's impressive enough to bamboozle people who fervently want to Believe that the government is protecting us from terrorism. After the next inevitable "incident," the TSA will react by adding yet another "enhancement" that will have no objective proof of effectiveness and will cause more difficulty for every traveler. As long as the Believers are bamboozled, it's worth the expense and hassle. And those who aren't bamboozled have a patriotic duty to remain silent, lest they undermine the Security Theater.
It is difficult to believe that the TSA's "behavior detection" program can identify terrorists. Terrorists, after all, are rather few and far between; so it's rather unlikely that the training has exposed the officers to any actual terrorists. And given the rarity of terrorists, I think it's unlikely that the simulated terrorists used for training are of any value in distinguishing the "signals" supposedly sent by evil-doers from the high background noise of passengers displaying all the stress that goes with flying.
The TSA trumpets the fact that it has had some success at detecting a small number of non-terrorist criminals as proof of the program's ability to detect an actual terrorist, should one happen to stumble into a checkpoint while the officers aren't preoccupied with looking for people carrying drugs or oversized bottles of perfume for the Monthly Numbers. But all it really proves is that the system is capable hassling large numbers of innocent travelers to produce a sufficient quota of false positives, a very few of which are (non-terrorist) criminals that are successfully convicted. But whether that provides any useful protection against the terrorist threat is one of those questions that are simply improper to ask.
The TSA is nothing more than a "faith-based" initiative.
Gladwell writes so much that he is bound to be right occasionally just by accident.
Add me to the list of people confused by this Bruce. You've alwars said it's more effective then TSA Rent-a-cops screening passengers.
A lot of people seem to think that 'fairness' ought to trump effectiveness, so we should introduce random scanning. Easily enough beaten by overwhelming the system. If you need 4 guys on the flight, and they are screening 10% randomly, you only need to use 6 to have a pretty good chance of success. You pretty much know the probabilities before you start. On the other hand if TSA is using some undocumented and not well understood process to pick up psylogical clues, you have no idea what your chances of getting adequate manpower through the check. While the line of questioning here makes sense from a scientific point of view, from an operational security point of view, it stinks pretty bad(ly).
We all know that the Israelies are doing this with pretty good success. Identified difference is they are using experienced agents. We have a numbers game where we pretty much are stuck using less experienced agents. So there is an attempt to come up with a 'cook book' approach. It's not perfect. Without trying, it will never get better. Never caught a terrorist, but appears to have a pretty good track record of not letting terrorists through (un-verifiable). But it has caught lots and lots of criminals. OMG!!! We caught lots and lots of criminals, we can't have that. Oh the horror. Criminals have had their rights to be criminals trampled on. How could we let this happen. The next thing you know we will be throwing criminals in jail.
Worse, some innocent people were inconvenienced. Wake up, that 75 feet from outside the concourse to inside the concourse is one big inconvenience. Everybody going through knows they will be inconvenienced and only hopes it won't be too bad. Did I pack 2 or 3 1 oz bottles of toiletries? Does my comb look like a knife on the X-Ray machine. Yea, it is bad. Crashing into a building is worse.
And of course to comment on my own comment, the problem is that is is much harder to deter terrorists than ordinary criminals. After all, for bank robbers, a result of "died trying" rates as a 0% success, much worse than never having tried... But for a suicide bomber that is something like a 50%-80% success: after all, they still end up in paradise and the still terrorize the rest of us.
"But it has caught lots and lots of criminals. OMG!!! We caught lots and lots of criminals, we can't have that."
They'd have to show that this program was, statistically, MORE effective at identifying criminals than random checks.
So far they have not been able to do that.
And their false positive rate (just for criminals) is around 99%.
For terrorists, their false positive rate is 100%.
How is it possible to be LESS effective than that?
Idea for a new series of kids books/tv shows:
It is difficult to believe that the TSA's "behavior detection" program can identify terrorists. .... And given the rarity of terrorists, I think it's unlikely that the simulated terrorists used for training are of any value in distinguishing the "signals" supposedly sent by evil-doers from the high background noise of passengers displaying all the stress that goes with flying.
When a normally honest person is trying to get away with something "clandestine", they will show signs of guilt and stress.
But, a terrorist doesn't think he is committing a "wrong". He may be at peace with his actions, and since he may be "on a mission from God", he has no guilt, and shows no outward signs that the behavioral screening program is attempting to detect.
El Al has been using behavioural profiling since the 70s and they have NEVER had a terrorist incident since it was put in place.
If you would do just as well "flipping a coin" then you miss just as many as you'd catch.
I don't believe in 40+ years of luck. It's a system that works when used by people who've been trained properly.
@Tom: " Next time I get jury duty, I'm going to tell the judge that I've read up on all of the latest behavior detection technologies and I'll be able to tell if the defendant is guilty just by looking at them."
Nah, many defendants don't testify. If you *really* want to get off of jury duty, say that you can tell if the PROSECUTOR is lying.
Removed from the jury pool in about 1.3 microseconds, I think.
See SPOT look.
See SPOT look and look.
See SPOT look and look and look.
See SPOT apply his intuitive hinkiness detector of dubious veracity to look for anomalous non-verbal subconscious behaviour cues suggestive of evil intent.
See SPOT catch the bad man without profiling.
We like SPOT.
Is it deceptive behavior if I maintain a neutral expression as I move through airport security even though the futile and gross invasion of privacy ticks me off no end?
It would be nice to think that we could employ well-trained individuals to practice behavioral screening, but such well-qualified persons would never tolerate the sort of "management" (cough, hack!) that security personnel are usually subjected to.
@HJohn: Shoes often have metal pieces in them that activate metal detectors. By requiring everyone to take off their shoes, they remove one source of extra time-consuming screening. (Yes, I despise it too.)
@Snarki "If you *really* want to get off of jury duty..."
Ask why there's gold fringe around the US Flag and if you are in an Admiralty court.
Found this in the comments for the Nature article: Interesting and plenty of potential material for regular comments here:
It demonstrates just how dumb the TSA thinks the traveling public is. You don't have to look very far to catch them in a "misrepresentation of the truth".
I addressed the comments on Israeli security and "hinky" in an edit to the post.
@"the problem with the Israeli security model is that it doesn't scale."
That's related to a point i often make that is a defense of poor screening techniques.
I attempt to clarify my opinion. The TSA screens over 750 million passengers each year across a continent wide country of 300 million and often has to deal with non-citizens. It is very difficult to recruit enough geniuses to do that job efficiently, and even more difficult to keep geniuses in such positions for a long period of time (i.e., there is high turnover).
Airport screening is not handled in the best manner, no argument from me there. But when considering the scale, skills needed, and unfortunately politics of it all, it isn't as simple as many on the outside may be inclined to believe. Even intelligent and capable people will likely never reach a consensus on the best approach.
@HJohn: "That's related to a point i often make that is a defense of poor screening techniques. "
That should read "That's related to a point i often make that is mistaken as a defense of poor screening techniques."
Omitting the words "mistaken as" will give my friendly rivals something to have quite a bit of fun with. Enjoy. ;)
My apologizes for the typo.
This is analagous to the screening issue in medicine. A test that finds cancer early may have so many false positives or cause so much tissue damage that the one life "saved" is offset by the collateral damage.
However, screening tests which are used selectively by experienced clinicians are quite effective (sensitive and specific).
The key, like what many others are posting, is training and experience and not relying on cookie-cutter approaches.
There are also other reasons why Israeli screening is "apparently" more effective than the TSA could ever be.
Firstly there is the passenger mix or ratio. Few people (in comparison to the US) fly to Israel without a fairly clear and easily checkable reason. Thus you don't get very many people of African or west/mid Asian descent flying to Israel let alone without a clear reason and I suspect never without luggage.
Unless they have changed their way of operating in the last twenty years El Al don't let anybody on one of their planes without putting them through the screening process. Even transit passengers who have already been screened get screened again.
As far as terrorists are concerned there are easier and less expensive ways to get at Israel than through it's flag carrier. And this is probably the real reason that El Al has not been attacked not it's profiling.
The question is the "cost", it is not monetary but retaliation Israel has in the past mounted what are effectivly "revenge missions" against uninvolved people in much the same way certain German soldiers did against French villages during WWII after a resistance attack.
Why would this appear to be the case well simply vandalism. El Al has been vandalised a number of times so their security is not actually that good...
@Joe: "This is analagous to the screening issue in medicine. A test that finds cancer early may have so many false positives or cause so much tissue damage that the one life "saved" is offset by the collateral damage."
Basically, the flawed thinking in hindsight bias and those who use exceptions to prove a rule.
Some things are easy to test for and known to be dangerous, so a periodic test is prudent. However, if they tested for every possible ailment I may have, they'd probably end up killing or maiming me, but dealing with certain tests and symptoms works pretty well with my health.
Same with airports. Some things make sense to check for (guns come to mind). Others, you'll destroy the system yourself without the enemy firing a shot if you go overboard. We haven't reached that point yet where I'd pick greyhound over taking my shoes off, but i'm sure someone could find a way eventually.
@ billswift at June 14, 2010 11:28 AM
Excellent point. And very appropriate.
@ Antonio at June 14, 2010 12:11 PM
"El Al has been using behavioural profiling since the 70s and they have NEVER had a terrorist incident since it was put in place."
So the *only* think El Al uses to capture terrorists is behavioural profiling?
Its good to know that its not down to racial/ethnic profiling, intelligence led assessments, sky marshalls, enhanced screening, reinforced cockpit doors, reinforced doors between passengers and cargo, etc.
The problem with claims like this is that they are also subject to variations on the "tiger protection rock." There is a stone in my garden that magically prevents tigers attacking. Ever since I have put it there, no tigers have attacked.
We know that there have been few attacks on El Al aircraft since the 1970s but is that because of Behavioural Profiling? While Israel is undoubtedly a high profile target there weren't even *that many* attacks in the 1960s
Also - not quite "no attacks" if this is to be belived: http://news.bbc.co.uk/1/hi/world/middle_east/...
Final point - we could also look at all the airlines to compare rates of hijackings vs non-hijacked flights and see if there are salutory lessons to be learned [for "fun": http://en.wikipedia.org/wiki/...
@anon: "But, a terrorist doesn't think he is committing a "wrong". He may be at peace with his actions, and since he may be "on a mission from God", he has no guilt, and shows no outward signs that the behavioral screening program is attempting to detect."
Indeed. Given the levels of stress and frustration now unavoidably associated with all aspects of air travel, only a terrorist can be reasonably sure of pleasant and timely arrival at his destination (i.e., Paradise). So anyone who appears serene and calm at an airport should attract the attention of the highly-trained officers and be subject to a maximally intrusive interrogation and search.
That approach has to be at least as effective as what the TSA are currently using, although it may not be as productive in terms of yielding false positive referrals of non-terrorist criminals to police.
@Carlo Graziani. How is this conscious or unconscious "gut feeling" any different than that fire marshal in Texas who testified to convict someone in spite of all the scientific fact to the contrary? As the saying goes, facts without theory are trivia; theory without facts is BS.
@Science Uber Alles:
The (rather obvious) difference is that a security screening is different from a court proceeding. Both in terms of the level of evidence that is considered reasonable to act upon and in terms of the consequences for an individual's rights.
A gut feeling by an expert is a ridiculous reason to send someone to prison. It is not a ridiculous reason to search someone's bag at an airport.
Cops stop people all the time for reasons that don't meet the same burdens of proof as court testimony. Experienced cops have a documentedly higher success rate at concentrating their stops on actual bad guys than do rookies --- search Bruce's posts for the term "Hinky" for examples. Those experienced cops can describe some of what they're looking for, but not everything. What they're really acting upon is a decade or two of experience in dealing with people in various stressful contexts. Believe it or not, they actually learn a thing or two that isn't that easy to teach.
I'm not sure how many here have had a real polygraph test by pros, but I have (related to work for NSA as a contractor).
Sure, you can fool the machine - easy. Not the point. It's the interviewer, using the machine as part of the psychological intimidation process you are not going to fool (not to mention the observers behind the one way mirror, who really get bent when you wave and smile at them and can appear to see them even through the glass, which in this case I actually could). They also get upset if you have really great hearing as I did and can hear them talking to the interviewer in the room with you from bleed out of their earpiece. I can't resist having a little fun sometimes...
The machine doesn't have to be always right, or even mostly right. It just has to cue the interviewer that they are getting into territory you are worried about -- at the most, so further questions are better directed until a savvy operator homes right in on what's causing you to worry.
It's a time saving device, pure and simple, in the hands of a skilled operator with a mission. Why waste time looking for larceny if you are a pedophile or something? In that situation, they WILL get it out of you. I'm pretty good at keeping secrets, but wow, that was impressive indeed -- even though I wasn't trying to keep any. Intellectually very quick on the draw and ready for anything those guys (they've probably seen it all).
The plan is pretty awesome, actually (this was well over 30 years ago I went through this). As a smoker, the first thing they do is deprive you of that till it's obvious you're hurting....they have some secretary watch you in the waiting room (who is probably more qualified than appearances would lead you to believe). Make you wait till you can't stand it anymore, of course in a very uncomfortable chair, freezing cold in my case, with nothing to distract you from worrying about what comes next.
Then they ask some fairly trick control questions that most people lie to, such as "have you ever taking something home from the office, like pens, you shouldn't have?". Then no matter what the machine or you say, the response is a very skeptical "Really?" and so on. They are real good at getting a calibration on you as a person, who gives a flip what that machine is saying to them?
It's just a prop, mainly.
In fact if you (as I did) go in with zero intent of lying, and even revealing some things you might think they don't like to hear -- they make you feel as if you were lying anyway, and if you reveal something slightly bad (for example, in this case I used to smoke pot) then of course you're trying to minimize something and you're really a major drug dealer instead. It's pretty brutal psychologically and they are really good at this.
Remember, in this case "it's all among friends" so there's no waterboard, but serious social pressure (and might considerably affect one's paycheck) so there's about as much pressure on the subject as possible, actually. Your whole career is at stake.
They are (were?) really good at this kind of thing.
The point is, the machine is merely a prop in what amounts to a parlor trick done by real experts. These people could probably pass themselves off as psychics to most other folks, and be pretty convincing at it, as they have a lot of practice at this game.
Now handing any machine to some dumb off the street security dude is another story altogether, as is the "hinky" detection by someone not good at it, and I'm not sure you can even train some people at that, seems you have it or you don't -- sure you could educate most people to be a little better, but not like these guys were (actually, the interviewer that impressed me so much was female, but I call everyone guys in a nod to feminism). This one wasn't good looking enough to be distracting like that great video clip linked above, and that wasn't important to this. They were however so sharp one wonders how they did that job without getting bored to death. People like that are in pretty short supply, so anything in bulk DHS tries based on that idea is going to fail -- as they say "it's the man, not the machine".
So I'm not the least bit surprised it doesn't work done the way they are attempting now, as I've seen what they do that does work, and this shares very little with that.
And now for a hopefully short story on failures in profiling. At one time, I looked precisely like running a meth lab and got hammered for that -- except I wasn't and they went away in shame (thank heavens) after doing some serious damage and stealing a few things from my chemistry lab "just in case". Which of course, I'll never get back, and it's not like quartz glassware is exactly cheap.
I ran a computer consultancy in the boonies. People came in and out at all hours, usually smiling when they left -- hey, it was a good place to work, and we were all getting rich -- roughly 500 bucks an hour split 3 ways. I had some occasional domestic troubles, in one case preventing a suicide attempt by my wife that brought the police who saw the chem lab (she called them in frustration after I tackled her and took away the gun).
We obviously all had "too much money" for where we lived in the boonies, in trailers. Because we'd earned it, by golly, and not making and selling drugs either, but by writing code like codecs now used in cel phones, and VOIP protocols for a large telecom outfit. Not obvious from the outside to mere cops though. Though the one local cop even told me and the DEA there was no way this was where they needed to be -- he knew me well.
So, wham, we get the DEA in full tactical assault mode, machine guns, sniper, agents, no knock, all that crap -- all dressed in black, refusal to ID themselves, the whole bit -- for all we could tell, we were being attacked by the Norwegian army rangers or something. Luckily, we didn't happen to have the "right" chemicals present as we really didn't make drugs at all. One funny is they kept asking where the "P2P" was, evidently a meth precursor to them, but to us computer guys we just didn't even understand what they were asking!
It all turned out OK in the end, due to my experience above, once they did some doggone homework and looked up who I was at my request -- I had quite a thick dossier at the higher levels of government, and we wound up getting some consulting work from them in Bruce's specialty -- we showed them how insecure their email was for example and explained the nasty things you could do with a warhol worm and DNS poisoning for the resulting security updates, and that really got their attention (and us, even more money).
The other funny -- these guys were saying things like "you need a maid" (it was dirty in our shop, the usual pizza boxes and so on) and we said we thought we'd married them (which actually did get a laugh), and "how can you work in a place like this" which brought a response of "ever worked in a cubicle?" which didn't get a laugh at all -- too close to home....And "where's the Armani and Ferrari if you've got all this dough" to which the answer was, of course, that those are for people pretending to be rich, not actual rich people who aren't dumb enough to waste money on those things.
But I'd have to call that total epic fail for profiling -- we matched the profile perfectly -- and it wasn't necessarily a flaw in the profile per se -- just that the profile didn't take all the necessary distinguishing data into account. Probably worked pretty well most of the time, but had we not gone limp and cooperated just right, there would have been shots fired -- they even brought an ambulance and obviously planned on that, pretty scary to look down the barrels of many loaded guns held by very nervous people (shaky) and even see the chambered round ready to come at you -- they did have pretty good aim so that was possible.
Profiling needs to be backed up with real homework, which unlike the "magic decoder ring" takes real human intelligent effort -- it's not something you can buy from some charlatan sipping at the public money supply. Again, qualified people are the limit, not the tech.
Have any terrorists boarded planes (or attempted to) in the US in the last few years? The high-profile cases (shoe-bomber and underpants-bomber) were in international flights. If there have been neither false negatives nor true positives, it's hard to evaluate the system's success.
Why not lump terrorists in the same category as criminals? Plotting to blow up a plane is a crime too. If we decide it's okay to screen for criminals (smuggling drugs, weapons, etc. onto a plane) at airports, the system should also catch terrorists (smuggling a bomb onto a plane). If we decide it's not okay to screen for criminals at airports, why single out one specific crime?
The single standout hijack event in the last 10 years would be 9/11 for almost anyone, wouldn't it? Your wiki link inexplicably fails to mention it.
Re experience. Good decisions come from experience (per the Israeli profilers), experience comes from bad decisions - guaranteed by TSA?
In the specific case described in the 'Nature' piece - absolutely typical TSA boneheadedness.
Would a 'real' Arab terrorist be carrying Arabic-language flash-cards? Seems unlikely, no?
Would a 'real' Arab terrorist be carrying a book 'critical of US foreign policy'? Seems unlikely, no? And, if this is now a marker for malintent - since a majority of US citizens are now 'critical of US foreign policy', it means that anyone carrying a copy of National Review, or the Wall Street Journal, cvan be deemed 'suspicious'.
Any 'real' terrorist trying to board a plane in the US today knows plenty about the 'security' techniques in use and will be careful to be completely inoccuous in every way.
To the larger issue - the various techniques discussed will, in vast majority of cases, become nothing more than a vehicle for the expression of the prejudices or simple ignorances of the user. Like the polygraph, a thin veneer of pseudo-scientific mumbo-jumbo. We will soon see people being detained because these approaches gave them an unrealistically-low score for malintent - which is also suspicious, don't you see! Like the polygraph, and FSTs, and other, similar witch-doctory, when the user already knows what the 'markers' are, and when those markers are highly-subjective, pretty soon , these systems can return any response that the user wants them to return. Having seen and dealt with a few TSA officers . . . .
I'm not saying that these are the purposes of those trying to develop these systems - I'm sure that they are honest and apply scientific rigiour to what they are trying to discover. But those are the outcomes that will occur when they are deployed in the real world.
I am a pilot; flying per se is not "stressful" to me (although I would probably rather be in the front seats). What is stressful is standing in a long line of people, waiting to take off my shoes and walk through a puddle of athletes-foot-fungus filled spilled coffee and wondering what innocuous inanimate object will be confiscated from me "this time", all the while knowing some maltrained rejected-by-McDonalds is scanning us looking for behaviors that will get him a bonus for flagging someone for "Special Hi-Intensity Testing" and get me 10 hours of waterboarding. And of course this stress will cause me to be selected...
Between the TSA pissing away my tax dollars with both hands, and the airlines nickel and diming me for every sheet of toilet paper I may use, I no longer do any commercial flying this (east) side of the MS river. I either drive or I fly myself. Only if I (really, really) need to go to CA or similar will I subject myself to this $@!#.
Having said that, I would point out that this is "screening" - It is like when I go to the doctor's office and they take my blood pressure with a cheap device exactly like the one I use on myself at home, or the puff or air they use to test for glaucoma. If it flags something, then a more effective test is used on those flagged.
@Tom: Be prepared to spend a lot of time discussing contempt of court with the judge and his/her friends.
The Original Bob wrote:
'It is like when I go to the doctor's office and they take my blood pressure with a cheap device exactly like the one I use on myself at home, or the puff or air they use to test for glaucoma. If it flags something, then a more effective test is used on those flagged.'
Well, WADR - the simple tests that your doctor or optometrist uses suffer only from the drawback that they may be somewhat inaccurate. For exemple, the BP numbers may be off by ±10 points instead of the ± 1-2 points of a quality device. But they do provide an objective measurement - even if it may be a little bit off.
These techniques, by contrast, have no measurable, repeatable or reliable accuracy, and are almost-entirely subjective. No, scratch that - they are (effectively) entirely subjective.They are actually nothing like the 'screening' that you go through at your doctor's office - because that actually works. As we see from the instant case, these techniques are way-more-likely to flag people with no malintent. No doctor in his right mind would use a screening technique with such a poor record for accuracy and such a high rate of false positives - the cost in wasted time and unnecessary additional tests would make this sheer madness. But, of course, when it comes to the time and convenience of the captive travelling public - no price is too high for them to pay!
It's just another bit of theatre - except now we're in a vaudeville, and we're being made to watch the mentalist's act. He's not very good . . . . .
"The problem with fast detection programs is that they don't work, and the problem with the Israeli security model is that it doesn't scale."
So what would you do (to either system) to improve detection, whilst minimising impacts on budgets and passenger annoyance?
@ Sysiphus at June 15, 2010 1:18 AM
Well it is Wikipedia, so you get what you pay for in quality of reporting. You could always add it in. I never meant to use the Wiki as an authoratative source, which is why mentioned it was for "fun" (fun in quotes because its fairly tedious so not really enjoyable to read though but its also not serious research).
I would be interested (seriously) in any studies that show the introduction of behavioural profiling has had a real, measurable, positive effect on any airline.
Any machine learning person will tell you that to do good learning, you need a large representative sample. When it comes to terrorists, we have a minuscule number of terrorist attacks on US soil in the last 20 years. Israel by contrast has had hundreds, which is why they have a much better profile than we do.
@Pete: "So what would you do (to either system) to improve detection, whilst minimising impacts on budgets and passenger annoyance?"
I think that is the problem with the scale of our environment, there will be an avalanche of criticism no matter what one does. I personally think they annoy too much and should quit with the shoe and liquid bit, but I also understand that what may be a negligle risk to me doesn't seem so to the person ultimately responsible for the safety of 750+ million travelors every year.
In other words, I don't like how they do their jobs, but I wouldn't want to shoulder their responsibility either.
"In other words, I don't like how they do their jobs, but I wouldn't want to shoulder their responsibility either. "
I totally agree.
However I also think that this is part of the problem.
It seems perfectly true to me that in the event of a successful terrorist attack, heads would roll in every sense of the phrase. There would be little room for saying "attacks are rare" or quoting statistical values when the secondary terrorism kicks in, and the structure is trying to find someone to blame (other than the terrorist who seems too obvious a candidate).
The fatal flaw with this seems to be that it leads to a self-referntial processes in which the fear of terrorism leads to more measures, which leads to more fear etc. None of it works, so the inevitable terrorist attack will happen, and the equally inevitable self-destructive "blame hunt" will take place. The next organisational head will promise to be even tougher on terrorism and the cirlcle restarts once more.
The only real result is that innocent passengers (who outnumber the terrorists by at least 1bn to 1 if the TSA are to be believed) are subject to excessive and expensive measures which take up valuable time for no real benefit. When challenged the organisation head can simply say that it was necessary because the alternative risk is "too great."
I think to fix it, we have to fix people. I also think this is pretty much impossible.
@GreenSquirrel: It seems perfectly true to me that in the event of a successful terrorist attack, heads would roll in every sense of the phrase. There would be little room for saying "attacks are rare" ...I think to fix it, we have to fix people. I also think this is pretty much impossible.
That sums it up well. The very same people who complain about security measures are the ones who would demand heads roll should security fail.
My position on almost every strategy is and has been that, while we will never all fully agree as to the extent of countermeasures, if we want rational decisions to be made we must be rational about the consequences of those decisions. We can't demand they don't screen shoes and then demand to know why a shoe bomb wasn't detected, for example... and that is just one example, shoe bombs are rare, so are undewear bombs, and a whole host of other threats, but if we aren't rational about our response to an incident we give them little choice but to address all of the known possibilities.
I remember once, when I was new at IT auditing, I got grilled by a manager because I missed something (it simply wasn't in my sample). Later, that same manager accused me of wasting time by over-sampling. She didn't realize that she all but asked me to over sample when she held me responsible for not detecting something that wasn't even supposed to be reviewed.
Sometimes, people get what they ask for without realizing they are asking for it.
While I could explain a few benefits of the program, none of them are really the result of its stated purpose, and I could think of examples where well-trained (as in prior experience or aptitude) TSOs accomplished the same thing without the "thought-police" act. Unfortunately, many of their ranks seem to buy into the hype that they're fed to motivate them, leading to an overconfidence which is both unseemly and distracting (kind of like what happened with "an army of one"). Like many programs that start up as "innovative" approaches in TSA, it has been transformed into a perk program
@GreenSquirrel: "The fatal flaw with this seems to be that it leads to a self-referntial processes in which the fear of terrorism leads to more measures, which leads to more fear etc. None of it works, so the inevitable terrorist attack will happen, and the equally inevitable self-destructive "blame hunt" will take place. The next organisational head will promise to be even tougher on terrorism and the cirlcle restarts once more."
Actually, they now seem to be bypassing the blame hunt. Obama's reaction to the Underwear Bomber was to call for "heightened airport security." He almost certainly realized that it's practically impossible hold anyone in the Homeland Security bureaucracy accountable for the failure, let alone herding all the cats and make them work together to fix the systemic problems that have only proliferated since 9/11. So he took the easiest approach of letting the bureaucracy off the hook, and instead calling for the punishment of all air travelers with more dubious "security" hassles.
This is exactly why the TSA will never be reformed, and why it will continue to enjoy a blank check to keep adding hassles and intrusions irrespective of their effectiveness. The TSA, as it currently exists, is simply too useful for politicians and bureaucrats. It's a facile way to cover up and to avoid correcting systemic problems. It's particularly useful for ensuring that nobody is ever held accountable for the inevitable failures. Better yet, it promotes the illusion that the "War on Terror" is somehow winnable, if we could only create impose arbitrary hassles and restrictions on everyone, employ enough inept bullies in uniforms to (inconsistently) enforce them, and (most importantly) do away with privacy and "civil liberties" that make us dangerously vulnerable to terrorists and get in the way of "victory."
None of this provides any actual security, of course. But that doesn't matter. Our Leaders have learned from the three-decade "War on Drugs" that "victory" is neither definable nor desirable, and that it's useful to have the public continually reminded to be terrified. The "War on Drugs" has done nothing to achieve a "drug-free America." But it has achieved significant success in making the United States the country with the highest incarceration rate in the world. The "War on Terror" is merely an improved version on the "War on Drugs," whose only goal is indefinite expansion.
"The only real result is that innocent passengers(who outnumber the terrorists by at least 1bn to if the TSA are to be believed) are subject to excessive and expensive measures which take up valuable time..."
Supposadly El Al spend ten times as much as the US per "on flight" passanger and fifty times as long talking to them with their "profiling" chat, unless of course they are recognised as a Jewish Israeli citizen which make up around nine in ten of El Al's 1.3million passengers a year where a simple greating suffices).
So the real El Al expense per "potential terrorist" is a hundred times that of the US and is close to 800USD...
As you note the politico's amongst others allow the TSA to get away with it,
"When challenged the organisation head can simply say that it was necessary because the alternative risk is "too great."
However the question,
'what does the near 8dollars per passenger actually get the US?'
And the arnswer might come back "20dollars of hurt".
I know of several people who have said "no more US holidays" and they used to go as much as three times a year. I also know quite a few business people who have said "no more US business trips".
It would be interesting to see an independant review of just what the DHS/TSA etc realy costs the US, then show it to those who have lost their jobs etc...
@ HJohn at June 15, 2010 2:02 PM
"Sometimes, people get what they ask for without realizing they are asking for it. "
A wonderful sentence and painfully true.
@ George at June 15, 2010 4:11 PM
I agree. It leaves me with an uneasy feeling about how self-sustaining this whole thing is. I wish I had an idea how to solve it, but I dont (short of enforced public education, but then the solution is the problem...).
@ Clive Robinson at June 15, 2010 9:09 PM
It would indeed be interesting to see how the actual numbers stack up.
I know there is a frequent claim that people will spend "whatever it takes" to save lives and combat terrorism but this is simply rhetorical crap.
There is a cold hard truth that tax payers are funding these monliths and should get at least *some* return for their investment.
A country that finds it too expensive to provide quality health care to its people, but will spend gazillions on screaning and shoe searches is, in my mind, getting things ass backwards.
Regarding the War on Terror:
Has anyone (in Gov't or Mil) ever officially defined the endstate for this war?
Are there victory conditions? Given the frequent call to make sacrifices as part of the war, there must be a point at which the war is decided to be officially over. What would achieve this?
I appreciate there may be some preaching to the choir on this particular blog, but I would be very interested to know of any leads to official statements / documents on the above. Or has officialdom simply accepted the fact the concept is barking mad and its only used to PR bad ideas onto the public?
@GreenSquirrel "people will spend "whatever it takes" to save lives and combat terrorism but this is simply rhetorical crap"
While I agree GS from the people I've talked to?
No. They mean it. They believe it. They just don't understand what it is they mean or what it really will cost.
But any money wasted in defense is MUCH better than money wasted on any social program. Social programs are for pansies, weaklings and malingerers. They morally corrupt our fiber intake and pollute our precious bodily fluids.
Though we are finally hearing complaints about money wasted on athletics in school. It used to be money on teach arts was 'wasted'. Of course that could be a consequence of all the cuts on arts programs already mean there are few programs left to cut.
Honestly people this hasn't been the 1800s for a century now.
Re: end state of the war
Either it ends when the Bush II GWOT objectives are met or 'people stop hating us for our freedoms.'
It's over now with the subsitution by the Obama administration of GWOT for Overseas Contingency Operation (OCO). And a mopup operation that requires the defeat of al-Qaeda.
They may believe that Bush administration has done all the damage it could to the US.
Given that the Korean war is still going on ... we may not have a good track record at neatly wrapping up our wars.
@Clive Robinson: "It would be interesting to see an independant review of just what the DHS/TSA etc realy costs the US, then show it to those who have lost their jobs etc..."
Don't forget to include the people who have been killed or injured on the highways because they chose to drive rather than submit to TSA restrictions and hassles.
Given the high risk associated with driving (several orders of magnitude greater than the risks of terrorism), my guess is that the number of deaths and injuries (indirectly) caused by the TSA now exceeds the toll of 9/11. But I doubt that any government bean counters would have any interest in actually compiling and calculating such statistics. Or if any such statistics exist, they're surely classified "for National Security reasons."
"No scientific evidence exists to support the detection or inference of future behaviour, including intent," declares a 2008 report prepared by the JASON defence advisory group.
This, sadly, will in no way prevent DHA/TSA or the USG from throwing away yet millions of more dollars! GO USA!
Wouldn't correlation of information be a better screen program? Similar to the algorithms used to screen emails for spam! The more an email looks like other emails that I've marked as spam then the more likely it is to be spam itself. So, wouldn't a person who's personal data and historical activity appear like others that have actually committed terrorism more likely be a terrorist themselves? And unfortunately this brings into play a number of "politically incorrect" levels of profiling.
@HJohn "The very same people who complain about security measures are the ones who would demand heads roll should security fail."
I see statements like that used so frequently that I believe they are the number one defense of poor security techniques and I feel that it is completely without merit.
Sure, the complainers will demand that heads roll. But the non-complainers are just as demanding too. After a security failure you don't hear people in favor of irrational security measures say things like, "Well, they did their best so, that's good enough."
The chorus is the same from both of those two groups. It's only the groups of people with more sophisticated understandings of the situation that hold off from unbridled criticism.
@ Anklebitersaurus: "I see statements like that used so frequently that I believe they are the number one defense of poor security techniques and I feel that it is completely without merit. The chorus is the same from both of those two groups. It's only the groups of people with more sophisticated understandings of the situation that hold off from unbridled criticism."
It can go either way. I've seen people use "we didn't know" as a (poor) excuse for things that should have been mitigated, and I've seen hindsight fingerpointers use "why didn't you know" (or, "connect the dots") for things that could not have been predicted.
Unfortunately, most people that open their mouths either way lack the sophisticated understanding you speak of.
I am interested in knowing the ratio of the number of deaths occuring in TSA screening lines (example - by heart attack) compared to the number of terrorist bombs detected by these screenings.
Schneier.com is a personal website. Opinions expressed are not necessarily those of BT.