Intelligence Can Never Be Perfect

Go read this article—”Setting impossible standards on intelligence”—on laying blame for the intelligence “failure” that allowed the Underwear Bomber to board an airplane on Christmas Day.

Although the CIA, FBI, and Defense, State, Treasury and Homeland Security departments have counterterrorism analytic units—some even with information-gathering operations—the assumption is that all of the data are passed on to NCTC.

The law, by the way, specifically says that the NCTC director “may not direct the execution of counterterrorism operations.”

The Senate committee’s list identifying “points of failure” shows that not all relevant information from some agencies landed at the NCTC.

Perhaps the leading example was the State Department’s failure to notify the NCTC in its initial reporting that Abdulmutallab—whose father had reported him missing in November and suspected “involvement with Yemeni-based extremists”—had an outstanding U.S. visa.

This initial fact, if contained in State’s first notice to the NCTC, would have raised the importance of his status. Instead, Abdulmutallab became one of hundreds of new names sent to the NCTC that day. The Senate panel blurs this in its report by focusing on State’s failure—as well as NCTC’s—to revoke the visa. Neither the department nor NCTC discovered the visa until it was too late.

Two other agencies also failed to report important relevant information.

[…]

How can the NCTC perform its role, which by law is “to serve as the central and shared knowledge bank on known and suspected terrorists and international terror groups,” if its analysts are unaware that additional intelligence exists at other agencies? The committee’s answer to that, listed as failure 10, was that the “NCTC’s watchlisting office did not conduct additional research to find additional derogatory information to place Abdulmutallab on a watchlist.”

True, NCTC analysts have access to most agency databases. But with hundreds of names arriving each day, which name does the NCTC select to then begin its search of 16 other agency databases? Especially when the expectation is that each agency has searched its own.

I’ve never been impressed with the “dots” that should have been connected regarding Abdulmutallab. On closer examination, they mostly evaporate. Nor do I consider Christmas Day a security failure. Plane lands safely, terrorist captured, no one hurt; what more do people want?

Tags: , , ,

Posted on June 2, 2010 at 6:39 AM • 63 Comments

Comments

clvrmnky • June 2, 2010 7:23 AM

“what more do people want?”

They want magic and unicorns, but also exciting news and the satisfaction that they are smarter than the gov’ment with their post hoc pronouncements.

MacFreek • June 2, 2010 7:25 AM

Two passengers (beside the suspect) were hurt. Nevertheless your conclusion stand. Terrorism attempt failed, which should confirm our trust in the existing security process, rather than calling for a change in it.

Frank Ch. Eigler • June 2, 2010 7:41 AM

“Plane lands safely, terrorist captured, no one hurt; what more do people want?”

I think what people want is a system that does not rely solely on the fortunate ineptitude of terrorists to make an attack into a “security success”.

BF Skinner • June 2, 2010 7:53 AM

@Bruce ‘what more do people want?’
Probably, everyone who posed or could or might or possibly pose a threat killed in a far off land. The rate of death due to aircraft failure compared to the rate of death due to terrorist act would be instructive but as we’ve covered before — when it’s deliberate somehow its worse. So people probably want to know that their problems are being worked.

This is a tie to the previous posting about roadblocks at Austin Int’l and the 4th ammendment postings that followed it.

Security people want to assign a trust level only what to they can test, inspect, and see. Whatever resides in the unknown/unknown quandrant tends to be rated high risk since there’s no information to assess it by. Though I’ve done it myself; I see this as a mistake.

So they tend to try and create their target of evaluation so that it’s same size as the population. (the number of discussions about system boundary’s that ‘include the internet’ I’ve had…sheesh) If there’s a risk of suspicious network activity the response is to inspect every packet. Since there are terrorists among us ALL must be scruitized to be sure they are safe. ALL must be innoculated so they don’t get sick. This is the Pod Person conflation. We developed as organisms that could tell friend from foe based on how near to us in the band they were. Xenophobia is based on that because the rule “if it’s different it’s dangerous” worked on the savanna. Once “different” became ideas in the head, virtual, light weight, shareable the rule becomes “people like us are dangerous” and “since I know I’m okay it must be you! Burn the witch.” In Europe that led Church “Security Auditors” to consign entire villages to the pyre since “God will know his own.”

Measuring and testing entire populations is rarely sucessful or possible (one reason the US Census Bureau keeps asking to do the Census statistically). There are too many people moving too fast and paying cash to run background checks, develop profiles and maintain survellience on everyone. There are very few terrorists. (Which is a good thing based on the people responding to this list I don’t want their intelligence applied to the problem of producing mass casualties)

The systems being developed to catch them should be focusing first on detection (identification and location), then on control (which at the moment is constrained to restricting their movements up to and including extra-judicical homicide), then (when the controls fail) incident response. Sound familar?

This IS the system in placed and it worked. (Including Air Marshals on board and how often does that happen?) Apparently it’s working all the time since there hasn’t been a significant highjacking since when? But the “system” has to stand up to the self-interested critique by self-appointed people with agendas holding the center of a self-amplifying echo chamber of the talking heads (who I am begining to think of as nattering nabobs of negativism).

The waste of our DHS dollars comes from measuring and testing (over and over and over and over and over again) people who are, if not safe, safeish.
But most security folk will say well that test only proved they were safe during the test. That was then. what about now?

PJ • June 2, 2010 8:39 AM

Hey Bruce – you’re mentioned in todays XKCD comic. Mouseover the comic for a little plug on your analysis of worst case scenarios 🙂

JR • June 2, 2010 9:29 AM

“What more do people want?”

Well, it would be nice if I did not have to consider which one of my fellow travelers might have a bomb in his or her undies and whether I could wrestle them to the ground.

I can understand the need for personal responsibility, but I don’t want to do the security checks myself.

So, he got on the plane with a bomb in his shorts. That is a failure of security.

That the plane landed safely is pure luck and has little, that I can see, to do with a security success.

AppSec • June 2, 2010 9:45 AM

@JR:
That sounds a lot like fans of teams that come up short in sports.. The winning side says: Hey, we won. Doesn’t matter that the final score came on a fluke play which happens only 1 in 100 times.. The other says: We played hard, it didn’t go the way we would have liked but the opportunity was there.

No system will ever be perfect. Make the choice not to worry about it and you’ll be better off.

Clive Robinson • June 2, 2010 10:05 AM

First off we need to invoke the law of small numbers. In neigh on nine years since 9/11 we have only had two bomers get on aircraft bound for the US.

Which means not enough data points to be meaningfull.

Yes there are ways the system could be improved but would we the people of the world put up with it?

That is all the DB’s get checked automatically when ever a persons name gets entered for anything and a score rating is produced.

If the score is above a certain value you pop the details out to a human evaluator to consider.

The down side is almost the same as credit checking by various profiling systems.

We as humans can automate these systems thus you apply for a provisional pilots licence and the bod looking you over can see you have a gun licence for a half inch Barret etc and can use the system to say “sorry you will have to go to appeal”

The problem with this apparent utopian solution is it won’t work because humans are involved and we are subject to error…

Roy • June 2, 2010 10:28 AM

The government has taken all power away from the passengers, forcing them to be submissive and defenseless, forcing them by virtue of threats of deadly force and unlimited imprisonment without due process of law. In return, the passengers correctly demand perfection.

If someone forces me to trust them, then the smallest mistake they make is complete betrayal.

Ralph Hitchens • June 2, 2010 10:35 AM

Two things: first & most important, data about a suspect individual that fortuitously includes information about possession of a valid visa for entry into the US ought to be flagged in whatever database he’s in, so that prompt attention is paid. The all-important CIA information about two al-Qaeda suspects identified in Malaysia in 2000 AND the fact that they had valid multiple-entry visas into the US was not shared with the FBI, resulting in almost 3,000 deaths on 9/11.

Second, the IT configuation at NCTC (originally the Terrorist Threat Information Center) was established with all personnel — detailed from the various agencies — on the same classified networks, BUT also having “reach-back” to their own individual agency classified networks through dedicated tail circuits, the idea being that even if everyone could not see all the agency-proprietary information, at least it would all be accessible under the same roof. This was, at the time (2002-03) a revolutionary concept in interagency information sharing.

Trichinosis USA • June 2, 2010 10:43 AM

What do people want? I suppose it depends on the people. The same people who were screaming that President Obama refused to cancel his holiday plans when the underwear bomber was captured are the ones who seemed to think it was perfectly ok for President Bush to continue reading a book to children while the WTC towers were collapsing.

mcb • June 2, 2010 10:52 AM

“Plane lands safely, terrorist captured, no one hurt; what more do people want?”

They want Jack Bauer doing the takedown…and the interrogation.

Tom M • June 2, 2010 10:59 AM

Keep in mind that many (not all) of those calling it a failure also claim that “There were no terrorist attacks in the U.S. during the Bush administration.”

kangaroo • June 2, 2010 11:40 AM

@JR: That the plane landed safely is pure luck and has little, that I can see, to do with a security success.

Which part of “probability” don’t you understand? Of course any large system is playing a game of statistics, of benefits analysis, or risks probabilities. Of course, any single event comes down to “luck” — they question is how MUCH luck do you need.

Obviously, from the number of planes going down, not very much to make it safely from one continent to another. That’s a security success — not an isolated analysis of a one-off shot, but a question of the probabilities that the one off shot even happen.

It hasn’t, so we bet it’s unlikely.

Your other choice, of course, is a world totalitarian state with cameras in everyone’s arses. That would guarantee in a much less probabilistic fashion your security.

J. J. Ramsey • June 2, 2010 11:49 AM

kangaroo: “Your other choice, of course, is a world totalitarian state with cameras in everyone’s arses. That would guarantee in a much less probabilistic fashion your security.”

Not really. With a totalitarian state, there would be far fewer legal barriers to collection of raw data, but that means a bigger haystack to sift through to find the needles.

Impossibly Stupid • June 2, 2010 11:55 AM

In 2010, is “hundreds of names arriving each day” really that great of a burden? If all we’re talking about is cross referencing existing databases of information, that seems like something that might take all of an hour. I don’t expect perfect intelligence, but I do expect at least sufficient technology to be in place so that you don’t need to manually prioritize names before you begin to gather intelligence.

kingsnake • June 2, 2010 12:03 PM

“… what more do people want?”

Their mommies.

Seriously.

Most people want to be protected by their parents like they were when they were infants and the world was all rosy and safe, and they never had to anything for themselves, or think about anything more complicated than why their diaper feels heavier than it did a minute ago.

HJohn • June 2, 2010 12:25 PM

@Trichnosis USA: “The same people who were screaming that President Obama refused to cancel his holiday plans when the underwear bomber was captured are the ones who seemed to think it was perfectly ok for President Bush to continue reading a book to children while the WTC towers were collapsing.”

Trich, that is just STUPID. But two can play at that game. The same people (you come to mind) that say things like that are probably the same people who would have complained had he caused a panic in the classroom. Bush was a lousy president, but he did what was probably the best thing while waiting for them to arrange his depature.

People can’t discuss flatulence without you finding some way to turn it into a shot against Bush or someone on the right. Get some therapy. Seriously.

Tom M • June 2, 2010 12:29 PM

@Impossibly Stupid

Just going by names doesn’t work too well: “Last week, Senator Ted Stevens of Alaska complained that his wife, Catherine Stevens, has been questioned at checkpoints because her name in its diminutive matches that of the singer formerly known as Cat Stevens. Now known as Yusuf Islam, he has been barred from entering the United States because of activities that the Department of Homeland Security said could be linked to terrorism. ”

http://travel2.nytimes.com/2006/02/14/business/14road.html

Also, since Arabic names have to be transliterated into English, a name on the watch list might not be an exact match to current documents held by an individual member of al-Qaeda. But it might be an exact match to dozens of innocent people with a common name.

Geek Prophet • June 2, 2010 12:31 PM

@JR

Yes, this is a security success.

With no security, he could have carried a powerful bomb with a professional-quality detonator on in his carry-on. Or, he could have put an even more powerful bomb in his stowed luggage, again with a professional-quality remote or barometric detonator.

Instead he brought a device so badly made it didn’t detonate and so small that the plane likely wouldn’t have gone down even if he had been able to set it off.

Security isn’t perfect. It cannot be perfect. But when no security clearly would have allowed success, but instead there is failure, then, according to the very best measurement we have, the security succeeded.

moo • June 2, 2010 12:46 PM

@Roy: I agree with uk_visa, you make a good point.

However, in a pragmatic sense, allowing passengers to carry guns and knives onto planes for self-defense would cause more problems than it solves.

In exchange for reducing the (already astronomically low) probability of a successful terrorist event, I think it would greatly increase the risk of agitated or unstable passengers causing injury or death to those around them. Some weird incidents happen on planes from time to time. A schizophrenic passenger attacking cabin crew, or other passengers, would be a much bigger threat if (s)he were armed.

I think there’s a rather big difference between allowing concealed carry on the street, and allowing it while sealed into a metal tube with 150 other people 20,000 feet up in the air.

moo • June 2, 2010 12:59 PM

@ Clive Robinson:

I think there’s a much bigger problem with the system you describe, which is that it discriminates against some people based on the contents of various databases. Since actual terrorists are so rare, in at least 99% of the cases, these will be innocent people who just happen to do a suspicious combination of things. You could ask people who always have “SSSS” scrawled on their boarding pass how they feel about such processes. Or you could ask people whose names are on the no-fly list (and have no idea why, and have had no success getting their names removed from the list). And unthinking trust of the information in these databases can be harmful too. Anybody can have wrong information entered into a database about them (like identity theft victims, or someone whose identity was confused with a convicted felon) and to then be silently discriminated against because of this, e.g. always selected for SSSS screening, is really unfair.

Obviously we have to try and focus the available human resources where they will have the most effect, which means some people are going to get a lot more scrutiny than others. It would be a lot better than the current de-facto racial profiling they do (“driving while black”). But I think it has to be done in a way that does not result in the continuous harassment of innocent people. And most of the people flagged as “risky” by any automated system like that, will after all not be criminals or terrorists, they will just be ordinary people who happen to meet some “risk factor” criteria in some automated intelligence-gathering system at some three-letter agency.

Max Lybbert • June 2, 2010 1:31 PM

Yes, the fact that few bombers have tried to take down planes in about a decade is a security success (shoe bomber, peroxide bombers, underwear bomber). Schneier has repeatedly shown that much of that success is unrelated to the security procedures in place, because the procedures are little more than security theater and it is in fact possible to board airplanes with concealed bombs.

Which makes the statement “Nor do I consider Christmas Day a security failure. Plane lands safely, terrorist captured, no one hurt; what more do people want?” all the more perplexing: the plane landed safely, the terrorist was captured, and nobody was hurt in spite of the security theater that allowed the bomber to board an airplane and trigger his bomb in flight. The only reason nobody was hurt was a lucky break. I have a hard time believing a security expert like Schneier — who has repeatedly pointed out that the security apparatus in place does not prevent people from boarding planes with viable bombs — doesn’t realize this, so I can only conclude he’s being disingenuous and intellectually dishonest in calling this a security success. But just in case “security has several definitions, but none of those definitions is ‘rely on lucky breaks for the public’s safety.'”

Brandioch Conner • June 2, 2010 1:39 PM

@moo
“Since actual terrorists are so rare, in at least 99% of the cases, these will be innocent people who just happen to do a suspicious combination of things.”

Exactly. Which means that, over time, the people evaluating the criteria WILL miss the real “terrorist” because they will have gone through hundreds (thousands?) of false alarms.

The problem I see is “information asymmetry”. YOU don’t have any visibility into how or why the “security” processes is how it is.

But the people in charge of it have the ability to disrupt your life.

People have to put up with missed flights … and when the real terrorist tries an attack … he’s let on without any problems.

J. J. Ramsey • June 2, 2010 1:52 PM

Max Lybbert: “The only reason nobody was hurt was a lucky break. I have a hard time believing a security expert like Schneier — who has repeatedly pointed out that the security apparatus in place does not prevent people from boarding planes with viable bombs — doesn’t realize this, so I can only conclude he’s being disingenuous and intellectually dishonest in calling this a security success.”

It wasn’t just a lucky break. Schneier had explained himself a while back:

“In order to get through airport security, Abdulmutallab — or, more precisely, whoever built the bomb — had to construct a far less reliable bomb than he would have otherwise; he had to resort to a much more ineffective detonation mechanism. And, as we’ve learned, detonating PETN is actually very hard.”

From http://www.schneier.com/blog/archives/2010/01/christmas_bombe.html

AppSec • June 2, 2010 2:04 PM

@JJ Ramsey:
I think Max is trying to point out that the security at airports can’t be security theater if it forced the builder of said bomb to put a device together which would have been ineffective or had a good chance of failing or causing minimum damage.

So, it’s more of a question of which is security theater — the disposing of liquid, scanning carry on luggage, removing luggage, no fly lists, etc.. and which of those legitimately cuased the device to be built in an unstable manner.

HJohn • June 2, 2010 2:09 PM

@Brandioch: “Exactly. Which means that, over time, the people evaluating the criteria WILL miss the real “terrorist” because they will have gone through hundreds (thousands?) of false alarms.”

I think that sums it up well. When tasks get monotonous, people sleepwalk through it and miss what they are supposed to be looking for.

J. J. Ramsey • June 2, 2010 3:30 PM

AppSec: “I think Max is trying to point out that the security at airports can’t be security theater if it forced the builder of said bomb to put a device together which would have been ineffective or had a good chance of failing or causing minimum damage.

“So, it’s more of a question of which is security theater …”

The security measures that forced terrorists to get creative with their bomb-making are pre-9/11 stuff. And yes, I include the shoe bomber in that, since I doubt the modifications to the bomber’s shoes would have been noticeable on an X-ray.

DayOwl • June 2, 2010 3:54 PM

I am amazed at how a potential bomber getting through airport security in another country, NOT the US, can be seen as a security failure on the part of an American Political Appointee who operates in the US.

Capt. Underpants got on a plane in Amsterdam, which is nowhere near the US. Or is the Netherlands relying on TSA personnel to man their checkpoints?

Ari Tai • June 2, 2010 4:00 PM

Seems that crediting those with a monopoly on force (both to deter by certainty of awful retribution on the responsible groups – with the same monopoly for the IC to do saturation surveillance outside of our borders with the FBI inside) with success the last two attacks is not truthful. The appropriate learning and urgency of response can only come from assuming that the attack was successful when doing the autopsy.

Granted, if the intelligence community had flooded the web (and the training camps) with bogus IED tutorials, and assured that the practice in the camps worked but failed in actual use then the government gets a gold star. Not likely but we can hope.

JonS • June 2, 2010 5:07 PM

Atri,
it’s not really a matter of giving the US Govt a gold star. The plane landed safely, Capt Sizzlepants was arrested, and almost everyone on board was unharmed. The terrorist failed*, which in a zero-sum world means that the security of the pane and it’s passengers was maintained. it matters not who, what, or how it was maintained – the simple fact is that the totality of explicit, implicit, overt, covert, witting and unwitting security measures that are in place – including, yes, luck – worked.

Jon

  • of course, the subsequent unforced error of over-reaction actually turned the failure into something of a success, but that was a total own goal.

Impossibly Stupid • June 2, 2010 5:31 PM

@Tom M

The pros and cons of which matching algorithms are used by security officials is another subject entirely. The issue I see here is that NCTC is saying “Oh, we weren’t able to check the visa status because we have so many people to check and that name, despite being an exact match, wasn’t at the top of our list” in an era where the check could have been run automatically against every submitted name and essentially returned immediately. You could argue that it might be a false-positive match against someone else, but I would say that flagging it for priority investigation as a result is still the right move. After all, how do they decide which of the hundreds of names to investigate first if not based on that kind of readily available intelligence?

JonS • June 2, 2010 6:26 PM

  • Impossibly Stupid,
    sure. That’s a good idea, so let’s do it that way.

Now, having done that, which of the many thousands of false-positive matches flagged for “priority investigation” are you going to /actually/ investigate first?

BF Skinner • June 2, 2010 7:01 PM

@moo “…which is that it discriminates against some people based on the contents of various databases.”

Well, yeah. But isn’t that the entire point of the databases? The people in it are judged to be a threat and the bias against permitting them to travel is wholely justified.

Where it breaks down (as you note) is what we’ve been discussing here for about 10 years.

How can ONLY the people who are a threat be identified and contained.

AND

In the cases where they don’t put legitamite people into the databases to preserve opsec; why bother?

Harry • June 2, 2010 7:24 PM

@Impossibly Stupid: In 2010, is “hundreds of names arriving each day” really that great of a burden?

Yes.

The systems involved aren’t perfect and probably “just grew” over the years, just as every other database in RL did. I’d be surprised if they talk to each other very well. Heck, even different parts of my bank don’t talk to each other very well. My bank is a private company, run by the same group, with a strong financial interest in well-organized infomation, using very simple information[1], and still can’t get it right. Why should different agencies that developed their systems for different purposes over – probably – many years, be any different?

Then there’s the problem of how to write and organize names from other countries. Let’s use Middle Eastern Arab as an example.

Middle Eastern Arabic names are TOUGH. They don’t use Western-style GivenName1 GivenName2 FamilyName. It’s GivenName, FatherName, GrandfatherName OR TribeName OR AreaName. And they use nicknames all the time. Once a Middle Eastern Arab has a child he is just as likely to be known as FatherOfChildName. Or maybe he’s known by his place of birth, such as al Baghdadi. Or he’s taken a nickname for purposes of war or writing or just because he wants to.

All of these things are normal in Middle Eastern Arab culture. It’s so hard figure out how to organize names that even Middle Eastern countries are stumped. Saudi Arabian phone books are organized by GivenName.

Further, Middle Eastern Arabs use far fewer different names than Westerns do. (Making up numbers here) if the top 100 names in the US represent 15% of all US given names, than the top 100 names in the Arab Middle East represent 85% of all given names. Duplication and confusion are the norm.

Then there’s the issue of transliteration: there’s no standard version. There’s widespread disagreement about the treatment of articles such as “al” – is it its own word or is it combined with the name that follows? There’s also the question of where to break up names.

The name of the Christmas Bomber is a fine example. Umar Farouk Abdul Mutallab, The US press usually renders it Umar Farouk Abdulmutallab. The last name (which is NOT his family name) could also be Abdul Mutallab or Abd ul Mutallab or abd-ul Mutallab. The middle name (which is NOT his secondary given name) could also be Farook or Farooq or Faruq. Omar instead of Umar is a trivial change compared to these but is yet another variation to keep in mind. He was also known as Umar Farouk al-Nigeri: Umar Faruq the Nigerian. Does that make al-Nigeri his “last” name? or Nigeri? You tell me. Now you figure out how to check all of these in even a perfect set of databases. After that, you do it with RL databases.

[1] Most of the information is financial, meaning numbers. The rest all originated in one language (English), based on one person (me) with a name that conforms to Western norms. Information about persons is messy, and easy to quantify or put in a relational database. When we’re talking about names, not all societies use naming conventions that conform to Western norms.

Davi Ottenheimer • June 2, 2010 11:44 PM

Thanks. Good article. For some reason your concluding comment reminds me of this story:

http://www.lacrossetribune.com/news/local/article_0ff40f7a-d4d1-11de-afb3-001cc4c002e0.html

“…student Adam Bauer has nearly 400 friends on Facebook. He got an offer for a new one about a month ago. […] He thinks that led to his invitation to come down to the La Crosse police station, where an officer laid out photos from Facebook of Bauer holding a beer — and then ticketed him for underage drinking.”

I assume it safe to say most people would not want this kind of intelligence and enforcement…

Impossibly Stupid • June 2, 2010 11:52 PM

@JonS

The burden of proof is on you to show that these lists are resulting in massive false positives. Note that even NCTC doesn’t seem to be making that claim. They simply claim that the list itself, numbering just a few hundred entries, is too big to even do an automated search on.

@Harry

The problem with your line of reasoning is the NCTC stated not that they couldn’t do the searches easily, but that they had assumed the State department would do the searches.

It’s not an issue of difficulty or duplicates or even perfection. It’s about poor communication and a failure to standardize basic evidence gathering. There are obvious improvements that can be made, and the failures should have been anticipated well in advance. Too many of these kinds of mistakes have been happening recently (I’m looking at you, BP!).

gopi • June 3, 2010 1:08 AM

@Harry has some very good points about Arabic names.

One thing I discovered in Kuwait is that you can usually tell how important or closely related to the Emir somebody is based on how many names are used to refer to them.

@name Al-Sabah means they’re top level. @name Al-FathersName Al-Sabah is probably slightly more distant. Sometimes there can be three or four names there.

Western naming conventions are so much easier to put into databases.

Don’t get me started on street names in the middle east…The US Postal Service has done wonders in the USA. Streets are all named, cataloged, etc. Dubai, you simply can’t reliably identify a location without landmarks.

yt • June 3, 2010 3:54 AM

@moo: I got the impression Clive was being a bit sarcastic there. I don’t think he really advocates a system like that.

@BF Skinner: “So they tend to try and create their target of evaluation so that it’s same size as the population. (the number of discussions about system boundary’s that ‘include the internet’ I’ve had…sheesh)”

TOE? System boundaries? You’re speaking in Common Criteria! In the interest of your mental health, I recommend a vacation ASAP. 😉

Harry • June 3, 2010 6:24 AM

@Impossibly Stupid –

Your saying my reasoning was faulty is inaccurate. I answered the question you asked about whether hundreds of names a day were a problem, and explained why the answer was “yes.”

But in response, you’re complaining that I didn’t answer a question that you didn’t ask. It may well be a good question but since you didn’t ask it I can’t tell.

bluedog2 • June 3, 2010 6:43 AM

“What more do people want?”

We don’t want the guy with the bomb to get on the airplane at all.

Because he actually got on the plane and for 6 hours could have blown his underpants and the plane out of the sky, the Christmas incident was a failure to detect and prevent.

Impossibly Stupid • June 3, 2010 10:18 AM

@Harry

You did not answer my question after examining the available evidence; do you work for the NCTC? Again, it’s not about the possibility of the hundreds of names having variations or false positives. It’s about them not even having the basic search results available in the first place. It is most definitely faulty reasoning to complain about the results when the search hasn’t even been done in the first place.

Harry • June 3, 2010 11:24 AM

@Impossibly Stupid:

Your original question, as I perceived it, was how much of a burden is it to check “hundreds of names” daily? My answer is that it is a large burden, and went into detail as to why I consider it so.

Perhaps, if you think I perceived your question differently than you mean it, you could carefully rephrase.

To answer your other question no, I dont work for NCTC. I expect if I did that I couldn’t talk about it. I do, however, have some experience with Middle Eastern Arab names and so can see how hard they are for Western systems to work with.

moo • June 3, 2010 11:36 AM

@HHO:

Is air terrorism really a problem any more? I think reinforced cockpit doors have basically solved the problem of planes being comandeered to use as a guided missile. Kooks trying to blow themselves up with no threats or demands or hostage-taking, we can’t do much about.

I’m far more afraid of the security goons finding something in some database and deciding to harass me, than I am of actual terrorists trying to kill me while I’m on a plane (which is probably more rare than planes being struck by lightning).

But maybe we could just issue crowbars to all of the passengers who want one, and give them instructions at the gate that if a terrorist tries to take over the plane, or threatens the stewardesses or brandishes a gun or knife while shouting at them, they are to rise from their seats and beat him to death.

By the argument I made above, this would most likely result in a couple of unfortunate non-terrorists being clubbed to death each year by overzealous passengers. But maybe its a price worth paying to satisfy our collective need to feel “secure” from terrorism. It would be a damn sight cheaper and easier than all of the security theatre they indulge in today (like the child-porn-producing body scanners and the confiscating of bottled water).

HJohn • June 3, 2010 12:24 PM

@Moo: “Kooks trying to blow themselves up with no threats or demands or hostage-taking, we can’t do much about.”

Not to mention there aren’t many of them.

There are a few million flights a year, and we go years between instances. It’s just not a significant risk.

Now, I think most critics of airline security, such as myself, would agree that if we were having frequent incidents that it may justify screening shoes or even body scanners, etc. It is all in proportion to the threat.

This is where I seem to contradict myself. Since the TSA screens over 750 million passengers a year, it’s tougher for them to pass on a risk. The odds of an incident that would affect them are much higher than the odds that one would affect us, so I’m not trying to blast them too much.

I’m just trying to keep things in perspective on both sides.

Telephone Systems • June 3, 2010 6:56 PM

These people need to work harder as the opportunity for mass murder on planes is a serious dissincentive to travel. I think that organisations such as the UK GCHQ should evolve to have more resource to sniff telephone systems and other electronic communications. The terrorists have to communicate and it is safest to catch them at the point of initial planning.

Clive Robinson • June 3, 2010 11:47 PM

@ moo,

Yup the problems are very many and as yt said my comment was not ment as a serious proposal but to get people to think of the consequences of such a solution.

One problem we have is that the problem is to complex to be solved by simple solutions.

@ Harry mentioned the issue with names but nobody has mentioned two other serious problems.

The first is it is not actually possible (currently) to link a human body to an identity document in a reliable way. Part of this as seen with the US election is a persons “birth record”. All a birth certificate does is record an event that a child was born on such and such a date at such and such a place and in some cases give a unique number for the event by which you can check the record in the register (note I do not say DB because for many people myself included the register was in fact a paper ledger book…). The assumption is that the holder of a birth certificate is the person to whom it refers but there is no way of establishing this reliably and less so as the populas becomes more mobile. Thus with a little effort and a reasonable degree of time a person can actually have two or more quite legitimate but entirely seperate identities and society is in general quite happy with this (think actors, authors, proffesional women, etc etc).

The second issue is “roles” a person is assumed to be unique or an “individual member of society” or a single entity. However they have many roles within the society they live. For instance within a family they can be all of the following,

1, Son
2, Husband
3, Father
4, Grandfather

Then there is the multitude of “roles” they have amongst society in the greater sense of lover, close friend, friend, and various forms of acquaintance.

Most of these roles whilst important to the individual are of little relevance to others or for the purposes of law.

However people have other roles via employment or position that do have relevance under law both criminal and civil. For instance a person could quite easily be,

1, Employed as an accountant for X
2, Be a club or society accountant for Z
3, Be a freelance accountant for Y.

Now let us assume that one of the organisations (X, Y or Z) gets into a civil or legal dispute. should that be of relavance to the other organisations.

The answer is “it depends”, and this is the problem, record keeping is imperfect there is no way it can be anything but unreliabe.

If an individual only has a single “identifier” then they can be linked to others who have then committed crimes etc. But have the inability to show that they where not involved in the crime just “associated by way of trade” to the criminal.

Thus it is desirable that people have many identifiers to prevent false assumptions blighting their life.

But this multiple roles/identities has it’s own issues both human and technical and for the majority of people is way beyond their abilities to manage simply because of the muddeling of roles together (social relationships with co-workers etc).

The more you think on these issues the more you realise that there are real and significant problems.

For instance “guilt by association” the US amongst others has tried to extradite or prosecute individuals as terrorists simply because they had a role that had an intersect with that of a suspected terrorist. In one case it revolved around telephone numbers in a diary of a third party…

Now if you consider “degrees of separation” it is known that for the vast majority of the US populas they are connected by as little as 4 degrees of separation from the majority of other US citizens, in many cases by as little as 1 (ie they use the same bank).

As the number of DBs increases more and more seperate degrees of separation between any two random individuals will be found to exist. At what point does this become “circumstantial evidence” by which an individual can be prosecuted successfully by a “jury of their peers” who do not understand or probably comprahend the idea of “degrees of separation”.

As the old saying has it, “thinking about it can keep you awake at night”.

Ray Te • June 3, 2010 11:57 PM

This post makes me question why I have Schneier’s blog as my home page.

I was wondering why so many of the responses to this post seemed to throw logic to the wind, but the clear political angle of so many of the responses erased any questions I had. Let’s look at this logically instead, shall we?

The logical implication of Bruce’s statement, “what more do people want?” is that the United States should make no changes in response to the Underwear Bomber. He argues that citizens should demand no response to this failed attack.

This is an invalid response from a security perspective. If a virus was able to enter your network, but due to metamorphic code or some other malfunction did not correctly deliver a payload, would you consider your firewall a success? No, you would consider the virus a failure. Failure of one thing(the virus, a terrorist) does not logically mean the success of another(your firewall, our security measures). Clearly the only reason the plane attack was a failure was due to the ineptitude of the terrorist, not the success of our airplane security.

By arguing that the attack was not a mistake or lapse of security, there is a logical argument AGAINST making any changes in response to the attack. Other than the clear political bias–or perhaps even to save professional face–why would you make this argument? Why aren’t more of you concerned with tuning the system?

Instead of the application of logic, I see the casting of this case in extremes. Bruce(and some of you above) sees no mistake at all–clearly an extreme view, and a mistake in my opinion. Others see this as a complete collapse of our security system–again an extreme view. I also see political biases at work here.

Instead of making excuses, we should look at the Underwear Bomber case logically, and as a lapse in security that should be corrected. This is a mistake that should be owned and dealt with. Otherwise we will not respond accordingly.

I am frankly stunned by the attitude of “nothing to see here” on display above. As security professionals we should be more interested in fixing this problem instead of arguing that it is not a problem.

GreenSquirrel • June 4, 2010 3:11 AM

@ Ray Te at June 3, 2010 11:57 PM

Cost effective security measures are the key.

On a daily basis networks come under attacks that fail. Do you reconfigure your firewall everytime someone fails to get through a port?

There is never going to be a point at which security is 100%, therefore we need to accept that there will always be occasions where a layer is penetrated. When this happens we should assess the situation and see if it is cost effective to make changes which will prevent this, and if it isnt then why worry about it?

Using another analogy: If a system has a 99% success rate, we should expect 1% failure. When there is 1 failure in 100 events, would you recommend we say the system has failed and we need to spend time and effort correcting the system to iron out that failure? (possibly introducing other failures along the way)

Now, crucially, I do think we should be concerned with fixing the problem but the problem isnt that an idiot set his underpants on fire on a plane.

Craig • June 4, 2010 8:31 AM

Always good to have some post analysis so procedures can be corrected.

Aiming for the unattainable 100% effectiveness.

Security is an ever evolving process.

AppSec • June 4, 2010 11:34 AM

@GreenSquirrel:

I think the problem is whether or not you call it a “success” or a lucky break. Calling it a success is taking a look at too high of a level.

J. J. Ramsey • June 4, 2010 1:33 PM

Ray Te: “If a virus was able to enter your network, but due to metamorphic code or some other malfunction did not correctly deliver a payload, would you consider your firewall a success?”

Bad analogy. A better one would be that the virus was only able to enter the network because it was designed in an exotic but flaky fashion that made it prone to failure when encountering other layers of security.

JimFive • June 4, 2010 3:04 PM

@Harry
While I agree that the name problem and the identity problem are difficult. I find it amazing that the NCTC doesn’t have a process that automatically searches names when a new record is entered. The what name to search for problem is alleviated, though not solved, by allowing multiple names to be attached to each file. Allowing native language names would make it even better. A soundex like algorithm could be developed over the whole name to get reasonable results.

Saying “We didn’t bother” is not an acceptable response from the NCTC. If it isn’t possible, tell us what you are doing to make it possible. Searching for 100 names in 16 database should take all of 10 minutes. Leaving 5 minutes per analyst per name to review the records and make a decision about what to do. Assuming they have more than one guy in a basement doing this that seems like plenty of time to get the ball rolling.

JimFive

Clive Robinson • June 4, 2010 3:44 PM

Hey guys cool it a bit…

I invoked the law of small numbers further up for good reason.

Two points in nine years is not enough data points to make a meaningfull argument.

My personal view about Captin Underpants and LCpl Hotfoot is that they are the equivalent of network “keep alive” pulses.

They where carried out by people that lets face it where a danger to themselves before they ever got the terrorist urge.

They where basicaly to dangerous to use as anything other than throw aways because they are Walter Mitty types living in a fantasy world where they think they matter. Saddly they where groomed in that belief and their make believe world became a reality.

Those that sent them basically did not care if the blew up or survived as the whole point was to send a message to the American people that “we are still hear”. And importantly the aircraft would have to land safely for the message to get through.

If either had brought the aircraft down it would have been a failed mission because women and children would have been killed and that would lose the terrorists sympathy in the areas that count at the moment (killing troops with IED’s etc).

The message would have got through if Captn Underpants had been pulled at AMS or even if he had been pulled before getting to airside for his first flight. Thus he had already won before he left before he even “doned the pants” that day…

Let me put it another way I don’t have to get through your computer security to send a message, just throwing a brick through the server room window will do that, or spraying car paint on the side walk wall over night. Even being arrested on the way with the brick or spray can will get the message through.

Take a step back and try and see the attacks for what they where. As a “message” it was almost certain it was going to get throughb to the US people with the way US Politicos and the DHS behave.

The only important thing to concern ourselves with was how did he get so far was it just simple circumstances working in his favour (unfortunatly yes) or where there security failings (unfortunatly yes). We then ask the next question can these issues be corrected (yes) is it pragmatic to do so (no).

Leave a comment

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.