Schneier on Security

Roger • August 8, 2006 3:31 AM

@Dave H:

the question remains are those resources better spent searching out threats through more traditional means?

One great thing about data mining is that, unlike almost any other kind of investigative technique, it answers this question at the same time that it finds the inferences. That is, it doesn’t just say “X knows Y”, but provides enough additional information that you can work out if the lead is actually worth following up, and potentially even in what order leads should be investigated for maximum likelihood of success at minimum cost.

In contrast the traditional model is to try every lead you can find, one after another, until you run out of time or money.

I would like to ask the believers if we should put ALL our efforts into datamining, would they feel safe then? I doubt it…

Of course not. That would be foolish. It happens to be a useful investigative technique, and often a susprisingly cheap and powerful one; but it is just one more tool on the belt. The odd thing, though, is that people who can see that option (putting ALL our resources into this one tool) would be a ridiculous extremum, seem to think that the only logical alternative is NONE!

Roger • August 8, 2006 9:45 PM

@Brandioch Conner:

No, what you end up with is the old “7 degrees of Kevin Bacon” situation.

You might, if you approached it naively.

If you actually used social link analysis — which is a well studied, formal discipline in sociology and network theory — you do much better, even with rules only slightly more complicated than “can be joined with 6 or fewer links”.

For example, see the URL linked through my name. In that real world case study, the author started with two suspects identified in January 2000, and by applying quite simple rules in mapping their social network, developed a map of 43 nodes in which:
* every person on the map turns out to have been proven, or strongly suspected, of being a terrorist for reasons unrelated to the map;
* all of the 9/11 hijackers and both of the USS Cole bombing suspects are on the map; and
* Mohammed Atta is identified as the probable leader.

The data to generate this analysis was all available to the US government before the 9/11 attacks. Hence the perhaps belated interest in this sort of analysis.

Incidentally, the original two suspects on this map (Nawaf al-Hazmi and Khalid al-Mihdhar) were initially identified through bugs in the house of Sameer Mohammed Ahmed al-Hada (Khalid’s brother-in-law) in Yemen, and then followed to a terrorist conference in Malaysia. Ahmed’s house had been identified as an Al-Qaeda safe house in investigations of the Nairobi embassy bombing, and tracing calls made to it was also the method by which Osama bin Laden’s sat phone was identified. It is claimed that this house continued to be a valuable source of intelligence until it was raided by Yemeni police in 2002 — for non-payment of rent!