Bruce Schneier | |||||||||||
Schneier on SecurityA blog covering security and security technology. « Stealing Source Code | Main | Protecting Your Privacy at International Borders » January 10, 2012Collecting Expert Predictions about Terrorist AttacksJohn Mueller has been collecting them: Some 116 of these Very People were surveyed in 2006 by Foreign Policy magazine in a joint project with the Center for America Progress. The magazine stressed that its survey drew from the "highest echelons of America’s foreign policy establishment" and included the occasional secretary of state and national security adviser, as well as top military commanders, seasoned members of the intelligence community, and academics and journalists of the most "distinguished" nature. Over three-quarters of them had been in government service, 41 percent for over ten years. The musings of this group, it was proposed, could provide "definitive conclusions" about the global war on terror. I've never heard this particular quote before, and find it particularly profound: In 2004, Russell Seitz plausibly proposed that "9/11 could join the Trojan Horse and Pearl Harbor among stratagems so uniquely surprising that their very success precludes their repetition".... More predictions here. Posted on January 10, 2012 at 6:56 AM • 28 Comments To receive these entries once a month by e-mail, sign up for the Crypto-Gram Newsletter. Rookie • January 10, 2012 7:55 AM It's easy to find fault with people who have the job of trying to predict the future, whether it's the weather, sports outcomes, or terrorism. Of course, they still need to be held accountable when their predictions drive public policy, but it's an inexact science by definition. What I would find more interesting is an analysis on why their predictions where wrong and there were no significant, coordinated terrorist attacks on US soil since 2011. I can't believe it's because they don't want to strike the US. Are the US intelligence agencies doing so well at interdicting the plans and the planners? Have the wars successfully carried the terrorist fight to their front door? Are the terrorist group's capabilities that degraded? I have my opinions, but I think it would be valuable to have a definitive answer to that question to help drive policy for the next 10 years. Nobodyspecial • January 10, 2012 8:00 AM Actually if the Trojans had the TSA - the Horse would have worked again. While of course banning any wooden horses, pictures of horses, mention of horses and country-and-western songs. Graham • January 10, 2012 8:02 AM Of course, it is possible for something to be "likely" in terms of having a high probability, and then for it to not happen, so for all we know the 70% could have been correct. All we can definitely claim is that reality was inconsistent with the view of the 9% who predicted certainty. Michael Toecker • January 10, 2012 8:07 AM I wonder if Stuxnet would fall into the category of "infamous, not repeatable" or if we haven't learned the lesson yet. Mike Toecker Mark R • January 10, 2012 8:24 AM I heard a talk recently about the perceived vs. actual value of experience in analytical fields. I apologize for not having any link to the study, but one specific example centered on polygraph testers. Experts in the field cite numerous techniques for identifying when a person is lying (tics, mannerisms, etc.). The study found that years of experience did not significantly improve the testers' ability to identify dishonest statements. They did, however, significantly increase the testers' confidence in their assessments. A further finding was that "asking a panel of experts" is a remarkably bad way of arriving at correct answers. Better results can be obtained by asking a single expert before all the experts have a chance to talk to each other. Clive Robinson • January 10, 2012 8:45 AM @ Bruce, Now you have some time on your hands before the "book signing" rounds.... You could have a look back at what posters to this blog have said in the way of the same predictions, and also where they have given their reasoning behind it, and the accuracy of their predictions. I've made a fairly consistant point that as long as the US are fighting on the home soil of various countries, 9/11 style attacks would not be attempted. I've also predicted that there would be more Richard "hotfoot" Ried style attacks that would achieve little or no harm but plenty of publicity to keep money roling into the terrorist coffers. And it kind of turned out that way including the "laser printer bombs". And I'm guessing that due to the Olympics being in London this year, we can expect to see some form of "attention grabbing" activities be it from the terrorist or from the authorities of which the latter I think will be an order or two of magnitude more likely. What I'm awaiting on is what will happen now the US has pulled out of certain territorial areas. If nothing else had changed I would expect more terrorist activity on or near US territory. However contrary to the way it is presented in the news, various political interests are starting to engage with the political elements of various "terrorist" organisations and I suspect that this "recognition" unless badly mismanaged will provide a limiting effect in the short term. This is important because "Arab Spring" has probably had more of an effect on the terrorist organisations than we give credit for. In the long term the current "arab terrorist" organisations days are effectivly numbered as the days of the opressive regimes are ending, and their only hope to survive with influence is to gain political representation before the time other interests get political power. Brandioch Conner • January 10, 2012 9:21 AM The only proof is prediction. If you cannot accurately predict, then you do not understand the situation. @Rookie Because terrorists are extremely rare. Terrorists who can operate in the USofA are even rarer. (Think about your ability to operate in Iran. Can you read the language? Speak the language? Tell a cop from a crossing-guard?) Terrorists who can operate in the USofA while keeping their mission a secret long enough get the supplies needed to carry it out are the rarest of all. They are so rare that no practical predictions can be made about them. Which is why I agree with Bruce on spending the money devoted to "fighting terrorism" on areas such as emergency response and police investigations. At least that will have some outside benefits. Michael Josem • January 10, 2012 9:43 AM Isn't the response to this, "It is like taking an umbrella outside when it is raining, and complaining about staying dry"? No One • January 10, 2012 9:56 AM Michael Josem: It's more like looking at the 10-day forecast from ten top meteorologists, having seven of them predict 70% chance of rain, one predict 100% chance of rain and two predict No One • January 10, 2012 9:57 AM <30% chance of rain and then there being no rain and only two or three vaguely threatening clouds. (Open angle bracket ate the end of my first comment.) Bryan • January 10, 2012 10:29 AM Some time ago I read a book by Steven Schnaars called Megamistakes. I think he found that only 21% of forecasts become "true". So, it looks like expert predictions are in the ball-park. Daniel • January 10, 2012 10:41 AM With all due respect to the cognoscenti I said what Russel Seitz said on 9/12/2001. Despite all the hoopla OBL did not exploit a security vulnerability; he exploited a psycho-cultural vulnerability. In fact, that vulnerability window in the American psyche didn't even last an entire day as flight 93 proved. That was my first realization. My second realization took a few more days to form which was that no one would care and we would spend billions of dollars fixing a problem that would never reoccur. My third realization which came about a mouth later was that I was too "nice" (read: In hindsight my one big mistake was that while I got the direction right I totally underestimated the magnitude of the effect. In wasn't until about three years later that I realized that the 9/11 overreaction was no passing fancy but rather a generational problem and that rather than billions of dollars it would be trillions. echowit • January 10, 2012 10:51 AM What might be interesting would be a comparison of the severity/certainty of their predections and their subsequent "popularity" (rise or failure to rise) within the bureaucracy/hierarchy of their respective fields. Any guesses? the other Alan • January 10, 2012 11:28 AM "Predictions are hard to make--especially about the future" --Yogi Berra Snarki, child of Loki • January 10, 2012 12:04 PM You can play the same game over the next 10 years, and guess what? Most likely, your prediction will be WRONG. Doesn't matter whether you predict big attacks, no attacks, whatever: most likely WRONG. So the only real question for those making public prediction is: what type of wrong prediction does your career/reputation more good or harm? Sure, you might get "lucky" and make an accurate prediction, but it would be foolish to stake your career on it. Anton • January 10, 2012 2:07 PM 'In 2004, Russell Seitz plausibly proposed that "9/11 could join the Trojan Horse and Pearl Harbor among stratagems so uniquely surprising that their very success precludes their repetition"....' This is more like a definition of which we have 3 instances already. Interesting will be what shape and form the next one will take. Figureitout • January 10, 2012 2:32 PM Wonder what their predictions are regarding: The next war? I imagine a scenario where the input into their modeling software caused the supercomputer to simply explode... ted • January 10, 2012 3:34 PM I have to wonder if the scarier predictions were made for no other reason so that laws like the patriot act could be passed with little opposition or to justify NSA eavesdropping. Brett O • January 10, 2012 3:54 PM 1. You cant predict what you cant define (terrorist, terrorism, etc.) berick • January 10, 2012 5:43 PM Pearl Harbor is not like the others. Americans in the Philippines were caught just as off-guard despite MacArthur having 9 hours warning. The Japanese executed a similar surprise attack, though naval instead of airborne, in 1904 against the Russians at Port Arthur. It is easy to forget the lessons of history, or to learn the wrong lessons. Dave • January 10, 2012 5:58 PM @Graham: Came here to say roughly the same thing, but I think you worded too weakly - we know that the 9% who claimed certainty where wrong, yes. We don't know that those who said it was likely were wrong - but we do know that they were probably wrong. S Taylor • January 10, 2012 7:39 PM Simply put, the predictions were a function of FUD. I bet we hear some of this kind of FUD after the conventions when the presidential campaigns are in full swing. NobodySpecial • January 10, 2012 8:28 PM Even more ironic, the attack on Pearl Harbour was an almost exact copy of the British navy's sinking of the Italian fleet in Taranto almost exactly one year earlier. This was the first use of carrier borne aircraft attack on another fleet and was carefully studied by the Japanese navy - although unfortunately not by the US navy. cagey_C • January 11, 2012 2:22 PM While all three listed "stratagems" were tactically successful, only the Trojan Horse brought about the strategic victory the attackers sought. One could argue that the other two (Pearl Harbor and 9/11) were actually strategic blunders. Petréa Mitchell • January 11, 2012 2:37 PM This makes an interesting context in which to consider the Doomsday Clock news. Is it possible it's just a longer-running version of the same phenomenon? I don't know if the Bulletin of Atomic Scientists has produced as many testable predictions, though. EIA • January 11, 2012 6:13 PM ..something about looking hard enough for something, and it being found - whether it's there or not, or in between. If that fails, perhaps manufacture it. Last resort, make a soap-opera, or sign NDAA. http://tinyurl.com/7tjt736 - ? With "Homeland Security" now part of nearly every university's curriculum, terror should be quite predictable. Joe G • January 11, 2012 10:10 PM @Mike Toecker I think that instead, Stuxnet follow-ups are Unfamous but Infinitely Repeatable johndburger • January 12, 2012 10:39 AM By Mueller's reasoning, the experts who said "unlikely" were also wrong, since "no chance" was a possible choice on the poll. The actual post facto probability being zero, the only correct choice was "no chance". This whole thing just illustrates how profoundly wrong-headed these kinds of formulations are.
Post a comment
Powered by Movable Type. Photo at top by Geoffrey Stone.
Schneier.com is a personal website. Opinions expressed are not necessarily those of BT. |
|
Comments