Entries Tagged "risk assessment"
Page 10 of 21
Mark Twain on Risk Analysis
From 1871:
I hunted up statistics, and was amazed to find that after all the glaring newspaper headings concerning railroad disasters, less than three hundred people had really lost their lives by those disasters in the preceding twelve months. The Erie road was set down as the most murderous in the list. It had killed forty-six—or twenty-six, I do not exactly remember which, but I know the number was double that of any other road. But the fact straightway suggested itself that the Erie was an immensely long road, and did more business than any other line in the country; so the double number of killed ceased to be matter for surprise.
By further figuring, it appeared that between New York and Rochester the Erie ran eight passenger trains each way every day—sixteen altogether; and carried a daily average of 6,000 persons. That is about a million in six months—the population of New York city. Well, the Erie kills from thirteen to twenty-three persons out of its million in six months; and in the same time 13,000 of New York’s million die in their beds! My flesh crept, my hair stood on end. “This is appalling!” I said. “The danger isn’t in travelling by rail, but in trusting to those deadly beds. I will never sleep in a bed again.”
Scaring the Senate Intelligence Committee
At Tuesday’s hearing, Senator Dianne Feinstein, Democrat of California and chairwoman of the Senate Intelligence Committee, asked Mr. Blair [the Director of National Intelligence] to assess the possibility of an attempted attack in the United States in the next three to six months.
He replied, “The priority is certain, I would say”—a response that was reaffirmed by the top officials of the C.I.A. and the F.B.I.
I don’t know what “the priority is certain” actually means, but now everyone is reporting that these agencies claim there will be a terrorist attack in the U.S. during the next six months.
The Power Law of Terrorism
Research result #1: “A Generalized Fission-Fusion Model for the Frequency of Severe Terrorist Attacks,” by Aaron Clauset and Frederik W. Wiegel.
Plot the number of people killed in terrorists attacks around the world since 1968 against the frequency with which such attacks occur and you’ll get a power law distribution, that’s a fancy way of saying a straight line when both axis have logarithmic scales.
The question, of course, is why? Why not a normal distribution, in which there would be many orders of magnitude fewer extreme events?
Aaron Clauset and Frederik Wiegel have built a model that might explain why. The model makes five simple assumptions about the way terrorist groups grow and fall apart and how often they carry out major attacks. And here’s the strange thing: this model almost exactly reproduces the distribution of terrorists attacks we see in the real world.
These assumptions are things like: terrorist groups grow by accretion (absorbing other groups) and fall apart by disintegrating into individuals. They must also be able to recruit from a more or less unlimited supply of willing terrorists within the population.
Research Result #2: “Universal Patterns Underlying Ongoing Wars and Terrorism,” by Neil F. Johnson, Mike Spagat, Jorge A. Restrepo, Oscar Becerra, Juan Camilo Bohorquez, Nicolas Suarez, Elvira Maria Restrepo, and Roberto Zarama.
In the case of the Iraq war, we might ask how many conflicts causing ten casualties are expected to occur over a one-year period. According to the data, the answer is the average number of events per year times 10-2.3, or 0.005. If we instead ask how many events will cause twenty casualties, the answer is proportional to 20-2.3. Taking into account the entire history of any given war, one finds that the frequency of events on all scales can be predicted by exactly the same exponent.
Professor Neil Johnson of Oxford University has come up with a remarkable result regarding these power laws: for several different wars, the exponent has about the same value. Johnson studied the long-standing conflict in Colombia, the war in Iraq, the global rate of terrorist attacks in non-G7 countries, and the war in Afghanistan. In each case, the power law exponent that predicted the distribution of conflicts was close to the value 2.5.
This doesn’t surprise me; power laws are common in naturally random phenomena.
The Comparative Risk of Terrorism
Good essay from the Wall Street Journal:
It might be unrealistic to expect the average citizen to have a nuanced grasp of statistically based risk analysis, but there is nothing nuanced about two basic facts:
(1) America is a country of 310 million people, in which thousands of horrible things happen every single day; and
(2) The chances that one of those horrible things will be that you’re subjected to a terrorist attack can, for all practical purposes, be calculated as zero.
Consider that on this very day about 6,700 Americans will die…. Consider then that around 1,900 of the Americans who die today will be less than 65, and that indeed about 140 will be children. Approximately 50 Americans will be murdered today, including several women killed by their husbands or boyfriends, and several children who will die from abuse and neglect. Around 85 of us will commit suicide, and another 120 will die in traffic accidents.
[…]
Indeed, if one does not utter the magic word “terrorism,” the notion that it is actually in the best interests of the country for the government to do everything possible to keep its citizens safe becomes self-evident nonsense. Consider again some of the things that will kill 6,700 Americans today. The country’s homicide rate is approximately six times higher than that of most other developed nations; we have 15,000 more murders per year than we would if the rate were comparable to that of otherwise similar countries. Americans own around 200 million firearms, which is to say there are nearly as many privately owned guns as there are adults in the country. In addition, there are about 200,000 convicted murderers walking free in America today (there have been more than 600,000 murders in America over the past 30 years, and the average time served for the crime is about 12 years).
Given these statistics, there is little doubt that banning private gun ownership and making life without parole mandatory for anyone convicted of murder would reduce the homicide rate in America significantly. It would almost surely make a major dent in the suicide rate as well: Half of the nation’s 31,000 suicides involve a handgun. How many people would support taking both these steps, which together would save exponentially more lives than even a—obviously hypothetical—perfect terrorist-prevention system? Fortunately, very few. (Although I admit a depressingly large number might support automatic life without parole.)
Or consider traffic accidents. All sorts of measures could be taken to reduce the current rate of automotive carnage from 120 fatalities a day—from lowering speed limits, to requiring mechanisms that make it impossible to start a car while drunk, to even more restrictive measures. Some of these measures may well be worth taking. But the point is that at present we seem to consider 43,000 traffic deaths per year an acceptable cost to pay for driving big fast cars.
Kevin Drum takes issue with the analysis:
Two things. First, this line of argument—that terrorism is statistically harmless compared to lots of other activities—will never work. For better or worse, it just won’t. So we should knock it off.
Second, even in the realm of pure logic it really doesn’t hold water. The fundamental fear of terrorism is that it’s not just random or unintentional, like car accidents or (for most of us) the threat of homicide. It’s carried out by people with a purpose. The panic caused by the underwear bomber wasn’t so much over the prospect of a planeload of casualties, it was over the reminder that al-Qaeda is still out there and still eager to expand its reach and kill thousands if we ever decide to let our guard down a little bit.
So even if you agree with Campos, as I do, that overreaction to al-Qaeda’s efforts is dumb and counterproductive, it’s perfectly reasonable to be more afraid of a highly motivated group with malign ideology and murderous intent than of things like traffic accidents or hurricanes. Suggesting otherwise, in some kind of hyperlogical a-death-is-a-death sense, strikes most people as naive and clueless. It’s an argument that probably hurts the cause of common sense more than it helps.
While I agree that arguing that terrorism is statistically harmless isn’t going to win any converts, I still think it’s an important point to make. We routinely overestimate rare risks and underestimate common risks, and the more we recognize that cognitive bias, the better chance we have for overcoming it.
And Kevin illustrates another cognitive bias: we fear risks deliberately perpetrated by other people more than we do risks that occur by accident. And while we fear the unknown—the “reminder that al-Qaeda is still out there and still eager to expand its reach and kill thousands if we ever decide to let our guard down a little bit”—more than the familiar, the reality is that automobiles will kill over 3,000 people this month, next month, and every month from now until the foreseeable future, irrespective of whether we let our guard down or not. There simply isn’t any reasonable scenario by which terrorism even approaches that death toll.
Yes, the risks are different. Your personal chance of dying in a car accident depends on where you live, how much you drive, whether or not you drink and drive, and so on. But your personal chance of dying in a terrorist attack also depends on these sorts of things: where you live, how often you fly, what you do for a living, and so on. (There’s also a control bias at work: we underestimate the risk in situations where we’re in control, or think we’re in control—like driving—and overestimate the risks in situations where we’re not in control.) But as a nation we get to set our priorities, and decide how to spend our money. No one is suggesting we ignore the risks of terrorism—and making people feel safe is a good thing to do—but it makes no sense to focus so much effort and money on it when there are far worse risks to Americans.
Jeffrey Rosen wrote about this last year. And similar sentiments from Baroness Murphy of the British House of Lords.
Remember, the terrorists want us to be terrorized, and they’ve chosen this tactic precisely because we have all these cognitive biases that magnify their actions. We can fight back by refusing to be terroroized.
My Second CNN.com Essay on the Underwear Bomber
This one is about our tendency to overreact to rare risks, and is an update of this 2007 essay.
I think we should start calling them the “underpants of mass destruction.”
Nate Silver on the Risks of Airplane Terrorism
Over at fivethirtyeight.com, Nate Silver crunches the numbers and concludes that, at least as far as terrorism is concerned, air travel is safer than it’s ever been:
In the 2000s, a total of 469 passengers (including crew and terrorists) were killed worldwide as the result of Violent Passenger Incidents, 265 of which were on 9/11 itself. No fatal incidents have occurred since nearly simultaneous bombings of two Russian aircraft on 8/24/2004; this makes for the longest streak without a fatal incident since World War II. The overall death toll during the 2000s is about the same as it was during the 1960s, and substantially less than in the 1970s and 1980s, when violent incidents peaked. The worst individual years were 1985, 1988 and 1989, in that order; 2001 ranks fourth.
Of course, there is a lot more air travel now than there was a couple of decades ago. Although worldwide data is difficult to obtain, U.S. air travel generally expanded at rates of 10-15% per year from the 1930s through 9/11. If we assume that U.S. air traffic represents about a third of the worldwide total (the U.S. share of global GDP, which is probably a reasonable proxy, has fairly consistently been between 26-28% during this period), we can estimate the number of deaths from Violent Passenger Incidents per one billion passenger boardings. By this measure, the 2000s tied the 1990s for being the safest on record, each of which were about six times safer than any previous decade. About 22 passengers per one billion enplanements were killed as the result of VPIs during the 2000s; this compares with a rate of about 191 deaths per billion enplanements during the 1960s.
Why? Because over the past decade, the risk of airplane terrorism has been very low:
Over the past decade, according to BTS, there have been 99,320,309 commercial airline departures that either originated or landed within the United States. Dividing by six, we get one terrorist incident per 16,553,385 departures.
These departures flew a collective 69,415,786,000 miles. That means there has been one terrorist incident per 11,569,297,667 mles flown. This distance is equivalent to 1,459,664 trips around the diameter of the Earth, 24,218 round trips to the Moon, or two round trips to Neptune.
Assuming an average airborne speed of 425 miles per hour, these airplanes were aloft for a total of 163,331,261 hours. Therefore, there has been one terrorist incident per 27,221,877 hours airborne. This can also be expressed as one incident per 1,134,245 days airborne, or one incident per 3,105 years airborne.
There were a total of 674 passengers, not counting crew or the terrorists themselves, on the flights on which these incidents occurred. By contrast, there have been 7,015,630,000 passenger enplanements over the past decade. Therefore, the odds of being on given departure which is the subject of a terrorist incident have been 1 in 10,408,947 over the past decade. By contrast, the odds of being struck by lightning in a given year are about 1 in 500,000. This means that you could board 20 flights per year and still be less likely to be the subject of an attempted terrorist attack than to be struck by lightning.
In 2008, 37,000 people died in automobile accidents—the lowest number since 1961. Even so, that’s more than a 9/11 worth of fatalities every month, month after month, year after year.
There are all sorts of psychological biases that cause us to both misjudge risk and overreact to rare risks, but we can do better than that if we stop and think rationally.
Immediacy Affects Risk Assessments
New experiment demonstrates what we already knew:
That’s because people tend to view their immediate emotions, such as their perceptions of threats or risks, as more intense and important than their previous emotions.
In one part of the study focusing on terrorist threats, using materials adapted from the U.S. Department of Homeland Security, Van Boven and his research colleagues presented two scenarios to people in a college laboratory depicting warnings about traveling abroad to two countries.
Participants were then asked to report which country seemed to have greater terrorist threats. Many of them reported that the country they last read about was more dangerous.
“What our study has shown is that when people learn about risks, even in very rapid succession where the information is presented to them in a very clear and vivid way, they still respond more strongly to what is right in front of them,” Van Boven said.
[…]
Human emotions stem from a very old system in the brain, Van Boven says. When it comes to reacting to threats, real or exaggerated, it goes against the grain of thousands of years of evolution to just turn off that emotional reaction. It’s not something most people can do, he said.
“And that’s a problem, because people’s emotions are fundamental to their judgments and decisions in everyday life,” Van Boven said. “When people are constantly being bombarded by new threats or things to be fearful of, they can forget about the genuinely big problems, like global warming, which really need to be dealt with on a large scale with public support.”
In today’s 24-hour society, talk radio, the Internet and extensive media coverage of the “threat of the day” only exacerbate the trait of focusing on our immediate emotions, he said.
“One of the things we know about how emotional reactions work is they are not very objective, so people can get outraged or become fearful of what might actually be a relatively minor threat,” Van Boven said. “One worry is some people are aware of these kinds of effects and can use them to manipulate our actions in ways that we may prefer to avoid.”
[…]
“If you’re interested in having an informed citizenry you tell people about all the relevant risks, but what our research shows is that is not sufficient because those things still happen in sequence and people will still respond immediately to whatever happens to be in front of them,” he said. “In order to make good decisions and craft good policies we need to know how people are going to respond.”
The Exaggerated Fears of Cyber-War
Good article, which basically says our policies are based more on fear than on reality.
On cyber-terrorism:
So why is there so much concern about “cyber-terrorism”? Answering a question with a question: who frames the debate? Much of the data are gathered by ultra-secretive government agencies—which need to justify their own existence—and cyber-security companies—which derive commercial benefits from popular anxiety. Journalists do not help. Gloomy scenarios and speculations about cyber-Armaggedon draw attention, even if they are relatively short on facts.
Politicians, too, deserve some blame, as they are usually quick to draw parallels between cyber-terrorism and conventional terrorism—often for geopolitical convenience—while glossing over the vast differences that make military metaphors inappropriate. In particular, cyber-terrorism is anonymous, decentralized, and even more detached than ordinary terrorism from physical locations. Cyber-terrorists do not need to hide in caves or failed states; “cyber-squads” typically reside in multiple geographic locations, which tend to be urban and well-connected to the global communications grid. Some might still argue that state sponsorship (or mere toleration) of cyber-terrorism could be treated as casus belli, but we are yet to see a significant instance of cyber-terrorists colluding with governments. All of this makes talk of large-scale retaliation impractical, if not irresponsible, but also understandable if one is trying to attract attention.
Much of the cyber-security problem, then, seems to be exaggerated: the economy is not about to be brought down, data and networks can be secured, and terrorists do not have the upper hand.
On cyber-war:
Putting these complexities aside and focusing just on states, it is important to bear in mind that the cyber-attacks on Estonia and especially Georgia did little damage, particularly when compared to the physical destruction caused by angry mobs in the former and troops in the latter. One argument about the Georgian case is that cyber-attacks played a strategic role by thwarting Georgia’s ability to communicate with the rest of the world and present its case to the international community. This argument both overestimates the Georgian government’s reliance on the Internet and underestimates how much international PR—particularly during wartime—is done by lobbyists and publicity firms based in Washington, Brussels, and London. There is, probably, an argument to be made about the vast psychological effects of cyber-attacks—particularly those that disrupt ordinary economic life. But there is a line between causing inconvenience and causing human suffering, and cyber-attacks have not crossed it yet.
The real risk isn’t cyber-war or cyber-terrorism, it’s cyber-crime.
Risk Intuition
People have a natural intuition about risk, and in many ways it’s very good. It fails at times due to a variety of cognitive biases, but for normal risks that people regularly encounter, it works surprisingly well: often better than we give it credit for.
This struck me as I listened to yet another conference presenter complaining about security awareness training. He was talking about the difficulty of getting employees at his company to actually follow his security policies: encrypting data on memory sticks, not sharing passwords, not logging in from untrusted wireless networks. “We have to make people understand the risks,” he said.
It seems to me that his co-workers understand the risks better than he does. They know what the real risks are at work, and that they all revolve around not getting the job done. Those risks are real and tangible, and employees feel them all the time. The risks of not following security procedures are much less real. Maybe the employee will get caught, but probably not. And even if he does get caught, the penalties aren’t serious.
Given this accurate risk analysis, any rational employee will regularly circumvent security to get his or her job done. That’s what the company rewards, and that’s what the company actually wants.
“Fire someone who breaks security procedure, quickly and publicly,” I suggested to the presenter. “That’ll increase security awareness faster than any of your posters or lectures or newsletters.” If the risks are real, people will get it.
You see the same sort of risk intuition on motorways. People are less careful about posted speed limits than they are about the actual speeds police issue tickets for. It’s also true on the streets: people respond to real crime rates, not public officials proclaiming that a neighbourhood is safe.
The warning stickers on ladders might make you think the things are considerably riskier than they are, but people have a good intuition about ladders and ignore most of the warnings. (This isn’t to say that some people don’t do stupid things around ladders, but for the most part they’re safe. The warnings are more about the risk of lawsuits to ladder manufacturers than risks to people who climb ladders.)
As a species, we are naturally tuned in to the risks inherent in our environment. Throughout our evolution, our survival depended on making reasonably accurate risk management decisions intuitively, and we’re so good at it, we don’t even realise we’re doing it.
Parents know this. Children have surprisingly perceptive risk intuition. They know when parents are serious about a threat and when their threats are empty. And they respond to the real risks of parental punishment, not the inflated risks based on parental rhetoric. Again, awareness training lectures don’t work; there have to be real consequences.
It gets even weirder. The University College London professor John Adams popularised the metaphor of a mental risk thermostat. We tend to seek some natural level of risk, and if something becomes less risky, we tend to make it more risky. Motorcycle riders who wear helmets drive faster than riders who don’t.
Our risk thermostats aren’t perfect (that newly helmeted motorcycle rider will still decrease his overall risk) and will tend to remain within the same domain (he might drive faster, but he won’t increase his risk by taking up smoking), but in general, people demonstrate an innate and finely tuned ability to understand and respond to risks.
Of course, our risk intuition fails spectacularly and often, with regards to rare risks , unknown risks, voluntary risks, and so on. But when it comes to the common risks we face every day—the kinds of risks our evolutionary survival depended on—we’re pretty good.
So whenever you see someone in a situation who you think doesn’t understand the risks, stop first and make sure you understand the risks. You might be surprised.
This essay previously appeared in The Guardian.
EDITED TO ADD (8/12): Commentary on risk thermostat.
Sidebar photo of Bruce Schneier by Joe MacInnis.