Schneier on Security

Looks as if DHS & TSA clearly understand that notion of Perceived Risk vs. Actual Risk. And decided for the security-marketing efficient actions.
Isn't a pragmatic behaviour ? Acting on people perceived needs ...

I'm surprised at a point that's missing from this list. I've always understood that people worry more about dying in an air crash than on the drive to the airport because they're not behind the wheel of the plane, so they don't feel in control.

"I'm surprised at a point that's missing from this list. I've always understood that people worry more about dying in an air crash than on the drive to the airport because they're not behind the wheel of the plane, so they don't feel in control."

That's covered in my fourth point.

The distinction between accidental and intentional perils is more significant than you give it credit. While accidents may be independently random events, deliberate attacks are anything but. There is an intelligent and malicious agent behind thee latter, whose future behaviour may well depend on our reactions to the attacks.

BTW Bruce the author is Daniel Gilbert (not Dennis)

> Europeans routinely perceive the U.S. as being full of guns.

Compared to (most of) Europe, it is. Of course, whether the increase in risk is large enough so that you really have to worry about it is another question entirely.

> > Europeans routinely perceive the U.S. as being full of guns.

> Compared to (most of) Europe, it is. Of course, whether the increase in risk is large enough [...]

There are also those who argue that an armed population can *decrease* the risk (due to self-defense / deterrence factors).

@Frank Ch. Eigler

The differences between random, non-intelligent threat sources and adaptive, intelligent threat sources are well-understood, but the differences do not justify the often disproportionate fear of high-profile low-probability events. Most potential victims of attacks are not targeted as individuals. It would be rational to treat deliberate high-profile attacks that have a limited potential to harm a huge number of victims as if they were random, probabilistic events, at least for the purpose of determining one's fear level.

It amazes me that I have been aware of this and have even preached it in my workplace, but I still succumb to the problems in my personal life.

I live in an area where the 2002 sniper incidents were occurring and remember actually feeling like I could be in someone's cross-hairs, but I continue to live there despite it being one of the most violent crime-ridden areas in the United States.

And I continue to question why my spouse rightly thinks it's stupid for me to want to ride a motorcycle when I have so many little dependents wanting me home every night.

I don't think my "add on utility" has even gotten into beta testing yet.

I have also recently finished reading Dan Gilbert's book, "Stumbling on Happiness", and would like to add my recommendation. The book is a fascinating look at how we really think, as opposed to how we think we think. And Gilbert is one of those rare writers (like Richard Dawkins, Steven Pinker, and Bruce Schneier) who not only has valuable things to say, but says them very well.

Regarding "if two planes were hit by lightning and hit a skyscraper" - hardly anybody remembers that a B-25 bomber hit the Empire State Building a number of years ago, much less the exact date.

http://www.disastershq.com/features/empire.asp

Risk is risk, whether the perpetrator does it on purpose or by accident. Fear is what changes our perception of that risk.

The U.S. has been a generally safe place because of freedom and rights, but now we're told that our safety requires we relinquish these cherished notions, and also increase the size of our tax burden (or debt based on the current "no need to tax to pay for what we're buying" policies).

He could also point out that people have no trouble spending other people's money the same as they do spending their own. That's why people are often for taxes on others and against taxes on them. Why not just be fair the way justice, freedom, democracy, capitalism and equal rights suggest? A common flat sales tax on all purchases (stocks, food, houses, medical care) will never be policy because it's simple and fair and doesn't pit one against the other.

As for airplane safety, I know the bit about "miles traveled" is safer, but it is also safer per "trip taken" or even "hours in transit"?

@Valdis Kletnieks: well, the 'number' is on the order of 60 years, thats asking a lot. A lot of people dont believe in the holocaust of about the same time frame and ~428,000 times as many people lost their lives in that.

What they should take away from the memory of the B-25 vs ESB, if anything, is that it did very little damage; it was below the "quantum event" level of airplanes hitting buildings.

@MSB
MSB wrote: "It would be rational to treat deliberate high-profile attacks as if they were random, probabilistic events"

I disagree. As human adversaries are likely to escalate their actions in reaction to success, higher fear levels just factor in the future results of a non-reaction.

Also, fear has an evolutionary role in propelling members of a group to beneficial long-term action which may increase their short-term risk; a group of cavemen attacking a predator (or a hostile tribe) may in fact have increased long-term survival for their group, but the attack itself, motivated by fear, was risky. The reality with human adversaries is that, if unchecked, the threat level tends to grow with time, unlike that of lightning strikes.

I think another major factor in people's fear of flying is also the perceived chance of catastrophic versus minor failure.

Simply put, when a plane falls out of midair, there are virtually always no survivors. When a car crashes, most of the time the passengers are relatively uninjured.

It's still an apples and oranges comparison, because we're mentally considering the worst-case scenario for air travel against a best-case scenario for car travel. Planes can have relatively small incidents (single engine failure, instrument failure, etc.) and cars can experience catastrophic failure (head-on crash at 70mph, etc.). We just don't perceive it that way.

Frank Ch. Eigler

The distinction between accidental and intentional perils is more significant than you give it credit. While accidents may be independently random events, deliberate attacks are anything but. There is an intelligent and malicious agent behind thee latter, whose future behaviour may well depend on our reactions to the attacks.

**********
There is some evolutionary truth to that. A predator needs to be treated differently than a forest fire. It's also true that humans tend to personify external forces (hence creating gods for volcanos, floods etc) to categorize the this way.

In many ways, however, 'terrorism' is more like accident or disease than conventional crime (theft, assault, murder) because it is primarily random in it's victims. Generalized precautions are more useful, as is the realization that the individual risk is quite low.

Here's a link to audio of a talk Mr. Gilbert gave on this topic at a conference:

http://www.boingboing.net/2006/06/17/psychology_of_bad_pr.html

It's a very interesting presentation on why we're so incredibly bad at evaluating risk and reward.

Well, I think they are not totally wrong to worry as much as they do about planes. The operative factor is not risk per mile travelled. After all, we could hardly drive to Australia, so the fact that it would be safer to fly than drive doesn't come into it. The operative factor is risk per trip. And I think risk per trip is probably higher for planes than cars. Or at least not so totally out of line as the risk per mile travelled obviously are.

Its the takeoffs and landings that are the problem.

I think a common problem is with the notion of risk avoidance. People often think as if giving up a risky thing is risk free, or has no profile.

As when an acquaintance systematically eliminated all fat from her diet, thinking as she did so that each food lost was getting rid of a source of risk. In the end I pointed out to her that calories from fat had probably fallen below 5%, and that this was not just risky, it was incredibly risky.

Another UK acquaintance of mine substituted third world canned beef for fresh meat during the BSE scare, assuming it to be risk free. Without the question of the cleanliness of the plants and the controls on the health of the cattle ever occurring to her.

@ David

> A common flat sales tax on all purchases (stocks, food, houses, medical care) will
> never be policy because it's simple and fair and doesn't pit one against the other.

That's a pretty sweeping statement to use as a throwaway comment without any sort of validation, and depends entirely upon your definition of "fair".

A common flat sales tax could certainly be regarded as not fair if a disproportionate amount of your income needs to be spent on cost of living. If you're spending 70% of your income on cost of living and 14% (arbitrarily assinging a 20% sales tax) of your income on the taxes, you only have 16% of your income to save or spend on luxury items. On the other hand, if you spend 20% of your income on cost of living you're only spending 4% of your income on "cost of living" taxes, and you have 76% of your income to save or spend on luxury items.

A flat sales tax seems on the surface to be a fair shake, but many people would say it hardly seems fair to tell those who have to spend most of their income making ends meet that they're going to have to shell out 14% of their meager income to the tax man while someone who makes gobs of money is only shelling out 4% to cover basic necessities.

@ Reasonable:

"As human adversaries are likely to escalate their actions in reaction to success..."

Precisely. Terrorists seek to advance their respective causes by spreading and leveraging fear. When we react to their actions with disproportionately high levels of fear, we show them that their actions are successful, and encourage them to escalate their actions.

A worthy and timely discussion! I agree most with postings by Frank Ch. Eigler and Reasonable. In my opinion, they address Bruce's thesis from the get-go. I'll simply add that I, for one, respond with more concern to a crime than an accident as I would respond to having lost something verses having something stolen. In my opinion, it is a perceived vulnerability which is the rational component.

In response to the post by: jayh: "In many ways, however, 'terrorism' is more like accident or disease than conventional crime (theft, assault, murder) because it is primarily random in it's victims. Generalized precautions are more useful, as is the realization that the individual risk is quite low."

jayh, I think in ‘some ways’ crime is like an accident or disease with respect to randomness but certainly not in ‘most ways’ as you suggest. I believe the perception of individual risk depends on how connected one feels to the victims, ergo one’s ‘perceived losses’. But objectively differentiating one's 'perceived losses' from 'actual loses'; how do you do that intellectually? Isn't loss subjective? Or are you suggesting that this discussion draws a boundary between physical loss and emotional loss?

Philosophically, I am reminded that the wiser do not suffer from losses when they can see beyond them. At the same time one’s suffering and tears are as genuine as those of a child. -Rael

A good article but I laughed out loud when I read "out of whose way it should right now get". Sometimes ending a sentence with a preposition is preferable.

@grammar pedant

I laughed, too.

I thought he was being facetious, not pedantic. I also thought he was paraphrasing Churchill (".. up with which I will not put").

I pictured the writer with a twinkle in his eye and a wry grin, not a bookkeeper's green eyeshade and grammarian tomes.

@ Rael:

The unspoken context of this discussion is which measure of risk is appropriate when making decisions about security trade-off's. I believe that in this context, your notion of "perceived loss" is irrelevant. Yes, we might feel more grief if a loved one dies by murder than by accident, but how does that matter when deciding about precautions, especially since accidents are more common than murders? I believe jayh was right, crime is exactly like accident or disease in most of the ways that matter, when we talk about about precautions. Perceptions should not get in the way of making rational decisions.

@Pat Cahalan, @David

There can be no such thing as a fair tax, because the whole point of a tax is to enrich one group at the expense of another.

---

Bruce,

I am somewhat confused on the issue of Perceived Risk vs. Actual Risk because it is written in collectivist speak.

What do you mean by Actual Risk?

I contend that no such thing can exist. There is only perceived risk, and actual fact of occurrence. This may sound pedantic, but the word Actual & Risk sound oxymoronic. Risk is of speculative nature, Actual is a fact of reality.

There is only VARYING DEGREES of RISK.

For example it is generally taken for granted in the investment world that buying government bonds is a totally risk-free action. The problem with that is that it ignores all the numerable instances in history where governments have repudiated their debts. Therefore such an activity is not risk-free.

Any attempt to describe 'Actual Risk' should just be called Certainty. It can only really be used to describe natural phenomena, not social phenomena where individual volition plays a dominant role.

You Sir should become a politician.
Thank you for this great insight into human behavior!

> There are also those who argue that an > armed population can *decrease* the risk > (due to self-defense / deterrence
> factors).

If you compare homicide rates of the US and Europe I would highly doubt that!

But then again, European governments don't freak me out as much as the American government does.

@quincunx
Humans are poor Bayesians. That is, in circumstances where the risks can be calculated, people often do not do calculate them accurately. The difference could be called the difference between perceived and actual risk.

Bruce,

Extremely well written article, it's been circulating on IRC as well.
I'd love to somehow see this permeate the corridors of upper management though :)

As what is written, really is not not news in a grey / black hat type of assembly, if you indulge in social engineering even less so.
Heck - just look at Allstate commercials.

:)

All in all - thanks from a fan.

Read Kahneman and Tversky (Google, Wikipedia,...).

Daily exercise, eating healthy, and flu shots to prevent illness.

How can you claim we worry more about Anthrax?

That aside, death isn't everything. If anything, we worry about the following problems a terrorist attack would cause - look at the immediate recession caused by the last one. You claim we wouldn't remember a similar date not caused by terrorists. How about Black Monday, October 29th? That date doesn't even have an overly significant number of deaths, it was just an omen of hardships to come, much like the terrorist attacks.

Very interesting piece.

These behaviors are hardwired into our brains of course.. If you sit in a tub of water that is getting slowly hotter, you barely notice,

however, if someone else gets into your tub of water with you (after it has slowly heated up to a nice good hotness) they are likely to jump right back out!

(if it is hot enough, will likely remember that occasion and tell you and their friends about the level of surprise many times)

apparently, our bodies and our minds are made of the same stuff. The earth changes under us slowly everyday, and we are built to handle it. It would be interesting to study creatures that are able to handle drastic change.

Please dear God, Bruce run for office already.

SLASHDOTTED!

>> Europeans routinely perceive the U.S. as being full of guns.

> Compared to (most of) Europe, it is. Of course, whether the increase in risk is large enough so that you really have to worry about it is another question entirely.

Yes it is. Think twice before jumping out of a car after an accident and yelling at the other driver in America. I have zero fear of terrorists in this country, but worry much more about hot heads with guns (now disguised by permission.) OTH, they may hit you, but they are not likely to shoot you in the rest of the industrial world.

More generally, the issue of guns in America shows the tension between two of Bruce's points: fear of intentional harm versus indifference induced by repetition. Americans are killed 10 to 1 more often, but they are also more conditioned to accept the murder rate. Gun killings rarely make the news relative to the number that occur (20K plus). Between just 1960 and 1990 civilian death by guns exceded the total of deaths in all the wars throughout American history, now including the mess in Iraq.

Likewise, in a war zone, local citizens often become indifferent to killings, as long as it does not get too close to them. Soldiers couldn't function in combat if they did not decondition to the horrors of war. And war is very intentional.

@rjd

> Think twice before jumping out of a car after an accident and yelling
> at the other driver in America.

I'm not sure this makes the point you were hoping to make.

@Al:

One number I read (based on US stats) is the chance of being killed in a single aircraft trip is around 54 million to one. For a single car trip, 8 million to one. Given the phenomenally higher number of car trips most people make, it's even more dangerous, relatively.

'Humans are poor Bayesians. That is, in circumstances where the risks can be calculated, people often do not do calculate them accurately. The difference could be called the difference between perceived and actual risk.'

Hmm, still not clear.

Humans might be poor bayesians, but humans must still feed the proper relevant data into a bayesian filter. Ergo it still seems like circular reasoning.

When one says driving is a higher risk then flying - they are making this judgment based on a total number of accidents per total activity, i.e. historical data.

That is not risk calculation (or Actual Risk), that is merely an interesting historic fact.

A fact that CAN be used to make risk judgments, but by itself is NOT risk.

Anyone who claims to be able to compute this mysterious 'Actual Risk' ex-ante would be the richest person in the world, forever. This person, I hate to break it to everyone, does not exist.

If by Actual Risk one means criticizing people ex-post-facto for not doing something else (perhaps on the grounds of not considering the historical data) then it makes more sense. Is that meaningful though?

It still seems to me that Perceived vs Actual Risk is just a false dichotomy and an oxymoron.

There is only Perceived Risk of varying degrees made by various people at various times and at various locations under various influences. It is a subjective concept. And any attempts to find COSTS for risk can only be achieved via trade.

"But if the rate of change is slow enough, the change will go undetected."
Hot water and frogs - anurban legend.
http://answers.google.com/answers/threadview?id=758865

My theory about risks:
Repeate a million times, that there is a serious risk, and most people still don't believe, that it is a serious one, but they believe that the others believe so.

I was disappointed to see Bruce casually compare cars and planes by passenger miles. This is a well known fallacy - the point is that plane trips are by their nature very long (100s or 1000s of miles), whereas car trips are generally short. Of course, planes typically have more passengers. Maybe planes are safer than cars, maybe not. But passenger miles is not a valid metric of comparison.

There appears to a confusion here. On the one hand, people look at a statistic such as "50000 people die of such and such a thing every year" and simply extend that as a probability to the population of the country or planet or whatever. This is what makes news. This, to me, is plain stupidity and is a clear indication of how the mind is not built to handle probability well and how people manipulate this incapacity to make things news worthy.

The other rather real notion of risk is only that which you have no power to eliminate. As I was reading all the cases listed here, I thought - "No. I do worry about the possibility of slipping on the bathroom floor. I place non-slip floor mats there." However, people slipping and dying on the bathroom floor won't make news, but it *is* a real risk that we do take into account. We've been planning for accident prevention for so many eons that we now have no clue how we ever made it this far at all. For instance, look at the things that can happen to babies and youngsters. Falling down stairs, knocking your head on the floor, eating garbage, choking, blah blah blah to infinity and beyond. We don't do all that as adults except when we are in the "i don't care" mode.

From this perspective, I think this whole topic is biased by those who need to worry about what to publish and what not to, rather than what people really care about.

You need to understand the evolutionary survival systems in individual phenotypes have not evolved to deal with mass population sizes and mass media where news that happens in bumfuck nowhere becomes national news, people percieve national and international news as if it were local to them, and it's a 'defect' of evolutionary survival sysems.

Lastly people focus on problems, not on successes, our survival systems are more worried about protecting us, then the are about the mundane success stories. Because what becomes common place or percieved as "normal" does not belong in our awareness.

Lastly lastly theirs transposition errors or mixed up reactions by assocation...

i.e. Nuclear bombs bring thoughts of hiroshima, nuclear meltdowns bring thoughts of chernobyl, therefore nuclear power = bad.

The problem is news always focuses on the out of place or bad things, they never tell you that was one out of thousands of nuclear plants that operate perfectly normally.

News focuses on anomalies, but our threat survival systems and emotional reaction systems, are still stuck in "tribal" or village models.

Although there are more advanced phenotypes emerging among indivuduals who are more adapted to modern society... i.e. youth that shun TV and mass media, distrustful of government, media, etc.

Lastly public education would go a long way to preventing such manipulation fo the public, as well as media organizations becoming more interested in educating the public, the problem is that the profit motive has made media the worst.

If you wan to blame something, blame capitalist society, I bet in a small communist tribe or village, you would never see such blatant propaganda. The profit motive threatens integrity and being well-informed in modern society.

The drive to the airport usually is fraught with more opportunity for danger than a flight in a plane, because pilots are very thoroughly and professionally trained and aircraft are maintained to meticulous levels. In an automobile, your safety is in the hands of countless other drivers out there on the roadways, virtually all of whom are basically amateurs and most cars get mechanically maintained to the most minimal levels by high-school G.E.D. rednecks from PepBoys and Walmart.

Besides, as the old saying goes: flying is not dangerous... it's crashing that's dangerous , so just don't ever crash.

it’s hard i think at the end of the day to put an actual figure on the amount of stress rare risks (like getting clobbered in a terror attack) generate over the more common ones (like getting clobbered by a medical mistake) but it just might be that the cultural noise the rare ones generate is so seemingly out of proportion precisely because those risks are so uncommon as to require a more or less constant communal reminder to avoid them at all. kind of like a verbal string on a finger. “what’s this then? oh yeah, don’t snort the powder on the envelop. right-o.� we don’t need to be told how easy it is to crack our toes on the coffee table but we just might need a little help remembering to soak the unexpected letter in the tub before opening precisely because that explosive risk is so rare, yet the cost so dear, and so easy to avoid. furthermore i’m not entirely convinced the old saw about fear vs control is altogether accurate. i mean i’m no more in control in the back of a manhattan cab than i am flying over kennedy, but my knuckles (usually) aren’t white downtown, only over it.

- js.

"I was disappointed to see Bruce casually compare cars and planes by passenger miles. This is a well known fallacy - the point is that plane trips are by their nature very long (100s or 1000s of miles), whereas car trips are generally short. Of course, planes typically have more passengers. Maybe planes are safer than cars, maybe not. But passenger miles is not a valid metric of comparison."

The metric doesn't matter. Passenger miles. Passenger trips. Total trips. By any measure, planes are much safer than cars.

The statistics about airplane safety is designed to placate people's fear of flying. The safety of the Space Shuttle is extremely good when calculated per passenger kilometer, but per trip, it is a gamble that no Las Vegas regular will take...

Fear caused by overblown perception of risks will not actually reduce the risk. For example, public fear actually encourage terrorism, as that is its only intent. Real damage of terrorism is not significant compared with all other things: car accidents, crimes and such. If an enemy can cause significant damage, it will not resort to the use of terrorism. Terrorism can be treated like a crime: be vigilant in police work and improving social conditions to minimize it recruits.

@frog
"For example, public fear actually encourage[s] terrorism, as that is its only intent."

I heard an interesting speculation on this subject which appears to be borne out in current events. It goes: terrorism gains its most effective leverage by encouraging the target state to become ever more repressive and authoritarian. Taken far enough, the state collapses through internal discord.

The stats are that airplane trips are on the order of 100 times safer per mile, indicating that they're often safer per trip than a car ride. They're certainly safer per hour, and for almost any person they're less likely to die in a plane crash than in a car crash during the course of their lives.

@combination

That would show car trips are say 6 times more dangerous than plane trips, and of course the problem with taking more of them is, probabilities multiplying. You're continually taking the same small chance of a random accident all the time, so in the end your chances of getting into one get very high indeed. Still, that 6 times factor is very different indeed from the per mile numbers usually quoted.

"People underestimate risks they willingly take and overestimate risks in situations they can’t control. When people voluntarily take a risk, they tend to underestimate it."

A valid point, but you may be able to consider this as more of a feature of the brain rather than a bug.

Let's apply these concepts to IT and use actual risk and try to allocate cost in minimizing a specific risk directly to its actual risk. We will measure actual risk based on the number of events that have occurred over a given period of time, and perceived risk as the number of times an incident has been reported in the media over time. (I know this may be oversimplified, but I think it fits in the context of the article).

Now lets look at 3 different use cases that have risk in IT. First is the case that an internal employee with good intentions is going to fat finger some data entry and accidentally wipe out some critical system. Second, a script kiddie with bad intent runs a script that happens to attack your infrastructure that exploits a well known flaw. Third, a well-orchestrated, specific attack against your systems with the intent of obtaining sensitive data. These all have very different levels of actual and perceived risk, and some are a lot easier to control than others, but I think that in some cases something with higher perceived risk and lower actual risk might be a better candidate for allocating more cost. But it really depends on a lot more than just actual and perceived risk to really make that decision.

All in all a good article, but sometimes when we look deeper at why our brain compensates for things by exaggerating different stimuli, it may be that our brains are more sophisticated than we give credit.

> we worry more about anthrax (with an annual death toll of roughly zero) than influenza (with
> an annual death toll of a quarter-million to a half-million people). Influenza is a natural accident

Anthrax is not a real concern, but I think you over-rate Influenza. It is mostly only a significant danger to people who are in quite poor health already and would probably die of something else if influenza didn't get them first. Saying that it kills up to a half million people annually is assigning it more credit (err, blame) than it really deserves. The common cold can kill you, but only if you're already in trouble. This sounds callous, but I'm not saying those lives aren't important: I'm only saying that influenza is not really the cause of death. Yes, it's the immediate, apparent cause, but it's not the real cause. Death by influenza is a symptom of a larger underlying medical problem, and it's the underlying problem, not influenza, that is worth worrying about.

Perhaps this raises another point: we over-estimate risks we can easily quantify and under-estimate risks that are more complex and require deeper understanding of the issues.

> We under-react to changes that occur slowly and over time.

Indeed. Conservatives have been saying this for decades, but liberals don't want to hear it.

Gun control is a political football that serves no real crime reduction purpose. Are you really interested in reducing violent crime rates or do you just believe it works because it's "common sense" ? See The Bias Against Guns, More Guns Less Crime, both by John R Lott.

http://www.johnlott.org/

Risk assessment is a well defined procedure. For an activity you consider what might happen. For each such outcome you consider the seriousness of the consequences, and the probability of its occurrance. This gives a position on a 2D grid. Obviously, the outcomes which you are most concerned about are the ones which are most probable and have serious consequences. The hardest to assess are those which are improbable but have serious consequences.

One way of expressing the article is to say that people are bad at both assessing the probabilities of outcomes, and the consequences (particularly if the consequences are significantly in the future).

So, there is a difference between perceived risk and actual risk.

@quincunx
I understand your point, you are saying "actual risk" should be "actual (or calculated) probability". Using the same word "risk" for both "perceived risk" and as a synonym for probability conflates two distinct ideas in ways that could be (and given the comments, obviously is) confusing.

For those interested, there are some very good books on the subject -

Phantom Risk: Scientific Inference and the Law, edited by Kenneth Foster, David Bernstein, and Peter Huber;

Michael Fumento's Science Under Siege;

Rethinking Risk & the Precautionary Principle, edited by Julian Morris.

Automobile accidents happen quickly. If you're on an airplane you might be aware of what is happening for many minutes before you hit the ground. The fear of dying is worse than death itself...

There's another factor to consider alongside risk assessment: threat assessment. The new-fangled predictive functions of our brains not only look at possible risks, but at how they themselves would take advantage of holes in security.

People recognize that it's in the nature of any animal capable of the most rudimentary learning to capitalize on any advantage. You can't capitalize on a distributed risk like car crashes. But you can take advantage of airline security weaknesses again and again. The Anthrax threat was similar. Low risk, but what would happen if it remained easy to send deadly disease through the mail?

So the predictive human wants very badly to close those weaknesses. If we don't remedy those chinks in the armor, bad people will use them until airline travel is truly risky, and nobody will deliver the mail.

That brings us to the second fallacy in risk assessment: the predictive brain assesses values alongside the risks. We apparently value liberty a great deal. We recognize that if our travel or commerce is limited, we risk enormous consequences. So, we're willing to spend a great deal to protect our liberty, even if there's little actual reduction in risk.

No, I don't think we always spend wisely, but I also don't think it's all wasted.

"because pilots are very thoroughly and professionally trained and aircraft are maintained to meticulous levels.
...
countless other drivers out there on the roadways, virtually all of whom are basically amateurs"

While I agree that flying is relatively safe, these kinds of statements make no sense to me.

Sure pilots are "professionally trained" but they are people too. That means they too will underestimate any risk that they themselves control. They could very well fly when they are tired, hung-over, even intoxicated simply because they feel the risk is "low enough". (They have done it before and it worked) Do you think it doesn't happen? You can be sure it does happen, only that they (and you) were "lucky", if we can call it that.

The fact that they are "professionally trained" or that there is a plane full of people behind them will unfortunately not offset their risk perception by much. Not because they're bad people or anything it's just that they are like the rest of us!

For some reason we perceive the risk of getting a sloppy or incompetent pilot (or surgeon) as non-existent. It is low, and they are "professionals" but really, there is no major difference compared to the "amateur driver".

Being sloppy or incompetent works in these cases for the same reason that many of the "amateur" drivers you mention are actually able to drive their cars their whole life without a serious accident. Probability makes it so that most of the time things are fine.

Furthermore, it bugs me when people claim off the bat that airplanes are maintained to "meticulous levels".
They are of course maintained only to the level that keeps the risk at at an acceptable level - no more. Because it costs money, time and takes time away from the mechanic's lunch break; a mechanic who also evaluates the risk of his work not being good enough according to his own internal set of rules.

Plane crashes, like car accidents, are going to continue at the level of probability that people accept.

And that means you can fly safely, only not for the reasons you claim.

That said, the discussion is wonderful. It is very interesting to study the way "other people" perceive risk to see how it influences your own risk, whether you're driving, flying or something else.

"Simply put, when a plane falls out of midair, there are virtually always no survivors. When a car crashes, most of the time the passengers are relatively uninjured...
...Planes can have relatively small incidents (single engine failure, instrument failure, etc.)"

Even here there's an element of perceived risk vs. actual - only 10% of aircraft crashes leave no survivors at all. A surprising number of even high speed crashes leave survivors (granted not always that many).

Bruce, as European country that have had another 11th ( March 11th) I can say that in our country nobody worry about this happen again, nothing change in our lives. I was in one of the trains, and I use to take the train every day and I do not feel worried about. Just was an accident for us.

My wife take the train every day, she never feel worried, she never feel on risk. She feel more risk diving by card from home to the station to get the train.

But never mind, that we use to be more pragmatic, than other people.

On the anniversary of March 11, some newspapers ask the people about, many people just forgot it. Just was something unusual terrific, that we have to forget to do not be repeated again.

I use to fly to USA, I never mind, I'm only worried about how many time I'll spend on the security line, only about if the TSA agent should have enough intelligent to understand why I'm traveling to USA.

But most of the problem that you have come from your decisions, you decide with the vote and you elect your government. They use this kind of acts as a propagan to support itself on the power possition for more time, that all.

Interesting article considering the recent DHS announcement of the 'permission to travel' requirement for American citizens beginning 14Jan06. Just more 'security theater' or sinister/stupid? Of course, any 'terrorist' knows our borders and ports are still basically wide open.

I just read that this "DHS announcement of the 'permission to travel' requirement" may just be pre-election FUD. Damn web!

cbunix23-

The same John Lott who had a sock puppet named "Mary" touting his brilliance?

The media & government feed people the risks, and what isn't popular, or common isn't reported hence isn't on the lips of everyone trying to survive.

Once things become commonplace and are ignored then we take it for granted. Like the frog boiling to death cause it was placed in lukewarm water.

What has not been said is that these reactions, feelings and fears are predominantly had by Americans - not the rest of the world.

Here in good ol' New Zealand, we don't worry about Earthquakes, even though we live atop the ring of fire. We don't stress about terrorism or street crime...

You see, we don't have "The Fear". The Fear has been foisted on Americans since the sceond world war: when Americans became so convinced that Russians were going to nuke them, they built bomb shelters. This despite the fact that Russia had no method of dilivering a Nuke to the continental USA at the time.

Since then, American life has been about being scared: scared of being mugged, scared of being robbed, scared of being arrested, scared of being killed, fear of terrorism. Fear of just about anything.

This is as planned. The US media and government work hand in hand to ensure the population is afraid. The media can't get ratings without projecting fear, and the government back it up with stupid laws designed to ensure that all Americans are guilty of felonies.

Only when all citizens are a) afraid and, b) criminals can the population be properly controlled.

It doesn't hurt if you spin complete bullshit to them for a few decades either.

There's an old story from and ancient chinese book by Chuangzi. Not that it's really useful or anything, just interesting and somewhat related.

The Empty Boat
Suppose a boat is crossing a river and another boat, an empty one, is about to collide with it. Even an irritable man would not lose his temper. But suppose there was someone in the second boat. Then the occupant of the first would shout to him to keep clear. And if he did not hear the first time, nor even when called to three times, bad language would inevitably follow. In the first case there was no anger, in the second there was—because in the first case the boat was empty, in the second it was occupied. And so it is with man. If he could only pass empty through life, who would be able to injure
him?

Your observations about human psychology are astute.

Fortunately, we have learned to observe and reason, so that we can overcome these unfortunate blinders.

Unfortunately, as we learn from history, we don't learn from history. We fail to make the propagation of observation and reason a primary mission of education. We fail to decidedly eliminate those belief systems which threaten the primacy of observation and reason.

So far we've managed to muddle through. But now, like the Krell in forbidden planet, we're creating potential disasters. They creep slowly enough that we're failing to see them. Worrying about asteroids colliding is more exciting at the water cooler than melting icecaps.

Meanwhile, those who yearn for Apocalypse are doing what they can to encourage one.

Your observations about human psychology are astute.

Fortunately, we have learned to observe and reason, so that we can overcome these unfortunate blinders.

Unfortunately, as we learn from history, we don't learn from history. We fail to make the propagation of observation and reason a primary mission of education. We fail to decidedly eliminate those belief systems which threaten the primacy of observation and reason.

So far we've managed to muddle through. But now, like the Krell in forbidden planet, we're creating potential disasters. They creep slowly enough that we're failing to see them. Worrying about asteroids colliding is more exciting at the water cooler than melting icecaps.

Meanwhile, those who yearn for Apocalypse are doing what they can to encourage one.

@Frank Ch. Eigler

With regards to the effect of a higher density of guns on overall safety I'd argue that the trend is negative as opposed to positive.

First consider the deterrence effect. This is covered by Bruce's forth point. The criminal is taking a voluntary risk and feels in control of the situation (particularly if they have their own gun). As a result they would significantly underestimate the risk from any possible weapon(s) the victim may have and not be as deterred.

Also while I don't think legality of guns significantly affects the number of criminals with guns I suspect it may affect their willingness to use their guns during a crime as they try to control the unknown risk of a victim who may have a weapon.

Finall the cost of those additional guns are not just deliberate shootings (between victims and criminals) but accidental shootings as well which are a risk which is very likely underestimated.

@billswift

'I understand your point, you are saying "actual risk" should be "actual (or calculated) probability". Using the same word "risk" for both "perceived risk" and as a synonym for probability conflates two distinct ideas in ways that could be (and given the comments, obviously is) confusing.'

Sorta. I would remove the word Actual altogether and just say:

Subjectively Perceived Risk vs. Historical Probability Spread

But then the task becomes in identifying the applicable situation.

For example, trying to measure entrepreneurial risk using historical tables of rates of business success is totally idiotic, especially in the case of a new good idea.

@ Bruce Schneier

Do you know what the stats are for bus transport versus air transport?

Was flying ALWAYS safer than driving a car?

If you confine the area of interest narrow enough, especially from the point of view for an individual actor I think it would become apparent that Actual (your word) risk changes.

In the case of air transport, you have a professional pilot moving many passengers through the air, in the case of a bus driver you have a professional driver moving many passangers on the roads.

If the crime rate in your town is low, but many of your relatives have been mugged, would you condemn someone for feeling an 'incorrect' perceived risk?

2 We do?

"If anything, we worry about the following problems a terrorist attack would cause - look at the immediate recession caused by the last one."

Score one for the current administration.

The economy slowdown happened BEFORE the attacks.

See, for example http://en.wikipedia.org/wiki/2001_recession

Tell me about it. I was laid off from a dot com in August of 2001. While everyone laid off two weeks after me got numerous extensions of unemployment benefits, I got crap. All they did was happen to get laid off after 9/11.

Is flying less dangerous than driving in a sober and responsible manner? What percentage of auto accident fatalities are people who were either driving drunk or passengers of a drunk? (looking it up...)

According to the NHTSA, in 2004 39% of motor fatalities were alcohol-related, and 87% of those were the drunk drivers or their passengers. So 1/3 of the risk of motor fatality can be avoided by not driving drunk or riding with drunks.

One may guess that other large fractions can be avoided by not speeding, having a car in good working order, and not driving in hazardous conditions.

The unavoidable risk is still probably greater than for flying as a commercial passenger, but by a much smaller ratio than was cited for total risk.

>> We under-react to changes that occur slowly and over time.

>Indeed. Conservatives have been saying this for decades, but liberals don't want to hear it.

Oh. You mean like global warming or creeping loss of civil liberties?
:)

I remember an article a few years ago in New Scientist about evaluating the risk of various forms of travel. Traditionally airlines would compare the risk of travel in terms of accidents per mile and airplanes come out very much safer in this comparison. If you consider that the real dangers in airtravel are taking off and especially landing you can compare accidents per journey instead. In this comparison airlplanes are slightly safer than cars but of course the accidents are way more damaging in aircraft.

I can't find any references, sorry.

The comparison between amateur car drivers and professional pilots is much greater than you might think, especially in ways which are relevant to this article. I have a private pilot's license, but that got me into reading the air accident reports (in the UK). The errors made by commercial transport pilots are very different to those made by private pilots. Whereas private pilots regularly fly into hillsides in bad weather, run off the end of a runway which is too short for their aircraft and fail to deal safely with in-flight technical failures, professional pilots almost never do these things. Commercial air transport accidents are almost without exception failures of the system in which the pilots operate, resulting in accidents where no action by the pilot could help, or where the pilot is overwhelmed by complexitty outside of his training. The main reason for the difference is precisly the point of this article. Professional pilots are constantly presented with genuine information on real accidents investigated scientifically, rather than simple (an usually inaccurate) personal perceptions. From day one of training and continuing throught their carreers, on a daily basis, real risk information is hammered home. This is much less the case with private pilots (who don't have safety managers) and not at all with car drivers.

THE KILLER SPINACH STRIKES AGAIN!
Well, if I needed another proof that people, and especially take-everything-for-granted-the-TV-tells-me Americans, have a notoriously poor conception of risk, I got it now. I'm Swiss, but currently in the U.S. for an extended assignment. Can you believe I can't buy a mixed salad in the supermarket that doesn't have a "does not contain spinach" sticker on it ? What about the Romaine, Lollo, Beet Greens which no doubt grew on a lush California field not far from where sewage water was "accidentally" used to hasten growth ? What about the odds of getting killed in an accident, without any involvement of lethal spinach, while driving to the supermarket? I'm just shaking my head in disbelief. But I have become an avid collector of "does not contain spinach" stickers, and have every intention to slap them all over our servers back at work, to keep reminding people that the thing that gets you is most likely not the thing that's top in the news.

What? You are still eating arugula? Do you have a death wish?

As Al pointed out above, the truly informative comparative statistic about travel is deaths per trip, not deaths per mile.

By that standard, the only thing more dangerous than commercial air travel is motorcycling.

So how does this apply to stories about bad voting machines?

Fascinating article. My comment is that fear is by definition an irrational response to a situation (a rational one might fall into the category of "risk management".) Therefore, what we fear is, (almost by definition), either not dangerous at all or so out of calibration with reality as to be non-useful. Our fears tell more about ourselves than about the environment around us. Education is the primary solution to this problem. People need to learn critical thinking skills and to move toward the "reality-based community".

@Reasonable:

It depends on what they are after. A terrorist wants to inspire fear (hence the inclusion of the word "terror"), and increased perception of risk serves their goals, and polarizes the population, drowning out moderate voices, making people demand that the government protect them from scenarios that are basically impossible to prevent, leading to security theater. If nobody cared about death related to terrorism, then terrorists would stop doing it. The response of a government to fear induced by terrorism is to take stronger security measures, tending towards fascism. Trying to fight extermism with extremism (you're either with us or against us) is not a great strategy. A truly rational response is to note that terrorism is unstoppable on a tactical level without creating a police state, but it is stoppable on a strategic level by removing root causes (feeling powerless and lacking a

@Ole J:
Based on my reading of the transcripts of the phone calls from the flight that crashed on 9/11 (which were primarily sky phone calls, not cellular), the callers were aware of the probability of their death, and were very calm, and many told their loved ones not to worry because their end would be swift.

The solution to all of this is to do a proper risk analysis, and to suppress emotion. Unfortunately most people, especially the reactionary extremists (think greenpeace and the religious right), prefer feeling to thinking. They're certainly better at it.

Regarding statistics versus tragedies, I will remind you that during Clinton's administration, if one soldier died in a war there was practically a congressional investigation. *Thousands* of people have died in Iraq and there's no whimper because it's unpatriotic to question it.

I'm surpised he didn't point out that evolution has not prepared us for situations like low-probability high-cost events, and for conditional probability estimates (as evidenced by the Monty Haul paradox).

If anyone can remind me of a nuclear meltdown on a nuclear sub or a nuclear aircraft carrier, I'd like to know they have happened. I suspect they haven't because they aren't having to compete with fossil fuels on cost. The real cost of oil is roughly twice the retail cost because of foreign loans and wars (like the current one) that are passed on to the taxpayer. These are effectively subsidies, and as long as they happen then we will not reduce our dependence on foreign oil because it's not economic. It unfairly benefits those that use the most oil at the expense of those that use it the least. By the way, an army may have traveled on its belly during the 1800s, but a mechanized army doesn't run on bread, it runs on oil. The eastern front in Europe wasn't about Moscow, it was about fossil fuels in Caucasus, and the Japanese strike at Pearl Harbor was in retaliation for an oil embargo. Iraq has used less of its known reserves than maybe one or two other countries. Now, why do you believe we're bogged down in a bloody quagmire in Iraq, with no intention of leaving? Oh yeah, it's WMDs, I forgot.

A co-worker who formerly maintained aircraft for the military says that commercial airline maintenance is shoddy. If a military aircraft taxies around but doesn't take off, it still has to undergo a full inspection. By contrast, commercial aircraft will get inspected, say, once a day, no matter how many takeoffs they have made.

The common thread between nuclear subs and military aircraft is that there's no alternative way to do it, and since there's no cheaper alternative (that has negative externalities on the public), they spend the money on doing it safely.

Politicians don't deal with actual risks, they deal entirely with perceptions. If people think you're being tough on crime, you are. And moral outrage is the most powerful force in politics.

Oh yeah, a child is 100 times as likely to be killed in a pool than by a handgun.

Source: Freakanomics

@Bruce: Is it the case that you introduced the terms "over-react" and "under-react" and that Dan Gilbert only talked about relative salience? He certainly didn't use such terms in the op-ed you linked to, but I haven't read his book yet.

And rather than look at these biases amongst humans as failings, it might be good to consider them from an adaptive standpoint. For example, isn't there a reason why we react stronger to threats backed by other people rather than those that are more random or based on natural phenomena?

Of course these biases evolved in a very different time. Collectively we have much greater impact on the world enviroment than we did long ago. And the amount of damage a few people can do to others is greatly amplified by modern technology.

Right on, dambrosi!

"Is it the case that you introduced the terms 'over-react' and 'under-react' and that Dan Gilbert only talked about relative salience? He certainly didn't use such terms in the op-ed you linked to, but I haven't read his book yet."

I introduced those terms; Gilbert does not use them in the op-ed. And the book is on a completely different topic.

"And rather than look at these biases amongst humans as failings, it might be good to consider them from an adaptive standpoint."

Of course. That is obviously the right way to look at them. I don't think either he or I have said otherwise.

You said "Commercial airplanes are ... much safer per passenger mile." I have heard this quoted for decades. The flaw is that the significant statistic is not "per passenger mile", but "per vehicle minute". Ride in a car, ride in an airplane, what you care about is the chance of your dying in the next sixty seconds. How many miles you travel in those sixty seconds is irrelevant. How many other people die with you is irrelevant.

Something that I'm looking into learning more about is "Black Swan Theory" by Nassim Nicholas Taleb.

He mentions Dan Gilbert as well as several others. It really complements a lot of what is being discussed in this post. Very very interesting stuff...

@Bruce
You need to get a column in a large circulation newspaper(WP, NYT, LAT, etc). Since security is so prevalent in everyone's mind right now, I'm sure many would read it.

@Andy Canfield
"The flaw is that the significant statistic is not "per passenger mile", but "per vehicle minute". Ride in a car, ride in an airplane, what you care about is the chance of your dying in the next sixty seconds"

No that's not the correct metric, because we don't travel by minutes, we travel by destination which involves miles (per minute would be the appropriate measure for amusement park rides). The basic measure is the goal of the trip, not the number of minutes spent getting there.

Consider StarTrek style teleportation which could get us anywhere in .1 second with exactly the same per trip risk. By your measure the risk per minute would be stupendous, but the actual practical risk involved in getting somewhere would still be the same.

I thoroughly enjoy reading about the vagaries of the human mind when it comes to evaluating risk, but I find myself regularly frustrated by the assumption that the rational evaluation of risk for a particular individual should only include those items that affect that particular individual directly. This is not an accurate assessment of being human. Human beings are social creatures who often make decisions based on what they believe is best for the group, not always on just what they think is best for themselves.

Thus a realistic evaluation of a terrorist threat needs to not only include the actual risk for an individual and the risk of certain reactions in escalating the attacks, but also the risk involved for people that the individual identifies with. This was touched on in the article's discussion about personified risks, but the implication was that this behavior is irrational. It is not. A soldier will take extreme personal risks to reduce the risk for people and social structures that the soldier cares about, but this should not necessarily be considered irrational.

Of course, people can make bad evaluations of the risk certain activities have for their social group, and much political manipulation is done with the purpose of fooling people into thinking that their group is at risk. But any attempt to objectively quantify risk for a particular individual needs to weigh the value that individual places on risks for others and for various social structures.

Interesting reading. I just had a discussion with a friend about malware and other risks associated with Internet technologies. Mr. Anonymous states that education is the primary solution to our fears. My thought is that there is no incentive to educate ourselves when we do not understand ourselves to be responsible for the problem. As long as we can blame someone else, there is no reason to proceed with any caution.

I recognize that the views of risk that you are describing are perhaps the most popular ones in the marketplace today, but I would invite you to consider the work that Kahneman and Tversky did in developing Prospect Theory. One of the essential conclusions of their work is that the reaction to risk is directly dependent on how the risk is presented. The same set of facts can result in conservative behavior in one group and trememdous risk taking in another. They believed that people are not risk averse, they are loss averse - they will take great chances to avoid loss.

I'm not sure all these anecdotes of how people respond to risk are to the point. The context of how the risk is presented is the key. People worry about anthrax because they have been terrorized by all the news stories. They don’t worry about influenza because the news doesn’t provide the same context – we see children getting flu shots in the doctor’s office. Context is the key in understanding and managing risk.

Remember what MacArthur said:

“There is no security in this world, only opportunity.�

Mr Schneier-

I notice in both this op-ed and your writings an occasional reference to the fear of Americans. We've been discussing this issue in a class I'm taking and we're wondering if you've studied the global effects of this phenomenon? As our instructor put it, "as the general level of global education rises and more cultures have access to "western" style news media...the unintended consequences should be interesting to observe".

> What do you mean by Actual Risk?
> I contend that no such thing can exist. There is only perceived risk, and actual fact of occurrence. This may sound pedantic, but the word Actual & Risk sound oxymoronic. Risk is of speculative nature, Actual is a fact of reality.

Perhaps Subjective and Objective risk would be better terms?

However I disagree that Actual & Risk are exclusive since some risks (probablilities) can be explicitly calculated. i.e. There is a 50% chance (when selecting one ball in a blind selection) of pulling a black ball from a bag containing one of each (otherwise identical) black and white balls. Notice here the careful wording to ensure there is no room for empirical observation to allow any other option. Compare the probably of a coin flip coming up heads - probably not 50% exactly since there is a very small chance the coin could land on its side. And who knows? Maybe from henceforth all coins will land on their sides. Thus, this example I agree could be considered speculative and relies on empirical observation to calculate the "objective" probabilities.

For another good example where Perceived and Actual risk seem quite apparent consider the Monty Hall game:

http://math.ucsd.edu/~crypto/Monty/monty.html

quincunx: I am somewhat confused on your stance concerning the issue of Perceived Risk vs. Actual Risk because it is written in adolescent ayn-randian catch-phrases. "Collectivist"? Care to tell me what the Actual Risk of sounding like an arsehole might be?

The point that we cope with risks that we are choosing better than ones we are not is very interesting.

I wonder if the desire to feel like we can control the risks and problems of everyday life is the origin of religion?

If we think that we can get God to ameliorate the risks of life if we pray enough, then that is like we can control those risks by proxy.

"Europeans routinely perceive the U.S. as being full of guns."

The number of guns in private hands in the US is about 200,000,000. Given that, a perception that the US is full of guns would not be that far from the mark.

"Europeans routinely perceive the U.S. as being full of guns."

The number of guns in private hands in the US is about 200,000,000. Given that, a perception that the US is full of guns would not be that far from the mark.

"Europeans routinely perceive the U.S. as being full of guns."

The number of guns in private hands in the US is about 200,000,000. Given that, a perception that the US is full of guns would not be that far from the mark.