Schneier on Security
A blog covering security and security technology.
« Third Annual Movie-Plot Threat Contest |
| NSA's Linux »
April 8, 2008
The Feeling and Reality of Security
Security is both a feeling and a reality, and they're different. You can feel secure even though you're not, and you can be secure even though you don't feel it. There are two different concepts mapped onto the same word -- the English language isn't working very well for us here -- and it can be hard to know which one we're talking about when we use the word.
There is considerable value in separating out the two concepts: in explaining how the two are different, and understanding when we're referring to one and when the other. There is value as well in recognizing when the two converge, understanding why they diverge, and knowing how they can be made to converge again.
Some fundamentals first. Viewed from the perspective of economics, security is a trade-off. There's no such thing as absolute security, and any security you get has some cost: in money, in convenience, in capabilities, in insecurities somewhere else, whatever. Every time someone makes a decision about security -- computer security, community security, national security -- he makes a trade-off.
People make these trade-offs as individuals. We all get to decide, individually, if the expense and inconvenience of having a home burglar alarm is worth the security. We all get to decide if wearing a bulletproof vest is worth the cost and tacky appearance. We all get to decide if we're getting our money's worth from the billions of dollars we're spending combating terrorism, and if invading Iraq was the best use of our counterterrorism resources. We might not have the power to implement our opinion, but we get to decide if we think it's worth it.
Now we may or may not have the expertise to make those trade-offs intelligently, but we make them anyway. All of us. People have a natural intuition about security trade-offs, and we make them, large and small, dozens of times throughout the day. We can't help it: It's part of being alive.
Imagine a rabbit, sitting in a field eating grass. And he sees a fox. He's going to make a security trade-off: Should he stay or should he flee? Over time, the rabbits that are good at making that trade-off will tend to reproduce, while the rabbits that are bad at it will tend to get eaten or starve.
So, as a successful species on the planet, you'd expect that human beings would be really good at making security trade-offs. Yet, at the same time, we can be hopelessly bad at it. We spend more money on terrorism than the data warrants. We fear flying and choose to drive instead. Why?
The short answer is that people make most trade-offs based on the feeling of security and not the reality.
I've written a lot about how people get security trade-offs wrong, and the cognitive biases that cause us to make mistakes. Humans have developed these biases because they make evolutionary sense. And most of the time, they work.
Most of the time -- and this is important -- our feeling of security matches the reality of security. Certainly, this is true of prehistory. Modern times are harder. Blame technology, blame the media, blame whatever. Our brains are much better optimized for the security trade-offs endemic to living in small family groups in the East African highlands in 100,000 B.C. than to those endemic to living in 2008 New York.
If we make security trade-offs based on the feeling of security rather than the reality, we choose security that makes us feel more secure over security that actually makes us more secure. And that's what governments, companies, family members and everyone else provide. Of course, there are two ways to make people feel more secure. The first is to make people actually more secure and hope they notice. The second is to make people feel more secure without making them actually more secure, and hope they don't notice.
The key here is whether we notice. The feeling and reality of security tend to converge when we take notice, and diverge when we don't. People notice when 1) there are enough positive and negative examples to draw a conclusion, and 2) there isn't too much emotion clouding the issue.
Both elements are important. If someone tries to convince us to spend money on a new type of home burglar alarm, we as society will know pretty quickly if he's got a clever security device or if he's a charlatan; we can monitor crime rates. But if that same person advocates a new national antiterrorism system, and there weren't any terrorist attacks before it was implemented, and there weren't any after it was implemented, how do we know if his system was effective?
People are more likely to realistically assess these incidents if they don't contradict preconceived notions about how the world works. For example: It's obvious that a wall keeps people out, so arguing against building a wall across America's southern border to keep illegal immigrants out is harder to do.
The other thing that matters is agenda. There are lots of people, politicians, companies and so on who deliberately try to manipulate your feeling of security for their own gain. They try to cause fear. They invent threats. They take minor threats and make them major. And when they talk about rare risks with only a few incidents to base an assessment on -- terrorism is the big example here -- they are more likely to succeed.
Unfortunately, there's no obvious antidote. Information is important. We can't understand security unless we understand it. But that's not enough: Few of us really understand cancer, yet we regularly make security decisions based on its risk. What we do is accept that there are experts who understand the risks of cancer, and trust them to make the security trade-offs for us.
There are some complex feedback loops going on here, between emotion and reason, between reality and our knowledge of it, between feeling and familiarity, and between the understanding of how we reason and feel about security and our analyses and feelings. We're never going to stop making security trade-offs based on the feeling of security, and we're never going to completely prevent those with specific agendas from trying to take care of us. But the more we know, the better trade-offs we'll make.
This article originally appeared on Wired.com.
Posted on April 8, 2008 at 5:50 AM
• 38 Comments
To receive these entries once a month by e-mail, sign up for the Crypto-Gram Newsletter.
One other issue between these two is the opportunities for arbitrage between the reality and the feeling of security. Many are exploiting this for commercial, political and ideological gain.
If we can - on a personal level - narrow the gap between reality and feelings of security then we are less open to this "abuse".
> We fear flying and choose to drive instead. Why?
Quite simple. Because the actions - and overreactions - taken by the TSA have made flying quite obnoxious. Look at the travel mode decision as an example of obnoxiousness minimization. You balance the annoyance of a long drive against the annoyance of getting to a plane. (and getting crammed into your seat.)
The annoyance of air travel has become great enough to elevate it not to the level of basic safety, but at least enough to become a factor in travel choice.
Come to think of it, this is somewhat reminiscent of the security vs usability tradeoffs we all talk about with computers.
> opportunities for arbitrage between the reality and the feeling of security.
Succinctly stated, and it seems to be what is missing from the security theater we daily witness.
> We fear flying and choose to drive instead. Why?
The illusion of control.
Dale, people took the irrational decision to drive long before the TSA became annoying.
The issue is with the control over the situation. In a psychology paper that I'm unable to find right now, it was concluded that people
* underestimate risks where they are in control
* overestimate risks where they are not.
Driving a car might be much more dangerous than flying, but in a car you don't have to give up control to the pilot and trust his skill.
Peter Galbavy, I quite like the idea of arbitrage between real and felt risk. After all, financial arbitrage performs a useful service to markets. Arbitrageurs exploit the differences in prices of the same commodities or securities in different markets, making money by buying at the lower price and selling at the higher price. The result of their activity is price convergence in the different markets.
If the idea could be applied somehow to the difference in "price" ascribed to real and perceived risk, the result would be far from abusive, it would produce precisely the sort of convergence of risk estimation that Bruce's essay appeals for. The abuses you seem to be thinking of are more in the nature of driving a wedge between the two "prices".
I have no good proposals for how one would implement such an arbitrage operation though.
Y'all (when will northern american english finally develop a much-needed word to mean "you, taken as a group" a la "Euch" in German?) seem to be assuming that security is the only input on the drive/fly decision. I would submit that convenience and cost are higher thought-level decision makers in fly/drive.
As Bruce has said before, if a child initiates contact with an adult for whatever reason (lost, hurt, sick, whatever) the likelihood that the person they choose is dangerous to them is very low, as opposed to when an adult initiates the contact. Pedophiles probably go to places where children are common if they are looking to abduct one. That means that as fear overcame logic, children riding subway alone (and many other things) became less common, pedophiles would be less likely to "cruise" there (stipulated "impulse" rather than "premeditated" attacks could still occur). Unfortunately this means if we as a society could get over our phobia of not allowing kids to do anything, it will get more dangerous again as pedophiles realize that subways (etc) are now a good place to find soloing children.
Flying already has a certain amount of latent fear. "If God had wanted man to fly he'd have given us wings." Add to that the mechanical underpinnings that can break, a general lack of knowledge about how planes stay in the air, stories about drunk pilots, and the lack of control aspect. Throw in a dash of crash footage and burned corpses from the handful of major accidents per decade and you have a rather large chunk of fear for the average passenger before you even deal with security.
When you add 9/11 to the mix and the fact that one group of passengers rebelled but were only successful in changing the location of their demise, security becomes the straw that broke the camel's back.
The TSA is just a pile of manure thrown on top of the security problem to cover the putrid stench of fear from the passengers. It doesn't actually deal with the security problem at all. However, it's enough to cover the fear for many, and they'll willingly trade away their constitutional rights and human dignity for the bliss of not having to smell their own fear.
We should be careful to get out of an experience only the wisdom that is
in it - and stay there, lest we be like the cat that sits down on a hot
stove-lid. She will never sit down on a hot stove-lid again - and that
is well; but also she will never sit down on a cold one any more.
-- Mark Twain
> And that's what governments, companies, family members and everyone else provide.
I'm not sure what this is saying. Is the claim that others provide the *feeling* of security? That they provide *only* the feeling of security? Surely they provide some real security, too.
There are some interactions b/w feeling secure and being secure that aren't mentioned in this post.
Imagine trying to turn around a neighborhood that has a "bad" reputation. The neighborhood will be safer if there are lots of people around than if it's desolate. But people will avoid the neighborhood because they don't feel safe there. It's a chicken and egg issue.
So helping people feel safer can be one step in making people truly safer.
> We can't understand security unless we understand it.
This sentence needs to replace the word "it" with its antecedent. As written, it's an empty statement.
> Security is both a feeling and a reality, and they're different. [...]
> There are two different concepts mapped onto the same word --
> the English language isn't working very well for us here [...]
Usually, you can see things when they are there, and you can't see things when they aren't there. We have special words for situations where this correspondence doesn't hold:
Transparent: it's there, but you can't see it (like the air)
Virtual: you can see it, but it isn't there (like a reflection)
Maybe we need words like that for security.
We've got "security theater", but I don't know how widely it is used, or understood. Also, the word "theater" is a bit charged; it suggests some kind of deceit or fraud. Sometimes it is, of course, but we need words that allow us to talk about things without making implicit value judgements.
Re: your rabbit example. This selects for rabbits that make the right security trade-off for their kittens, not for themselves. There's no mileage in taking no risks and dying old and single, compared to taking some foolhardy risks to impress the other bunnies.
Similarly humans should have evolved to take "unnecessary" risks while young, then to become risk-averse once they have children to protect.
"The other thing that matters is agenda. There are lots of people, politicians, companies and so on who deliberately try to manipulate your feeling of security for their own gain."
As in, the manufacturers of trigger locks? (E.g., the pilot who discharged his firearm in the plane, a week or so ago, trying to install an FAA-mandated trigger lock.)
Is information really the cure? How many times have the gun nuts ranted about how fundamentally stupid an idea is thinking that sticking something inside the trigger guard of a gun is going to make it safer?
I gave you MY reasons for the balance of fly/drive decisions changing. We make guesses about the general case of fly/drive decisions that "they" make, but we don't really know.
Of course in recent news, we hear about lapses of inspection permitted by the good-old-boy network, which changes both perceived and real risks of flying. (Though the former probably changes more than the latter.)
We can see the feeling vs. reality in our organizational structures. Those adverse to risk are probably more likely to remain lower in the organization, and those who take the most risks will probably be found higher in the chain. This could be why security is a hard sell to management.
"Virtual: you can see it, but it isn't there (like a reflection)"
Imaginary: you can see it, but it isn't there (like a hallucination).
Interesting how they have the same definition, yet are so different that we have to further qualify them.
We fear flying and choose to drive instead. Why?
Well, personally, as someone with more friends on the "Random" Increased Screening list than I could comfortably fit on a Greyhound bus, I choose to drive because of the Security Theatre at the airport. If I don't speed, and keep my car in reasonable condition, I'm unlikely to be harassed by the Highway Patrol. As various friends of mine have inadvertently demonstrated for me, there's a non-zero chance I could end up in jail for a few hours or days simply for trying to get on an airplane.
Not to mention the fact that I just consider the treatment one receives at airports these days to be simply insulting, and have minimal interest in financially rewarding anyone for behaving in such a fashion.
When people compare the risks of flying and driving, they almost always do so in hopelessly naive ways, and end up reinforcing the airline industry/FAA trope that flying is much safer than driving.
For example, are you twice as safe flying on a crowded plane as on a sparse one? Probably not. But the way the flight safety statistics are calculated this is often the apparent result. Flight safety statistics almost always ignore criminal risks or things which happen "coincidental" to the flight (like medical crises for which you can't get help) which are not ignored in other modes of transportation.
Check out http://philip.greenspun.com/flying/safety , which has some (naive) numbers to put this into perspective. For example, "big commercial airliners" have .34 fatal accidents per million flight hours, which, ball-park normalizing, might be about a tenth of the rate of driving.
But that's only one kind of flying versus all kinds of driving. Smaller turboprop commuter airplanes (which, because of my location, I must always use when flying) crash five times more often. And averaging an accident rate over all kinds of driving, conditions, drivers, and so forth, doesn't necessarily make sense, either.
A trip is composed of several modes of transportation. Very few people take off flying from the top of their roof and land at their destination. A flight replaces only part of a car trip--and, it turns out, it replaces the safest part.
People are aware that driving around town within short distances of their home are the parts of a trip most likely to incur an accident--this part of the trip, whether flying or driving, however, is unavoidable.
Once you are on the interstate, the accident rate is much, much lower. If driving, averaging all kinds of driving together, is only a few times more "dangerous" than flying, then are you actually safer flying for any given trip?
You don't have complete control over your safety in a car. But you do have some. Some drivers are safer and get into less accidents than others--so in your personal risk scenario you might be twice as safe as the average driver (note: most people *think* they're much safer, but often aren't). And the part you're flying instead of driving is the safest part--maybe twice as safe as all kinds of driving. If you add in other risk aversion (such as not driving in inclement conditions--you have no control over this judgment in commercial aircraft) I don't think it's unreasonable, as a ballpark estimate, to think that the risk of a long car trip is similar to that of a flight.
The idea that flying is "much" safer than driving, in every circumstance--so much so that it must be "irrational" to consider driving somewhere instead of flying--is a mere trope. It's "conventional wisdom" that gets repeated so often people take it as given. But it's not--comparing these risks requires nuanced consideration, and, while I don't disagree that people often make decisions irrationally, it's not inherently irrational to consider driving the safer--at least not less safe--course.
There is one correction that might be important under some circumstances: security, whether perceived or actual, isn't always a tradeoff against convenience, privacy, cost and so forth. Every now and then technologies come along where increased security (or at least safety) goes along with increased convenience/privacy/cost/etc.
@Phil: What a great story, and what a great mom.
My mother is a wonderful lady in many ways, and reasonably adventurous herself, but certainly had (and has) problems with perceived-versus-real security around her children. I'm 49, do weight lifting, and have studied martial arts for years, but she had serious heebie-jeebies when I recently went to Seoul, and she's very worried that on an upcoming business trip to Istanbul, some horror she can't name will befall me. But she, a rather frail 77-year-old, is bunking off to vacation in Egypt. Go figure.
Anyway, my godson is 7 years old, and I consider it one of my missions in life to teach him courage. That includes helping him learn to judge real risk and then, if suitable, "go for it".
I feel like I am reading The Republic, by Plato, a BLOOM, not cornford, dog food crap, AGAIN. Philosophers, Kings, and justice = security. Oh well.
Keep up the reporting B.S., although Cephalus, sure is taking a long time to leave these days. De evolution of the gene pool, power can do that. GRR.
Nice blog again.
TSA has consistently shown us we are the rabbits. They routinely make only one trade off: the appearance of security for security. And now, Chertoff wants to start a "Manhattan Project" to protect the internet. (http://www.news.com/8301-10784_3-9914391-7.html?part=rss&subj=news&tag=2547-1_3-0-20) Oh, my God!!!
@bob: when will northern american english finally develop a much-needed word to mean "you, taken as a group" a la "Euch" in German?
"You" does mean "you, taken as a group." Thou needst to brush up on thy pronouns.
I don't know if the problem with the word "security" is solved in other languages (in Spanish is the same, we use also the word "seguridad" in both cases).
After reading the post, I feel like when I was studing Economics... What is happening here is that we are in front of a market with imperfect information where we cannot aply the same rules that in markets with perfect information. So, it's OK, security is a trade-off, but we have to consider is that the traditional rules of Economics are not going to be useful, we need to use other theories like 'Game Theory'.
No sé si el problema con la palabra "security" está resuelto en otros idiomas (en español tenemos el mismo problema ya que usamos la palabra "seguridad" también en ambos casos).
Después de leer este post, me parece que estoy de nuevo estudiando Económicas... Lo que está pasando es que estamos frente a un mercado con información imperfecta donde no podemos aplicar las mismas reglas que en los mercados con información perfecta. Por tanto, perfecto, la seguridad es un intercambio, pero tenemos que tener en cuenta que las reglas tradicionales de Economía no van a ser útiles y tendremos que usar otras como la Teoría de Juegos.
> Security is both a feeling and a reality, and they're different. You can feel secure even though you're not, and you can be secure even though you don't feel it. There are two different concepts mapped onto the same word -- the English language isn't working very well for us here -- and it can be hard to know which one we're talking about when we use the word.
Enter Norwegian, in which both "security" and "safety" translates to the same word, "sikkerhet". It makes for headaches when trying to explain the differences and complications from "safety" to "security" for the lay-man.
I always enjoy Bruce's references to the risk decisions we all make everyday. It is a useful method to draw attention to how human's measure risk.
I was very surprised to actually see Bruce demonstrate this principle at the RSA Security Conference on the afternoon of April 7th. I was in a taxi headed toward my hotel when we came upon Howard Street. The traffic light in front of us turned yellow, my taxi driver gunned it and then, flying across the street (mind you not the intersection) comes Bruce Schneier. Mind you it wasn't that close of a call, Bruce had plenty of time to spare before we actually hit him. I couldn't help myself from chuckling a bit though despite the thought of what might have happened if Bruce's risk decision had been slightly more flawed.
Bruce - perhaps for the sake of your admirers, maybe you could use the crosswalks in the future :) Just in case!
> For example, are you twice as safe flying on a crowded plane as on a sparse one? Probably not. But the way the flight safety statistics are calculated this is often the apparent result.
The paradoxical apparent result is wrong because one can't apply the statistics to his immediate case. For instance, after that the ball of a roulette wheel has fallen on Black three times in a row, one might be tempted to bet Red "because of the Law of Large Numbers". But he would be wrong as Red's probability in this immediate case is still 50% (let's not consider 0 and 00). However, the LLN still affirms that, on a large number of draws, Black and Red results are each around 50%.
I think there is another viewpoint in this discussion. Most people aren't responsible for some part of their security: at the airport, at work, etc. Some "security professional" like a manager, sysadmin, or head guard is. There is a divide between "security" and "CYA", which a lot of these professionals are opting for. It doesn't make you feel safe in the physical sense but legally it's as good as being "physically safe". Most professionals opt for "CYA" because the probability of them getting killed in an attack is small ( except in Die Hard movies) unlike the people they are responsible for protecting.
Many sidewalk cafes in Europe are bordered with potted plants. People feel secure within these "walls," and set down their purses and backpacks. Bagsnatchers come along and, with a wire hanger, grab the goodies and dive into the nearby metro. One restaurant told me this happens "ten times a day."
I actually think that what is called "real security" should only be called "technical security" because "feeling of security" is not less real than the implementation of security mechanims. If you think as a security provider, you may want to forget the users' feeling, but you'd very quickly end up with social engineering problems (people's feelings can be exploited, they can be seen as "vulnerabilities"). This gives a much more balanced way of the situation, where both security providers and users are failing to understand each other, the former thinking he's doing things more "real" while the latter thinks his feelings are the most "real" thing. What's really important here is to understand that both security practioners and uses need to be educated about each others, there is a gap to fill so that the "security chain" is whole again. I believe that the deep root of this problem is "trust" and security cannot work fully without a "broken trust" where security providers do not feel like users and users ignore the technical aspects.
Enjoyed your presentation at RSA this week.
Unfortunately it provoked a BAD thought, which may not be pleasant for "us engineers".
What if "X bar" - the average person, is not as fascinated with Reality as we are? Maybe even motivated to dismiss Reality.
You used the smoking example, but I have an example I prefer: credit card debt. With smoking, there is no guarantee that you will get cancer, etc, only probabilities. With credit card debt, there is a 100% chance that fees and interest will be (often obscenely) charged. No guessing there. So, a person choosing credit card debt knows for sure that when they spend more than they have/make there will certainly be bad consequences. They do it anyway.
Problem is, Reality is "no fun". Feelings are way fun. Even Models are more fun than Reality.
My hypothesis is that security theatrics succeed for this reason. The average guy doesn't want to take time out of his busy day to understand Reality. As both smoking and credit card debt tend to prove, the average person is not interested in being rational.
While education on topics like security tends to help, I'm not sure there is a clear cut way to make the Reality of security a driving force to the average person. Also seems like the Model-makers have their own agendas and are not always trusted.
Again, really enjoyed the thought-provoking presentation.
It always amazes me that each of us is so different in our perception of reality. Every aspect of our lives is dictated by it.
The way we "feel" is governed by how secure we feel.
Wish I could have gotten to RSA. From what Ive heard, it was great.
Part of getting a handle on risk is to have numerical comparisons. Many people have a feeling that familiar activities are safe. E.g. Driving a car.
I recently had a problem with risk assessment for an outdoor program I was working with at a school. There is very little public stats on risks in everyday lives.
Downhill skiing was one area I was able to find some numbers. (14 year old males have about a death per 5 million ski-days)
E.g.: What is the risk of death/serious injury for a student on the bus to the football game, compared to playing in the game?
Confounding this is catastrophic versus incremental problem. E.g. If I let/make my kid walk to school, and play outside, he is at risk of being abducted by a paedophile. If I drive him everywhere, he becomes fearful, timid and fat.
People perceive risk depending on their information network. I don't think it's more dangerous now than it was 100 years ago. We just hear more about it.
E.g. Kid drowns in a swimming incident on a school picnic. Every school program person in Canada hears about it, and policies are adjusted.
Since everyone hears about every kid tragedy, we see being a kid as being very dangerous. We then swaddle them in protection to the point they can't be kids anymore.
100 years ago, kid drowns in swimming incident would make the local weekly, and be unheard of anywhere else.
Since it's rare, it's just one of those things.
This distinction still occurs in rural Alberta. I hire local kids to help me on the farm. Initially I was worried about liability and disclaimers, and giving people ways to check up on me/their kids.
It's a non-starter. Some of the kids come to work illegally on quads or motorbikes. Their parents don't care. I have one girl whose dropped off at 8 a.m. every day. Her mom made no inquiries about whether I was safe to work for, who else worked here.
We're far enough from the big city that most people don't pay attention to city news.
Schneier.com is a personal website. Opinions expressed are not necessarily those of Co3 Systems, Inc.