Entries Tagged "psychology of security"

Page 6 of 26

Intelligence Analysis and the Connect-the-Dots Metaphor

The FBI and the CIA are being criticized for not keeping better track of Tamerlan Tsarnaev in the months before the Boston Marathon bombings. How could they have ignored such a dangerous person? How do we reform the intelligence community to ensure this kind of failure doesn’t happen again?

It’s an old song by now, one we heard after the 9/11 attacks in 2001 and after the Underwear Bomber’s failed attack in 2009. The problem is that connecting the dots is a bad metaphor, and focusing on it makes us more likely to implement useless reforms.

Connecting the dots in a coloring book is easy and fun. They’re right there on the page, and they’re all numbered. All you have to do is move your pencil from one dot to the next, and when you’re done, you’ve drawn a sailboat. Or a tiger. It’s so simple that 5-year-olds can do it.

But in real life, the dots can only be numbered after the fact. With the benefit of hindsight, it’s easy to draw lines from a Russian request for information to a foreign visit to some other piece of information that might have been collected.

In hindsight, we know who the bad guys are. Before the fact, there are an enormous number of potential bad guys.

How many? We don’t know. But we know that the no-fly list had 21,000 people on it last year. The Terrorist Identities Datamart Environment, also known as the watch list, has 700,000 names on it.

We have no idea how many potential “dots” the FBI, CIA, NSA and other agencies collect, but it’s easily in the millions. It’s easy to work backwards through the data and see all the obvious warning signs. But before a terrorist attack, when there are millions of dots—some important but the vast majority unimportant—uncovering plots is a lot harder.

Rather than thinking of intelligence as a simple connect-the-dots picture, think of it as a million unnumbered pictures superimposed on top of each other. Or a random-dot stereogram. Is it a sailboat, a puppy, two guys with pressure-cooker bombs, or just an unintelligible mess of dots? You try to figure it out.

It’s not a matter of not enough data, either.

Piling more data onto the mix makes it harder, not easier. The best way to think of it is a needle-in-a-haystack problem; the last thing you want to do is increase the amount of hay you have to search through. The television show Person of Interest is fiction, not fact.

There’s a name for this sort of logical fallacy: hindsight bias. First explained by psychologists Daniel Kahneman and Amos Tversky, it’s surprisingly common. Since what actually happened is so obvious once it happens, we overestimate how obvious it was before it happened.

We actually misremember what we once thought, believing that we knew all along that what happened would happen. It’s a surprisingly strong tendency, one that has been observed in countless laboratory experiments and real-world examples of behavior. And it’s what all the post-Boston-Marathon bombing dot-connectors are doing.

Before we start blaming agencies for failing to stop the Boston bombers, and before we push “intelligence reforms” that will shred civil liberties without making us any safer, we need to stop seeing the past as a bunch of obvious dots that need connecting.

Kahneman, a Nobel prize winner, wisely noted: “Actions that seemed prudent in foresight can look irresponsibly negligent in hindsight.” Kahneman calls it “the illusion of understanding,” explaining that the past is only so understandable because we have cast it as simple inevitable stories and leave out the rest.

Nassim Taleb, an expert on risk engineering, calls this tendency the “narrative fallacy.” We humans are natural storytellers, and the world of stories is much more tidy, predictable and coherent than the real world.

Millions of people behave strangely enough to warrant the FBI’s notice, and almost all of them are harmless. It is simply not possible to find every plot beforehand, especially when the perpetrators act alone and on impulse.

We have to accept that there always will be a risk of terrorism, and that when the occasional plot succeeds, it’s not necessarily because our law enforcement systems have failed.

This essay previously appeared on CNN.

EDITED TO ADD (5/7): The hindsight bias was actually first discovered by Baruch Fischhoff: “Hindsight is not equal to foresight: The effect of outcome knowledge on judgment under uncertainty,” Journal of Experimental Psychology: Human Perception and Performance, 1(3), 1975, pp. 288-299.

Posted on May 7, 2013 at 6:10 AM • View Comments

Random Links on the Boston Terrorist Attack

Encouraging poll data says that maybe Americans are starting to have realistic fears about terrorism, or at least are refusing to be terrorized.

Good essay by Scott Atran on terrorism and our reaction.

Reddit apologizes. I think this is a big story. The Internet is going to help in everything, including trying to identify terrorists. This will happen whether or not the help is needed, wanted, or even helpful. I think this took the FBI by surprise. (Here’s a good commentary on this sort of thing.)

Facial recognition software didn’t help. I agree with this, though; it will only get better.

EDITED TO ADD (4/25): “Hapless, Disorganized, and Irrational“: John Mueller and Mark Stewart describe the Boston—and most other—terrorists.

Posted on April 25, 2013 at 6:42 AM • View Comments

Initial Thoughts on the Boston Bombings

I rewrote my “refuse to be terrorized” essay for the Atlantic. David Rothkopf (author of the great book Power, Inc.) wrote something similar, and so did John Cole.

It’s interesting to see how much more resonance this idea has today than it did a dozen years ago. If other people have written similar essays, please post links in the comments.

EDITED TO ADD (4/16): Two good essays.

EDITED TO ADD (4/16): I did a Q&A on the Washington Post blog. And—I can hardly believe it—President Obama said “the American people refuse to be terrorized” in a press briefing today.

EDITED TO ADD (4/16): I did a podcast interview and another press interview.

EDITED TO ADD (4/16): This, on the other hand, is pitiful.

EDITED TO ADD (4/17): Another audio interview with me.

EDITED TO ADD (4/19): I have done a lot of press this week. Here’s a link to a “To the Point” segment, and two Huffington Post Live segments. I was on The Steve Malzberg Show, which I didn’t realize was shouting conservative talk radio until it was too late.

EDITED TO ADD (4/20): That Atlantic essay had 40,000 Facebook likes and 6800 Tweets. The editor told me it had about 360,000 hits. That makes it the most popular piece I’ve ever written.

EDITED TO ADD (5/14): More links here.

Posted on April 16, 2013 at 9:19 AM • View Comments

Elite Panic

I hadn’t heard of this term before, but it’s an interesting one. The excerpt below is from an interview with Rebecca Solnit, author of A Paradise Built in Hell: The Extraordinary Communities That Arise in Disaster:

The term “elite panic” was coined by Caron Chess and Lee Clarke of Rutgers. From the beginning of the field in the 1950s to the present, the major sociologists of disaster—Charles Fritz, Enrico Quarantelli, Kathleen Tierney, and Lee Clarke—proceeding in the most cautious, methodical, and clearly attempting-to-be-politically-neutral way of social scientists, arrived via their research at this enormous confidence in human nature and deep critique of institutional authority. It’s quite remarkable.

Elites tend to believe in a venal, selfish, and essentially monstrous version of human nature, which I sometimes think is their own human nature. I mean, people don’t become incredibly wealthy and powerful by being angelic, necessarily. They believe that only their power keeps the rest of us in line and that when it somehow shrinks away, our seething violence will rise to the surface—that was very clear in Katrina. Timothy Garton Ash and Maureen Dowd and all these other people immediately jumped on the bandwagon and started writing commentaries based on the assumption that the rumors of mass violence during Katrina were true. A lot of people have never understood that the rumors were dispelled and that those things didn’t actually happen; it’s tragic.

But there’s also an elite fear—going back to the 19th century—that there will be urban insurrection. It’s a valid fear. I see these moments of crisis as moments of popular power and positive social change. The major example in my book is Mexico City, where the ’85 earthquake prompted public disaffection with the one-party system and, therefore, the rebirth of civil society.

Posted on April 8, 2013 at 1:30 PM • View Comments

Security Awareness Training

Should companies spend money on security awareness training for their employees? It’s a contentious topic, with respected experts on both sides of the debate. I personally believe that training users in security is generally a waste of time, and that the money can be spent better elsewhere. Moreover, I believe that our industry’s focus on training serves to obscure greater failings in security design.

In order to understand my argument, it’s useful to look at training’s successes and failures. One area where it doesn’t work very well is health. We are forever trying to train people to have healthier lifestyles: eat better, exercise more, whatever. And people are forever ignoring the lessons. One basic reason is psychological: we just aren’t very good at trading off immediate gratification for long-term benefit. A healthier you is an abstract eventually; sitting in front of the television all afternoon with a McDonald’s Super Monster Meal sounds really good right now. Similarly, computer security is an abstract benefit that gets in the way of enjoying the Internet. Good practices might protect me from a theoretical attack at some time in the future, but they’re a lot of bother right now and I have more fun things to think about. This is the same trick Facebook uses to get people to give away their privacy; no one reads through new privacy policies; it’s much easier to just click "OK" and start chatting with your friends. In short: security is never salient.

Another reason health training works poorly is that it’s hard to link behaviors with benefits. We can train anyone—even laboratory rats—with a simple reward mechanism: push the button, get a food pellet. But with health, the connection is more abstract. If you’re unhealthy, what caused it? It might have been something you did or didn’t do years ago, it might have been one of the dozen things you have been doing and not doing for months, or it might have been the genes you were born with. Computer security is a lot like this, too.

Training laypeople in pharmacology also isn’t very effective. We expect people to make all sorts of medical decisions at the drugstore, and they’re not very good at it. Turns out that it’s hard to teach expertise. We can’t expect every mother to have the knowledge of a doctor or pharmacist or RN, and we certainly can’t expect her to become an expert when most of the advice she’s exposed to comes from manufacturers’ advertising. In computer security, too, a lot of advice comes from companies with products and services to sell.

One area of health that is a training success is HIV prevention. HIV may be very complicated, but the rules for preventing it are pretty simple. And aside from certain sub-Saharan countries, we have taught people a new model of their health, and have dramatically changed their behavior. This is important: most lay medical expertise stems from folk models of health. Similarly, people have folk models of computer security. Maybe they’re right and maybe they’re wrong, but they’re how people organize their thinking. This points to a possible way that computer security training can succeed. We should stop trying to teach expertise, and pick a few simple metaphors of security and train people to make decisions using those metaphors.

On the other hand, we still have trouble teaching people to wash their hands—even though it’s easy, fairly effective, and simple to explain. Notice the difference, though. The risks of catching HIV are huge, and the cause of the security failure is obvious. The risks of not washing your hands are low, and it’s not easy to tie the resultant disease to a particular not-washing decision. Computer security is more like hand washing than HIV.

Another area where training works is driving. We trained, either through formal courses or one-on-one tutoring, and passed a government test, to be allowed to drive a car. One reason that works is because driving is a near-term, really cool, obtainable goal. Another reason is even though the technology of driving has changed dramatically over the past century, that complexity has been largely hidden behind a fairly static interface. You might have learned to drive thirty years ago, but that knowledge is still relevant today. On the other hand, password advice from ten years ago isn’t relevant today. Can I bank from my browser? Are PDFs safe? Are untrusted networks okay? Is JavaScript good or bad? Are my photos more secure in the cloud or on my own hard drive? The ‘interface’ we use to interact with computers and the Internet changes all the time, along with best practices for computer security. This makes training a lot harder.

Food safety is my final example. We have a bunch of simple rules—cooking temperatures for meat, expiration dates on refrigerated goods, the three-second rule for food being dropped on the floor—that are mostly right, but often ignored. If we can’t get people to follow these rules, what hope do we have for computer security training?

To those who think that training users in security is a good idea, I want to ask: “Have you ever met an actual user?” They’re not experts, and we can’t expect them to become experts. The threats change constantly, the likelihood of failure is low, and there is enough complexity that it’s hard for people to understand how to connect their behavior to eventual outcomes. So they turn to folk remedies that, while simple, don’t really address the threats.

Even if we could invent an effective computer security training program, there’s one last problem. HIV prevention training works because affecting what the average person does is valuable. Even if only half the population practices safe sex, those actions dramatically reduce the spread of HIV. But computer security is often only as strong as the weakest link. If four-fifths of company employees learn to choose better passwords, or not to click on dodgy links, one-fifth still get it wrong and the bad guys still get in. As long as we build systems that are vulnerable to the worst case, raising the average case won’t make them more secure.

The whole concept of security awareness training demonstrates how the computer industry has failed. We should be designing systems that won’t let users choose lousy passwords and don’t care what links a user clicks on. We should be designing systems that conform to their folk beliefs of security, rather than forcing them to learn new ones. Microsoft has a great rule about system messages that require the user to make a decision. They should be NEAT: necessary, explained, actionable, and tested. That’s how we should be designing security interfaces. And we should be spending money on security training for developers. These are people who can be taught expertise in a fast-changing environment, and this is a situation where raising the average behavior increases the security of the overall system.

If we security engineers do our job right, users will get their awareness training informally and organically, from their colleagues and friends. People will learn the correct folk models of security, and be able to make decisions using them. Then maybe an organization can spend an hour a year reminding their employees what good security means at that organization, both on the computer and off. That makes a whole lot more sense.

This essay originally appeared on DarkReading.com.

There is lots of commentary on this one.

EDITED TO ADD (4/4): Another commentary.

EDITED TO ADD (4/8): more commentary.

EDITED TO ADD (4/23): Another opinion.

Posted on March 27, 2013 at 6:47 AM • View Comments

Getting Security Incentives Right

One of the problems with motivating proper security behavior within an organization is that the incentives are all wrong. It doesn’t matter how much management tells employees that security is important, employees know when it really isn’t—when getting the job done cheaply and on schedule is much more important.

It seems to me that his co-workers understand the risks better than he does. They know what the real risks are at work, and that they all revolve around not getting the job done. Those risks are real and tangible, and employees feel them all the time. The risks of not following security procedures are much less real. Maybe the employee will get caught, but probably not. And even if he does get caught, the penalties aren’t serious.

Given this accurate risk analysis, any rational employee will regularly circumvent security to get his or her job done. That’s what the company rewards, and that’s what the company actually wants.

“Fire someone who breaks security procedure, quickly and publicly,” I suggested to the presenter. “That’ll increase security awareness faster than any of your posters or lectures or newsletters.” If the risks are real, people will get it.

Similarly, there’s a supposedly an old Chinese proverb that goes “hang one, warn a thousand.” Or to put it another way, we’re really good at risk management. And there’s John Byng, whose execution gave rise to the Voltaire quote (in French): “in this country, it is good to kill an admiral from time to time, in order to encourage the others.”

I thought of all this when I read about the new security procedures surrounding the upcoming papal election:

According to the order, which the Vatican made available in English on Monday afternoon, those few who are allowed into the secret vote to act as aides will be required to take an oath of secrecy.

“I will observe absolute and perpetual secrecy with all who are not part of the College of Cardinal electors concerning all matters directly or indirectly related to the ballots cast and their scrutiny for the election of the Supreme Pontiff,” the oath reads.

“I declare that I take this oath fully aware that an infraction thereof will make me subject to the penalty of excommunication ‘latae sententiae’, which is reserved to the Apostolic See,” it continues.

Excommunication is like being fired, only it lasts for eternity.

I’m not optimistic about the College of Cardinals being able to maintain absolute secrecy during the election, because electronic devices have become so small, and electronic communications so ubiquitous. Unless someone wins on one of the first ballots—a 2/3 majority is required to elect the next pope, so if the various factions entrench they could be at it for a while—there are going to be leaks. Perhaps accidental, perhaps strategic: these cardinals are fallible men, after all.

Posted on March 4, 2013 at 6:38 AM • View Comments

Massive Police Shootout in Cleveland Despite Lack of Criminals

This is an amazing story. I urge you to read the whole thing, but here’s the basics:

A November car chase ended in a “full blown-out” firefight, with glass and bullets flying, according to Cleveland police officers who described for investigators the chaotic scene at the end of the deadly 25-minute pursuit.

But when the smoky haze—caused by rapid fire of nearly 140 bullets in less than 30 seconds—dissipated, it soon became clear that more than a dozen officers had been firing at one another across a middle school parking lot in East Cleveland.

At the end of the scene, both unarmed—and presumably innocent—people in the car were dead.

There’s a lot that can be said here, but I don’t feel qualified to say it. There’s a whole body of research on decision making under stress—police, firefighters, soldiers—and how easy it is to get caught up in the heat of the moment. I have read one book on that subject, Sources of Power, but that was years ago.

What interests me right now is how this whole situation was colored by what “society” is talking about and afraid of, which became the preconceptions the officers brought to the event. School shootings are in the news, so as soon as the car drove into a school parking lot, the police assumed the worst. Firefights with dangerous criminals are what we see on TV, so that’s not unexpected, either. When you read the story, it’s clear how many of the elements that the officers believed—police cars being rammed, for example—are right out of television violence. This would have turned out very differently if the officers had assumed that, as is almost always true, the two people in the car were just two people in a car.

I’m also curious as to how much technology contributed to this. Reports on the radio brought more and more officers to the scene, and misinformation was broadcast over the radio.

Again, I’m not really qualified to write about any of this. But it’s what I’ve been thinking about.

Posted on February 12, 2013 at 12:55 PM • View Comments

Jared Diamond on Common Risks

Jared Diamond has an op-ed in the New York Times where he talks about how we overestimate rare risks and underestimate common ones. Nothing new here—I and others have written about this sort of thing extensively—but he says that this is a bias found more in developed countries than in primitive cultures.

I first became aware of the New Guineans’ attitude toward risk on a trip into a forest when I proposed pitching our tents under a tall and beautiful tree. To my surprise, my New Guinea friends absolutely refused. They explained that the tree was dead and might fall on us.

Yes, I had to agree, it was indeed dead. But I objected that it was so solid that it would be standing for many years. The New Guineans were unswayed, opting instead to sleep in the open without a tent.

I thought that their fears were greatly exaggerated, verging on paranoia. In the following years, though, I came to realize that every night that I camped in a New Guinea forest, I heard a tree falling. And when I did a frequency/risk calculation, I understood their point of view.

Consider: If you’re a New Guinean living in the forest, and if you adopt the bad habit of sleeping under dead trees whose odds of falling on you that particular night are only 1 in 1,000, you’ll be dead within a few years. In fact, my wife was nearly killed by a falling tree last year, and I’ve survived numerous nearly fatal situations in New Guinea.

Diamond has a point. While it’s universally true that humans exaggerate rare and spectacular risks and downplay mundane and common risks, we in developed countries do it more. The reason, I think, is how fears propagate. If someone in New Guinea gets eaten by a tiger—do they even have tigers in New Guinea?—then those who know the victim or hear about it learn to fear tiger attacks. If it happens in the U.S., it’s the lead story on every news program, and the entire country fears tigers. Technology magnifies rare risks. Think of plane crashes versus car crashes. Think of school shooters versus home accidents. Think of 9/11 versus everything else.

On the other side of the coin, we in the developed world have largely made the pedestrian risks invisible. Diamond makes the point that, for an older man, falling is a huge risk, and showering is especially dangerous. How many people do you know who have fallen in the shower and seriously hurt themselves? I can’t think of anyone. We tend to compartmentalize our old, our poor, our different—and their accidents don’t make the news. Unless it’s someone we know personally, we don’t hear about it.

EDITED TO ADD (2/21): George Burns fatally fell in the shower at age 98.

Posted on February 1, 2013 at 6:08 AM • View Comments

Book Review: Against Security

Against Security: How We Go Wrong at Airports, Subways, and Other Sites of Ambiguous Danger, by Harvey Molotch, Princeton University Press, 278 pages, $35.

Security is both a feeling and a reality, and the two are different things. People can feel secure when they’re actually not, and they can be secure even when they believe otherwise.

This discord explains much of what passes for our national discourse on security policy. Security measures often are nothing more than security theater, making people feel safer without actually increasing their protection.

A lot of psychological research has tried to make sense out of security, fear, risk, and safety. But however fascinating the academic literature is, it often misses the broader social dynamics. New York University’s Harvey Molotch helpfully brings a sociologist’s perspective to the subject in his new book Against Security.

Molotch delves deeply into a few examples and uses them to derive general principles. He starts Against Security with a mundane topic: the security of public restrooms. It’s a setting he knows better than most, having authored Toilet: The Public Restroom and the Politics of Sharing (New York University Press) in 2010. It turns out the toilet is not a bad place to begin a discussion of the sociology of security.

People fear various things in public restrooms: crime, disease, embarrassment. Different cultures either ignore those fears or address them in culture-specific ways. Many public lavatories, for example, have no-touch flushing mechanisms, no-touch sinks, no-touch towel dispensers, and even no-touch doors, while some Japanese commodes play prerecorded sounds of water running, to better disguise the embarrassing tinkle.

Restrooms have also been places where, historically and in some locations, people could do drugs or engage in gay sex. Sen. Larry Craig (R-Idaho) was arrested in 2007 for soliciting sex in the bathroom at the Minneapolis-St. Paul International Airport, suggesting that such behavior is not a thing of the past. To combat these risks, the managers of some bathrooms—men’s rooms in American bus stations, in particular—have taken to removing the doors from the toilet stalls, forcing everyone to defecate in public to ensure that no one does anything untoward (or unsafe) behind closed doors.

Subsequent chapters discuss security in subways, at airports, and on airplanes; at Ground Zero in lower Manhattan; and after Hurricane Katrina in New Orleans. Each of these chapters is an interesting sociological discussion of both the feeling and reality of security, and all of them make for fascinating reading. Molotch has clearly done his homework, conducting interviews on the ground, asking questions designed to elicit surprising information.

Molotch demonstrates how complex and interdependent the factors that comprise security are. Sometimes we implement security measures against one threat, only to magnify another. He points out that more people have died in car crashes since 9/11 because they were afraid to fly—or because they didn’t want to deal with airport security—than died during the terrorist attacks. Or to take a more prosaic example, special “high-entry” subway turn­stiles make it much harder for people to sneak in for a free ride but also make platform evacuations much slower in the case of an emergency.

The common thread in Against Security is that effective security comes less from the top down and more from the bottom up. Molotch’s subtitle telegraphs this conclusion: “How We Go Wrong at Airports, Subways, and Other Sites of Ambiguous Danger.” It’s the word ambiguous that’s important here. When we don’t know what sort of threats we want to defend against, it makes sense to give the people closest to whatever is happening the authority and the flexibility to do what is necessary. In many of Molotch’s anecdotes and examples, the authority figure—a subway train driver, a policeman—has to break existing rules to provide the security needed in a particular situation. Many security failures are exacerbated by a reflexive adherence to regulations.

Molotch is absolutely right to home in on this kind of individual initiative and resilience as a critical source of true security. Current U.S. security policy is overly focused on specific threats. We defend individual buildings and monuments. We defend airplanes against certain terrorist tactics: shoe bombs, liquid bombs, underwear bombs. These measures have limited value because the number of potential terrorist tactics and targets is much greater than the ones we have recently observed. Does it really make sense to spend a gazillion dollars just to force terrorists to switch tactics? Or drive to a different target? In the face of modern society’s ambiguous dangers, it is flexibility that makes security effective.

We get much more bang for our security dollar by not trying to guess what terrorists are going to do next. Investigation, intelligence, and emergency response are where we should be spending our money. That doesn’t mean mass surveillance of everyone or the entrapment of incompetent terrorist wannabes; it means tracking down leads—the sort of thing that caught the 2006 U.K. liquid bombers. They chose their tactic specifically to evade established airport security at the time, but they were arrested in their London apartments well before they got to the airport on the strength of other kinds of intelligence.

In his review of Against Security in Times Higher Education, aviation security expert Omar Malik takes issue with the book’s seeming trivialization of the airplane threat and Molotch’s failure to discuss terrorist tactics. “Nor does he touch on the multitude of objects and materials that can be turned into weapons,” Malik laments. But this is precisely the point. Our fears of terrorism are wildly out of proportion to the actual threat, and an analysis of various movie-plot threats does nothing to make us safer.

In addition to urging people to be more reasonable about potential threats, Molotch makes a strong case for optimism and kindness. Treating every air traveler as a potential terrorist and every Hurricane Katrina refugee as a potential looter is dehumanizing. Molotch argues that we do better as a society when we trust and respect people more. Yes, the occasional bad thing will happen, but 1) it happens less often, and is less damaging, than you probably think, and 2) individuals naturally organize to defend each other. This is what happened during the evacuation of the Twin Towers and in the aftermath of Katrina before official security took over. Those in charge often do a worse job than the common people on the ground.

While that message will please skeptics of authority, Molotch sees a role for government as well. In fact, many of his lessons are primarily aimed at government agencies, to help them design and implement more effective security systems. His final chapter is invaluable on that score, discussing how we should focus on nurturing the good in most people—by giving them the ability and freedom to self-organize in the event of a security disaster, for example—rather than focusing solely on the evil of the very few. It is a hopeful yet realistic message for an irrationally anxious time. Whether those government agencies will listen is another question entirely.

This review was originally published at reason.com.

Posted on December 14, 2012 at 12:24 PM • View Comments

1 … 4 5 6 7 8 … 26

Sidebar photo of Bruce Schneier by Joe MacInnis.