Page 12 of 26
I have been thinking a lot about security against psychopaths. Or, at least, how we have traditionally secured social systems against these sorts of people, and how we can secure our socio-technical systems against them. I don’t know if I have any conclusions yet, only a short reading list.
EDITED TO ADD (12/12): Good article from 2001. The sociobiology of sociopathy. Psychopathic fraudsters and how they function in bureaucracies.
It’s kind of an amazing story. A young Asian man used a rubber mask to disguise himself as an old Caucasian man and, with a passport photo that matched his disguise, got through all customs and airport security checks and onto a plane to Canada.
The fact that this sort of thing happens occasionally doesn’t surprise me. It’s human nature that we miss this sort of thing. I wrote about it in Beyond Fear (pages 153–4):
No matter how much training they get, airport screeners routinely miss guns and knives packed in carry-on luggage. In part, that’s the result of human beings having developed the evolutionary survival skill of pattern matching: the ability to pick out patterns from masses of random visual data. Is that a ripe fruit on that tree? Is that a lion stalking quietly through the grass? We are so good at this that we see patterns in anything, even if they’re not really there: faces in inkblots, images in clouds, and trends in graphs of random data. Generating false positives helped us stay alive; maybe that wasn’t a lion that your ancestor saw, but it was better to be safe than sorry. Unfortunately, that survival skill also has a failure mode. As talented as we are at detecting patterns in random data, we are equally terrible at detecting exceptions in uniform data. The quality-control inspector at Spacely Sprockets, staring at a production line filled with identical sprockets looking for the one that is different, can’t do it. The brain quickly concludes that all the sprockets are the same, so there’s no point paying attention. Each new sprocket confirms the pattern. By the time an anomalous sprocket rolls off the assembly line, the brain simply doesn’t notice it. This psychological problem has been identified in inspectors of all kinds; people can’t remain alert to rare events, so they slip by.
A customs officer spends hours looking at people and comparing their faces with their passport photos. They do it on autopilot. Will they catch someone in a rubber mask that looks like their passport photo? Probably, but certainly not all the time.
Yes, this is a security risk, but it’s not a big one. Because while—occasionally—a gun can slip through a metal detector or a masked man can slip through customs, it doesn’t happen reliably. So the bad guys can’t build a plot around it.
One last point: the young man in the old-man mask was captured by Canadian police. His fellow passengers noticed him. So in the end, his plot failed. Security didn’t fail, although a bunch of pieces of it did.
EDITED TO ADD (11/10): Comment (from below) about what actually happened.
From the Wall Street Journal:
Take “stranger danger,” the classic Halloween horror. Even when I was a kid, back in the “Bewitched” and “Brady Bunch” costume era, parents were already worried about neighbors poisoning candy. Sure, the folks down the street might smile and wave the rest of the year, but apparently they were just biding their time before stuffing us silly with strychnine-laced Smarties.
That was a wacky idea, but we bought it. We still buy it, even though Joel Best, a sociologist at the University of Delaware, has researched the topic and spends every October telling the press that there has never been a single case of any child being killed by a stranger’s Halloween candy. (Oh, yes, he concedes, there was once a Texas boy poisoned by a Pixie Stix. But his dad did it for the insurance money. He was executed.)
Anyway, you’d think that word would get out: poisoned candy not happening. But instead, most Halloween articles to this day tell parents to feed children a big meal before they go trick-or-treating, so they won’t be tempted to eat any candy before bringing it home for inspection.
[…]
Then along came new fears. Parents are warned annually not to let their children wear costumes that are too tight—those could seriously restrict breathing! But not too loose either—kids could trip! Fall! Die!
Treating parents like idiots who couldn’t possibly notice that their kid is turning blue or falling on his face might seem like a losing proposition, but it caught on too.
Halloween taught marketers that parents are willing to be warned about anything, no matter how preposterous, and then they’re willing to be sold whatever solutions the market can come up with. Face paint so no mask will obscure a child’s vision. Purell, so no child touches a germ. And the biggest boondoggle of all: an adult-supervised party, so no child encounters anything exciting, er, “dangerous.”
I remember one year when I filled a few Pixie Stix with garlic powder. But that was a long time ago.
EDITED TO ADD (11/2): Interesting essay:
The precise methods of the imaginary Halloween sadist are especially interesting. Apples and home goods occasionally appear in the stories, but the most common culprit is regular candy. This crazed person would purchase candy, open the wrapper, and DO SOMETHING to it, something that would be designed to hurt the unsuspecting child. But also something that would be sufficiently obvious and clumsy that the vigilant parent could spot it (hence the primacy of candy inspection).
The idea that someone, even a greedy child, might consume candies hiding razor blades and needles without noticing seems to strain credulity. And how, exactly, a person might go about coating a jelly bean with arsenic or lacing a molasses chew with Drano has never been clear to me. Yet it is an undisputed fact of Halloween hygiene: Unwrapped candy is the number-one suspect. If Halloween candy is missing a wrapper, or if the wrapper seems loose or flimsy, the candy goes straight into the trash.
Here is where I think we can discover some deeper meanings in the myth of the Halloween sadist. It’s all about the wrappers.
Wrappers are like candy condoms: Safe candy is candy that is covered and sealed. And not just any wrapper will do. Loose, casual, cheap wrappers, the kind of wrappers one might find on locally produced candies or non-brand-name candies, are also liable to send candy to Halloween purgatory. The close, tight factory wrapper says “sealed for your protection.” And the recognized brand name on the wrapper also lends a reassuring aura of corporate responsibility and accountability. It’s a basic axiom of consumer faith: The bigger the brand, the safer the candy.
Ironic, since we know that the most serious food dangers are those that originate from just the kind of large-scale industrial food processing environments that also bring us name-brand, mass-market candies. Salmonella, E. coli, and their bacterial buddies lurking in bagged salads and pre-formed hamburger patties are real food dangers; home-made cookies laced with ground glass are not.
EDITED TO ADD (11/11): Wondermark comments.
This is no surprise:
The people behind the new study start by asking a pretty obvious question: “Why do members of the public disagree—sharply and persistently—about facts on which expert scientists largely agree?” (Elsewhere, they refer to the “intense political contestation over empirical issues on which technical experts largely agree.”) In this regard, the numbers from the Pew survey are pretty informative. Ninety-seven percent of the members of the American Association for the Advancement of Science accept the evidence for evolution, but at least 40 percent of the public thinks that major differences remain in scientific opinion on this topic. Clearly, the scientific community isn’t succeeding in making the public aware of its opinion.
According to the new study, this isn’t necessarily the fault of the scientists, though. The authors favor a model, called the cultural cognition of risk, which “refers to the tendency of individuals to form risk perceptions that are congenial to their values.” This wouldn’t apply directly to evolution, but would to climate change: if your cultural values make you less likely to accept the policy implications of our current scientific understanding, then you’ll be less likely to accept the science.
But, as the authors note, opponents of a scientific consensus often try to claim to be opposing it on scientific, rather than cultural grounds. “Public debates rarely feature open resistance to science,” they note, “the parties to such disputes are much more likely to advance diametrically opposed claims about what the scientific evidence really shows.” To get there, those doing the arguing must ultimately be selective about what evidence and experts they accept—they listen to, and remember, those who tell them what they want to hear. “The cultural cognition thesis predicts that individuals will more readily recall instances of experts taking the position that is consistent with their cultural predisposition than ones taking positions inconsistent with it,” the paper suggests.
[…]
So, it’s not just a matter of the public not understanding the expert opinions of places like the National Academies of science; they simply discount the expertise associated with any opinion they’d rather not hear.
Here’s the paper.
From NPR:
Based on surveys Barnes collected, the top five worries of parents are, in order:
- Kidnapping
- School snipers
- Terrorists
- Dangerous strangers
- Drugs
But how do children really get hurt or killed?
- Car accidents
- Homicide (usually committed by a person who knows the child, not a stranger)
- Abuse
- Suicide
- Drowning
Why such a big discrepancy between worries and reality? Barnes says parents fixate on rare events because they internalize horrific stories they hear on the news or from a friend without stopping to think about the odds the same thing could happen to their children.
No surprise to any regular reader of this blog.
How do most wrongful convictions come about?
The primary cause is mistaken identification. Actually, I wouldn’t call it mistaken identification; I’d call it misidentification, because you often find that there was some sort of misconduct by the police. In a lot of cases, the victim initially wasn’t so sure. And then the police say, “Oh, no, you got the right guy. In fact, we think he’s done two others that we just couldn’t get him for.” Or: “Yup, that’s who we thought it was all along, great call.”
It’s disturbing that misidentifications still play such a large role in wrongful convictions, given that we’ve known about the fallibility of eyewitness testimony for over a century.
In terms of empirical studies, that’s right. And 30 or 40 years ago, the Supreme Court acknowledged that eyewitness identification is problematic and can lead to wrongful convictions. The trouble is, it instructed lower courts to determine the validity of eyewitness testimony based on a lot of factors that are irrelevant, like the certainty of the witness. But the certainty you express [in court] a year and half later has nothing to do with how certain you felt two days after the event when you picked the photograph out of the array or picked the guy out of the lineup. You become more certain over time; that’s just the way the mind works. With the passage of time, your story becomes your reality. You get wedded to your own version.
And the police participate in this. They show the victim the same picture again and again to prepare her for the trial. So at a certain point you’re no longer remembering the event; you’re just remembering this picture that you keep seeing.
David Ropeik is a writer and consultant who specializes in risk perception and communication. His book, How Risky Is It, Really?: Why Our Fears Don’t Always Match the Facts, is a solid introduction to the biology, psychology, and sociology of risk. If you’re well-read on the topic already, you won’t find much you didn’t already know. But if this is a new topic for you, or if you want a well-organized guide to the current research on risk perception all in one place, this pretty close to the perfect book.
Ropeik builds his model of human risk perception from the inside out. Chapter 1 is about fear, our largely subconscious reaction to risk. Chapter 2 discusses bounded rationality, the cognitive shortcuts that allow us to efficiently make risk trade-offs. Chapter 3 discusses some of the common cognitive biases we have that cause us to either overestimate or underestimate risk: trust, control, choice, natural vs. man-made, fairness, etc.—thirteen in all. Finally, Chapter 4 discusses the sociological aspects of risk perception: how our estimation of risk depends on that of the people around us.
The book is primarily about how we humans get risk wrong: how our perception of risk differs from the reality of risk. But Ropeik is careful not to use the word “wrong,” and repeatedly warns us not to do it. Risk perception is not right or wrong, he says; it simply is. I don’t agree with this. There is both a feeling and reality of risk and security, and when they differ, we make bad security trade-offs. If you think your risk of dying in a terrorist attack, or of your children being kidnapped, is higher than it really is, you’re going to make bad security trade-offs. Yes, security theater has its place, but we should try to make that place as small as we can.
In Chapter 5, Ropeik tries his hand at solutions to this problem: “closing the perception gap” is how he puts it; reducing the difference between the feeling of security and the reality is how I like to explain it. This is his weakest chapter, but it’s also a very hard problem. My writings along this line are similarly weak. Still, his ideas are worth reading and thinking about.
I don’t have any other complaints with the book. Ropeik nicely balances readability with scientific rigor, his examples are interesting and illustrative, and he is comprehensive without being boring. Extensive footnotes allow the reader to explore the actual research behind the generalities. Even though I didn’t learn much from reading it, I enjoyed the ride.
How Risky Is It, Really? is available in hardcover and for the Kindle. Presumably a paperback will come out in a year or so. Ropeik has a blog, although he doesn’t update it much.
Here’s a book from 1921 on how to profile people.
This sign is from a gas station in the U.K.
<img alt=”sign saying ‘Police Notice: Don’t Commit Crime'” src=”/images/dont-commit-crime.jpg” width=500 height=400″>
My first reaction was to laugh, but then I started thinking about it. We know that signs like “No Shoplifting” reduce shoplifting in the area around the sign, but those are warnings against a specific crime. Could a sign this general be effective? Clearly some comparative studies are needed.
EDITED TO ADD (7/7): This is part of a larger sign. Presumably, whoever put up the sign I saw cut the top and bottom off.
I’m at SHB 2010, the Third Interdisciplinary Workshop on Security and Human Behavior, at Cambridge University. This is a two-day gathering of computer security researchers, psychologists, behavioral economists, sociologists, philosophers, and others—all of whom are studying the human side of security—organized by Ross Anderson, Alessandro Acquisti, and myself.
Here is the program. The list of attendees contains links to readings from each of them—definitely a good place to browse for more information on this topic.
Here are links to my posts on the first and second SHB workshops. Follow those links to find summaries, papers, and audio recordings of the workshops. I may liveblog this workshop—I did it last year—but I may just pay attention. Ross Anderson has liveblogged the previous two years, and is very likely to do so again. There will also be audio.
EDITED TO ADD (6/28): Ross is liveblogging the workshop here. I’m not; I find I pay better attention when I’m not trying to take coherent and accessible notes.
Sidebar photo of Bruce Schneier by Joe MacInnis.