Schneier on Security
A blog covering security and security technology.
« Canadian Privacy Commissioner Comments on the No-Fly List |
| Redefining Privacy »
November 14, 2007
The Sham of Criminal Profiling
Malcolm Gladwell makes a convincing case that criminal profiling is nothing more than a "cold reading" magic trick.
A few years ago, Alison went back to the case of the teacher who was murdered on the roof of her building in the Bronx. He wanted to know why, if the F.B.I.'s approach to criminal profiling was based on such simplistic psychology, it continues to have such a sterling reputation. The answer, he suspected, lay in the way the profiles were written, and, sure enough, when he broke down the rooftop-killer analysis, sentence by sentence, he found that it was so full of unverifiable and contradictory and ambiguous language that it could support virtually any interpretation.
Astrologers and psychics have known these tricks for years. The magician Ian Rowland, in his classic "The Full Facts Book of Cold Reading," itemizes them one by one, in what could easily serve as a manual for the beginner profiler. First is the Rainbow Ruse -- the "statement which credits the client with both a personality trait and its opposite." ("I would say that on the whole you can be rather a quiet, self effacing type, but when the circumstances are right, you can be quite the life and soul of the party if the mood strikes you.") The Jacques Statement, named for the character in "As You Like It" who gives the Seven Ages of Man speech, tailors the prediction to the age of the subject. To someone in his late thirties or early forties, for example, the psychic says, "If you are honest about it, you often get to wondering what happened to all those dreams you had when you were younger." There is the Barnum Statement, the assertion so general that anyone would agree, and the Fuzzy Fact, the seemingly factual statement couched in a way that "leaves plenty of scope to be developed into something more specific." ("I can see a connection with Europe, possibly Britain, or it could be the warmer, Mediterranean part?") And that's only the start: there is the Greener Grass technique, the Diverted Question, the Russian Doll, Sugar Lumps, not to mention Forking and the Good Chance Guess -- all of which, when put together in skillful combination, can convince even the most skeptical observer that he or she is in the presence of real insight.
They had been at it for almost six hours. The best minds in the F.B.I. had given the Wichita detectives a blueprint for their investigation. Look for an American male with a possible connection to the military. His I.Q. will be above 105. He will like to masturbate, and will be aloof and selfish in bed. He will drive a decent car. He will be a "now" person. He won't be comfortable with women. But he may have women friends. He will be a lone wolf. But he will be able to function in social settings. He won't be unmemorable. But he will be unknowable. He will be either never married, divorced, or married, and if he was or is married his wife will be younger or older. He may or may not live in a rental, and might be lower class, upper lower class, lower middle class or middle class. And he will be crazy like a fox, as opposed to being mental. If you're keeping score, that's a Jacques Statement, two Barnum Statements, four Rainbow Ruses, a Good Chance Guess, two predictions that aren't really predictions because they could never be verified -- and nothing even close to the salient fact that BTK was a pillar of his community, the president of his church and the married father of two.
Posted on November 14, 2007 at 6:47 AM
• 36 Comments
To receive these entries once a month by e-mail, sign up for the Crypto-Gram Newsletter.
Gladwell makes some good points. Makes me want to look up Rowland's book.
I wonder how applicable his findings are to the 'expert commentary' field - the assortment of talking heads on the Sunday morning talk shows, and so on.
His findings are applicable to much predictive commentary. But some predictions are given credit for having science or math as its basis. Sports, economics, and politics (polls) all have math. Weather has science. Psychology - in this case profiling - has science.
But here the science is clearly not, in fact, able to draw meaningful predictions out of the details of a crime scene.
It's true that modern psychology is a scientific discipline, but the only scientific pyschological approaches examining criminal profiling have been those that disprove its effectiveness (as detailed in the article).
The psychology on which profiling is based seems to originate in Freudian psychology, which is fundamentally unscientific (it makes no specific predictions that can be tested conclusively), and reasoning based on anecdotes, intuition, and introspection (such as the haphazard discussions with inmates the profilers used as a basis for their system).
Also, I doubt whether or not most political commentators even need to tricks for "cold reading," since they can for the most part make whatever predictions they want without any justification, and often suffer no consequences for being wrong.
The issue isn't so much whether science can draw predictions as whether the output of the profiling effort actually contains any. I don't mind paying for people to apply science, but I don't want to pay for bunkum. If we're paying profilers to come up with predictions, they appear to be doing a poor job in the cited profiles (but scientifically speaking, we may be looking at a skewed sample).
If on the other hand we're paying them to motivate law enforcement personnel, and they're using this method to accomplish that, then they deserve the pay. But then I want smarter law enforcement personnel. Either way it's unsatisfactory :).
Scientific predictions are useful because they are testable. If you believe them they point in a particular direction and if that direction is wrong they point out flaws in the models used to create them. If I predict that tomorrow will be sunny with a chance of clouds or foul weather, have I said anything testable? There's science in weather prediction but my forecast isn't science, so I shouldn't be paid (in the role of a scientist) to produce it.
The bottom-line on profiling seems to be: Spend lots of money to generate a multisentence expansion of the the statment "The criminal exists". Sounds like progress to me.
I've mentioned it a couple of times, but the "profile" they came up with for the serial killer in Louisiana could have been any white male over the age of 20. They rounded up several hundred young white guys and DNA swabbed them. When they finally, actually caught the serial killer, it turned out to be a black male. Predictably, the DNA swabs of the falsely accused were not destroyed in light of this "new" evidence - after all, no one is truly innocent.
"It's true that modern psychology is a scientific discipline"
I honestly cannot agree with that statement. Its a humanities discipline which tries to apply the scientific method, and when science cannot provide an answer a whole bunch of crap gets spakled into the cracks. Then when anyone tries to argue with the results some one trots out the " its based in science" quote. Its like trowing a pinch of diamond dust into plaster of paris and claiming the resulting wall is a s strong as diamond.
Profiling is really just the tip of the iceberg.
So how does this compare with behavioural profiling that Bruce recommends for airport security?
It may be true that some profile is badly defined, particularly to cover todays sensitivities of religion ethnicity and the like, but to condemn the idea of criminal profiling is naive.
The word "criminal" describes a profile by itself and leaving some random acts aside, it's a pretty good starting point.
>> The word "criminal" describes a profile by itself
As opposed to "crazy" I presume, or perhaps "white collar criminal" on the other end.
The gradient seems to involve a lot of poor impulse control, driven by greed, or mental illness, or in some cases frustrated ambition.
Criminology is one of the few social sciences where laypeople immediately assume professional expertise on the basis of watching TV and movies. CSI has only worsened this phenomenon.
I would recommend you check out Nassem Nicholas Taleb's book 'The Black Swan.' He makes a pretty compelling case against any effort at prediction, and provides some pretty specific examples, particularly pertaining to the modern financial industry. It really makes you wonder if the supposedly scientific underpinings of some of these forecasts are good for anything more than producing the veneer of rational justification. How much more so, then, for psychology, where it's that much harder to quantify individual mental state and behavior in a consistent and broadly applicable way?
Weather prediction is also math, as it is nothing more than, "In the past (for which we have records) weather conditions matching our current weather have been followed by ..." which is statistics. That's where the "30 percent chance of rain" comes from -- it just means that in the past, 30% of instances of our current weather have been followed by rain. Our predictive abilities have gotten better not because of new ideas about weather, but because our records are becoming greater in number and are recording more information.
Science is only useful for testing (validating or disproving) hypotheses. Once proven by the scientific method, application usually involves math.
@spider: "[Psychology is] a humanities discipline which tries to apply the scientific method, and when science cannot provide an answer a whole bunch of crap gets spakled into the cracks."
There is certainly a lot of crap out there that claims to be "psychology", but that doesn't mean psychology isn't a science.
@Tom Davis, there is more to weather prediction than you acknowledge. There is an admittedly imperfect but improving understanding of mechanisms involved in producing both micro and macro scale weather events.
Incidentally, the "30 per cent chance of showers" is sometimes best translated as "100% chance of showers covering 30% of the forecast area". In other words, we're sure it's going to rain but we're not sure exactly where...
@sooth_sayer>> The word "criminal" describes a profile by itself
no, the word "criminal" describes an atomic feature, that being a person who has committed a crime. It says nothing about any other features in a profile. "White collar criminal" or "con artist" might be profiles, but even they are limited in the scope of information they convey.
A few years back I found there was a fad of bringing criminal profilers on to the worthless 24 hour news networks so they could tell us the profile of the D.C. Sniper or the unabomber. In both cases they used the same kinds of tricks. "A young man, possibly a bit older..." etc.
It is pretty much worthless astrology.
Probably applies to all the psychics you see on these cold case shows or criminal investigation shows as well. Someone should apply the same analysis to them.
Does this mean the TV show wasn't real?
Indeed, several someones have. See "Psychic Sleuths" by Joe Nickell, in particular.
>>"It's true that modern psychology is a scientific discipline"
>I honestly cannot agree with that statement. Its a humanities discipline which tries to apply the scientific method
I'd just like to point out that there are at least three disciplines within 'psychology', with rather little in common with each other : perceptional psychology, animal psych, human psych. (Other people may divide things in other ways.)
Perceptional psychology includes questions like how close in time two stimuli can be before they interfere with each other. This is very different from "why does little Johnny torture cats" psychology.
(I'm not a psychologist, but I know a perceptional psychologist well.)
"laypeople immediately assume professional expertise on the basis of watching TV and movies.."
Laypeople "develop" expertise as they are the other end of every crime.
There was a time when I knew not a single New Yorker who wasn't robbed (mugged as per the TV channels) in the city.
To say one can't have a view about their environment without a PhD in "criminology" is crazy .. even Jesse Jackson stated ON RECORD that he would be scared if he met young men of certain ethnic background in a dark alley.
Is that criminal profiling or racial profiling, and what input lets you correlate one to the other .. these are simple observations however writing papers on them would always be impossible .. even if you have a PhD.
To understand the value of PhD one can watch wizad of oz again .. or read this blog :-(())
The New Yorker article did a pretty good debunking job on the profilers.
Personally, I am reluctant to abandon the profiling concept completely and I humbly suggest a few things that could be done to improve matters:
- All the profiling should be done in writing.
- Profilers should be required to make specific assertions and explain those assertions and attempt to give a relative weighting to their assertions.
- The profilers work should always be retrospectively assessed if the criminal is caught.
An obvious corollary is the more successful profiles should be studied carefully to determine which assertions and justifications seem to work relatively well.
It seems possible that the above might even be used to build up a body of useful knowledge that could be used by the police without the assistance of a 'professional'. Of course the profilers will absolutely hate this and fight against it all the way.
That Rowland book is excellent. It told me that you often think about that scar on your left knee. And you wish you had kept up your interest in music from your school years.
There's a remarkable amount of confusion in this thread between the profiling discussed here--which is to take a crime committed by person or persons unknown and attempt to infer the characteristics of the criminal--and the sort which involves looking at someone and guessing whether they're likely to be dangerous, or for that matter whether they're smuggling something. That latter can be broken down in all sorts of ways--"he looks nervous" is a different kind of clue than "this apparently fat person has a suitcase full of size 4 clothing," both of which are different from deciding that since x% of crimes are committed by people of a certain ethnicity, or y% are committed by males between the ages of 10 and 50, you should pay more attention to people who fit those descriptions.
The assertion that criminal profiling in the sense discussed by Gladwell and our host is a form of the cold reading trick has nothing to do with the other two concepts.
I'd like to see more betting pools on this kind of thing. Doesn't need to be money, could just be some abstract money-like quantity.
If some cop's gut instinct does better than the best psychological profiling out there, more power to them.
What good does it do to assess a profile's accuracy after the fact when one of the acknowledged 'experts' of the field (John Douglas) admits he can't explain how he turns the evidence left at the crime scene into a profile of the offender? Further, as the article points out, you can see similar behavior occuring for different reasons, so it's not always possible to conclude that evidence X correlates to behavior Y. Besides, if you make enough predictions, some of them will prove right by sheer chance, and analyzing them won't tell us how to do better in the future.
I'm guessing, by the comments here, that the previous poster is a male ... young 20's or a very young-at-heart 30's with brown hair, a bit overweight. A college education -- but didn't finish his college ambitions. He's caucasian, living in the northeast. Married but either no children or his children are very young.
Where do I sign up to be a profiler??
@Mycroft: You're mistaking methods for results.
It matters not whether the profiler can explain how she analyzed the situation and decided what was important. What matters is whether she was right. Given a track record with a significant percentage of hits, it'll likely be more useful to consult her than other profilers.
Of course, this requires said profiler to come up with some actual specifics, not a Delphic oracle that can be read both ways.
Excellent article - thank you for pointing it out!
I feel it's rather obvious, but you haven't covered it, there has to be a desire by the listener/viewer/reader, to believe what is being dished out to them.
In the case of "terrorists", they are the current legitimate object of hate, so people feel good about hating them and hearing news about terrorists being under attack. All of which leaves one gullible.
What or who will be the next legitimate object of hate? Well I suppose it takes the heat off fat/gay/black people for now? Unless people think that some fat, gay, black person is also a terrorist.
Solid, sensible article. Robert Ressler, the former director of the FBI's Violent Criminal Apprehension Program, and the man who coined "serial killer" has for more than 20 years supported psychic medium Noreen Renier's activities alongside police. Ressler even sanctioned Renier to lecture at the FBI and also directly on a case involving a crashed aircraft. He has admitted to furnishing her name to "dozens" of law enforcement personnel. For those of you who do not know the FBI's profiling/behavioral pseudoscience activities I might suggest reviewing my own website's overview on the bizarre connections to one particular case. Please see http://www.amindformurder.com/runwaypart1.htm Again, good article. Much more needs to be explored on the masquerades taking place with public funds connected to paranormal nonsense.
The web site for background material covering FBI officer Robert Ressler and his 30-years of paranormal support behind TV psychic detective Noreen Renier has been changed. The original site is now managed by Noreen Renier herself and the critical overview is posted at http://www.commentarybysherlock.com/
It might be accurate. Was watch a movie(stop profiling now) that showed a killer cutting a throat.
One the person should have been a farmer/hunter(throat,sheep /deer).Two wouldn't someone just stick a knife into someone, if they didn't have any knowledge to use that option.
A Soldier/hunter might have gone for the heart or the brain/spinal cord.
Some one that didn't have any connection to the victim would use 2-10 stocks, someone that did 20-50.
What would be the accuracy?
I should learn to shut up :rolleyes:
Schneier.com is a personal website. Opinions expressed are not necessarily those of Co3 Systems, Inc.