Schneier on Security
A blog covering security and security technology.
« Cost/Benefit Questions NSA Surveillance |
| Petition the NSA to Subject its Surveillance Program to Public Comment »
June 19, 2013
Finding Sociopaths on Facebook
On his blog, Scott Adams suggests that it might be possible to identify sociopaths based on their interactions on social media.
My hypothesis is that science will someday be able to identify sociopaths and terrorists by their patterns of Facebook and Internet use. I'll bet normal people interact with Facebook in ways that sociopaths and terrorists couldn't duplicate.
Anyone can post fake photos and acquire lots of friends who are actually acquaintances. But I'll bet there are so many patterns and tendencies of "normal" use on Facebook that a terrorist wouldn't be able to successfully fake it.
Okay, but so what? Imagine you had such an amazingly accurate test...then what? Do we investigate those who test positive, even though there's no suspicion that they've actually done anything? Do we follow them around? Subject them to additional screening at airports? Throw them in jail because we know the streets will be safer because of it? Do we want to live in a Minority Report world?
The problem isn't just that such a system is wrong, it's that the mathematics of testing makes this sort of thing pretty ineffective in practice. It's called the "base rate fallacy." Suppose you have a test that's 90% accurate in identifying both sociopaths and non-sociopaths. If you assume that 4% of people are sociopaths, then the chance of someone who tests positive actually being a sociopath is 26%. (For every thousand people tested, 90% of the 40 sociopaths will test positive, but so will 10% of the 960 non-sociopaths.) You have postulate a test with an amazing 99% accuracy -- only a 1% false positive rate -- even to have an 80% chance of someone testing positive actually being a sociopath.
This fallacy isn't new. It's the same thinking that caused us to intern Japanese-Americans during World War II, stop people in their cars because they're black, and frisk them at airports because they're Muslim. It's the same thinking behind massive NSA surveillance programs like PRISM. It's one of the things that scares me about police DNA databases.
Many authors have written stories about thoughtcrime. Who has written about genecrime?
BTW, if you want to meet an actual sociopath, I recommend this book (review here) and this blog.
Posted on June 19, 2013 at 11:19 AM
• 57 Comments
To receive these entries once a month by e-mail, sign up for the Crypto-Gram Newsletter.
Hmm... about the Japanese-Americans interned during World War II, I heard someone in a documentary state that the authorities could identify Japanese spies but did not want anyone to know they knew who they were. So they put them all together in one place. This may have been equally egregious, but I found it plausible since the same tactic was used in other places at other times.
I'm stuck on trying to understand the premise. Even if one can distinguish "normal" from "sociopathic" usage, sociopath trying to emulate "normal" usage only has to duplicate the usage pattern of one non-sociopath to be successful, right? The more versions of "normal" usage there are, the harder it is to detect non-"normal" usage, right?
We would have to put them in positions of power because, they can get things done without caring who gets crushed and stepped on. Oh wait..., that's how it's already done.
Until you know what sociopaths don't do "normally," wouldn't they be included in the group of "normal" behavior?
You would think that Scott Adams, of all people, would know that the sociopaths would be in charge of the algorithm.
A couple weeks ago I was talking to someone at a social gathering at a bar who worked on AML detection algorithms for banks and regulators. He said the best thing criminals could do to outsmart the algorithm but don't is to periodically exceed the $10,000 reporting threshold. When money launders are trying to avoid detection, they pretty quickly figure out the smurfing strategy, and even change deposits and transfers to use random amounts below 10k, instead of $9,999, which over and over again is pretty obviously suspicious.
The one thing legitimate cash business do that most criminals don't, however, is randomly exceed the reporting threshold. If you're legit and your receipts are randomly distributed within a certain time-income range, then there's nothing magical about $10,000. In other words, they're not trying to look for people who break the threshold, but people who look like they're trying to avoid it. That's why they're not concerned about the threshold being a secret.
The fallacy still applies here though. The number of businesses who just have daily takes just shy of 10k are probably much larger than the number money laundering operations. If we jailed everyone on that basis alone, the policy would collapse pretty quickly.
A big problem by trying to make a determination by what people don't do, is assuming that people all use it for the same purpose and with the same frequency. Like trying to prove something by its lack of evidence to the contrary.
I had a Facebook profile at one time and I hardly used it. So are normal people the ones who use Facebook a lot? If you don't use Facebook are you anti-social or a sociopath? When I closed my account, I had a few people ask me what was wrong. As though I must have been in psychological turmoil to close my Facebook account. That mentality in itself is scary to me. Kind of like The Stepford Wives.
my wife gave me something of a hard time when you she looked at where the law professor who wrote this taught. But if he's saying these things at that University, Kudos to him. I wonder how long he'll last there.
But I noted, if there is any essay on "why i have nothing to hide" that destroys the argument, this is it.
I can't even get past the premise - I know too many people who are intelligent and articulate in person, but who wouldn't pass the Turing test on FB. Social media interaction is not a good gague of character.
Scott Adams' idea reminds me of a different PKD story, Do Androids Dream of Electric Sheep? Being able to comb facebook for the sociopathic "signature" could be quite useful in discovering the androids among us.
@simon According to family lore, at the time of the internment, a friend of one of my great-aunts was questioned by the FBI over the course of many days because she had some suspicious documents: knitting instructions.
Here is an interesting thing... plugging in a 95% accuracy, which is usually considered acceptable, and literally half of the people who test positive are false.
The danger is the false positives combined with ok? What to do with that information? Many gifted, talented, or creative people would fail a cursory look for "normal".
Steve jobs was be on antipsychotic drugs if evaluated during one of his temper tantrums. In his early days Carnegie would be viewed as a sociopath for lack of empathy and ill treatment of workers. He had to win was his view. But look at his philosophy and giving away almost everything to charity. Compare with Rockefeller's amassing wealth and giving It to family.
Point is human being are complex. Just as misguided as the uk trying to diagnose some preschoolers as future criminals.
Statistically significant has its limits.
I think this is one thing that scares me about the "what could have been done to prevent this" argument, detailing the mental backgrounds of those like Seung-Hui Cho and Adam Lanza. While it is true those monsters had troubled backgrounds, it is also true that there are countless others who match their profile and who never -- and will never -- harm anyone. We'd have to literally hurt thousands for every one that might be spared. You can't just go after people up for being weird, isolated, and/or angry. Those factors are only relevant after the fact when tying motive to an act, but are perfectly legal personality quirks before the fact.
Naturally, this raises the question of how to categorise people who (like me) were invited by (actual) friends to sign up for Facebook, read Facebook's security and privacy policies, thought "you have got to be [elided] joking", and therefore aren't on Facebook...
I've been [slightly] worried about Scott Adams for years now, as he's been trending towards lunacy for quite some time.
First you'll need a useful definition of "sociopath" that is not, for practical purposes, equivalent to "non-conformist" or "adherent to a religion not on the approved list", etc.
You may underestimate the ability to create tautological tests. If you define a sociopath as someone who fails the sociopathy test, then the sociopathy test is 100% accurate in identifying sociopaths.
That's all well and good, until people begin to think that an attribute thus defined is useful for anything.
This fallacy isn't new. It's the same thinking that caused us to intern Japanese-Americans during World War II, stop people in their cars because they're black, and frisk them at airports because they're Muslim. It's the same thinking behind massive NSA surveillance programs like PRISM.
..and requiring all of us to submit to background checks even though we've done nothing wrong, and are no more culpable for the Sandy Hook shooting than immigrants, Muslims, and welfare recipients are for the Boston Marathon bombings.
Apart from numbers there is the implicit assumption that all terrorists are sociopaths.
The definition of what a sociopath exactly is not all that easy to answer, let alone to capture in an algorithm.
I would advise Scott to read "Mask of Sanity" by Hervey Cleckley.
Sociopaths *and* terrorists? Why, that's great. Next we just have to check if the people on that list are sociopaths. If they are, then they walk free, and if they are not, then they must be terrorists.
Seriously, why would sociopaths and terrorists have *anything* in common, let alone their Facebook habits?
Is there a medical or psychological test that can separate a "terrorist" from a "freedom fighter"?
My hypothesis is that science will someday be able to identify sociopaths and terrorists by their patterns of Facebook and Internet use.
I do not believe that that would be possible. I think that there are too many variations in the "normal" range for any test(s) to determine "sociopath" from Facebook posts.
sure about the 26% in the example?
isn't it 36/96=37.5% ?
A book I would recommend on sociopaths is the Psychopath Test. It lends some healthy doubt to the modern theories about sociopathy.
He does it in a well reasoned way: by, for instance, showing, as opposed to saying and letting the reader draw their own conclusions.
On finding "terrorists" by facebook, well, if they are talking about and recruiting on terrorism on facebook -- there you go.
But if you are talking about a super secret terrorist who never, ever mentions terrorism... well... uhm... what?
Maybe they do stuff like go to titty bars (like some 911 terrorists did) just to throw people.
So, you basically end up looking for people who are *secretly* terrorists. Could be anyone. Have you wiretapped your mother recently?
Have your kids been indoctrinated? Better bug their phones.
Glib, charming sociopaths or sullen sociopaths? Sociopaths who are the center or attention, or sociopaths who live in their mothers basement? CEO sociopaths or lone gunman sociopaths?
Ronson basically paints a picture of the guy who came up with the initial studies and test as a sociopath. And he presents damned good evidence for that, too.
But, why bother with pretensions? The global superpower already has mapped out networks of everyone and who they know, how well they know them via phone, facebook, email (probably). They can separate people into far more meaningful categories.
Who is a Republican, who is a Democrat, who is in the Tea Party, who is in Occupy, who is in Anonymous (for that matter), who is Baptist, who is Hindu.
The construction of the tower of babylon is well on the way. People were always content with only their divine leaders actually ever getting to the top. They were okay with just being cogs in the slavery which gets them there.
If I remember correctly, there was a competition on kaggle (http://www.kaggle.com) earlier in the year to do just this. Identify sociopathic tendencies (and other traits) based on peoples twitter feeds.
We are not talking about the future, this is a something of the here and now.
Of course any time you can create an algorithm, you can use it to create a persona which matches or does not.
Also there hasn't been a definition of "sociopath", or an implication that it is a binary state and not a continuum. And the threat of politicization: Is being a pro-life tea-party member "sociopathic", or is an obamacare guncontroller "sociopathatic".
Or where there is correlation, but the causation might be uncertain or inconvenient, e.g. many women are attracted to sociopaths (aka "bad boys"), so having a large number of women "followers" is an indicator. Like rock stars (or even Franz Lizst).
haarp: I got 36 true positives / 132 total positives as your true positive ratio (rate) which is 27%, not 26 or not 37.
Great blog post, Mr. Schneier.
An example of genecrime in film is "Gattaca", in which the protagonist illegally presents someone elses DNA as his own in order to circumvent rules which only allow genetically correct individuals to pursue particular careers.
Of course WWII was in many ways founded on genecrime, though generally we call it eugenics.
I think his base premise is flawed. Weren't the twitter/facebook friends of the younger Boston Marathon bomber, Dzhokhar Tsarnaev, really surprised that he could be a terrorist?
To name one obvious thing you could do with the hypothetical algorithm: do research on sociopaths.
Even if the resulting set is only, say, 50% sociopathic, that's still a huge amplification from the base rate of ~1%, and probably competitive in reliability with administering the HARE; now you have a huge ecologically-valid dataset of likely-sociopaths to mine for all sorts of interesting correlations, targeted surveys with rewards, etc.
A researcher wouldn't be concerned about the sample being 'only' 50% sociopath, since this would merely dilute any correlations or effects - they'd be more concerned about either half of the sample being systematically biased in some way.
> I can't even get past the premise - I know too many people who are intelligent and articulate in person, but who wouldn't pass the Turing test on FB. Social media interaction is not a good gague of character.
Completely irrelevant. One wouldn't be administering a Turing test. The question is merely whether some observable trait distinguishes; a measure of character could be liking the color green, for all we care! The question is only whether it non-zero statistically predicts or correlates with something.
Interesting post. Scott Adams - annoying as his ploys sometimes are, and as faulty as his logic may be, is often successful at purposely creating spirited discussion on absurd notions. But I think this time he's way off base. I think sociopaths could be much more easily identified from blog comments than from Facebook postings. (I think he knows this and throws out absurdities like this just to build up a database for profiling.)
I suggested something similar to Google a while ago, as part of their "Solve for X" campaign, although that was looking for psychopaths.
It is indeed possible to detect psychopaths by their online behaviour, as they tend to follow certain patterns. If anyone is interested, check out the book Nobody's Perfect by Saher Jones on Amazon. This book gives a lot of data, including actual transcripts of the online conversations, where you can see how psychopaths lure in their victims. It is a shocking, true story and the author wrote it as a warning, to help other people avoid the same traps.
Full disclosure: Saher Jones is my wife and I helped her write the book.
"The sad truth is that most evil is done by people who never make up their minds to be good or evil." -Hannah Arendt
this is the reason for why Medical people have multiple tests.
test#1 might have a 25% failure rate, but it is a screen for moving people to the slightly more invasive test#2.
test#2 might have a 10% failure rate for those who have passed test#1 (it may have a hirer rate for the General Population, that does not matter here).
those who pass test#2 are in for the highly invasive test#3 which has a 5% rate of FAIL.
those who pass all three tests are 95.7% likely to have the contagion and will be operated on.
similar layered testing used to exist in policing - eg go door-to-door to find the black people to create a shortlist of suspects.
In "Distraction" Bruce Sterling imagined what would happen if this technology actually worked. It let the Secret Service keep an eye on potentially dangerous nutters, but it was also subverted by bad actors. Basically, if you wanted to harass someone you would spam your nutter list with allegations that the target was a drug-dealing terrorist pervert, and this would cause a small minority of the nutters to attack your target in a way that could never be traced back to you.
So how long before the government starts to identify & track everybody who falls more than a couple of standard deviations away from "normal" behaviour. I wonder what solutions to the "problem" of social deviance various governments through history would have come up with, if they had been given the data?
Absolutely brilliant. Can't wait till some genius team of democratically elected representatives and their corporate lobbyists translate it into the "Protect the Children" bill and give the IC the powers and resources to not only spy on everyone, but also create psychological profiles in the process.
P(Test = +ve | Socio) = 0.9 = P(Test = -ve | not Socio), then
P(Socio | Test = +ve) = 0.9*0.04/(0.9*0.04 + 0.1*0.96) = 27%
if P(Test = +ve | Socio) = 0.9 = P(Test = -ve | not Soc), then
P(Socio | Test = +ve) = 0.9*0.04/(0.99*0.04 + 0.01*0.96) = 73%
for anyone interested.
Full disclosure: Saher Jones is my wife and I helped her write the book.>>
I think the concern is not on legitimate criminal profiling but the fluff taken to be legitimate criminal profiling.
According to the modern sociopathic craze, basically one in eight people are sociopaths. CEOs tend to be sociopathic, the argument goes. Lawyers, Journalists.
It is such a fuzzy standard you can scope it out to include whomever you want.
Consider: Take people's opinions. Who is for war? Who takes the Bible to say everyday people will be relentlessly tortured forever? Who strongly advocates capital punishment? Who thinks torture should be swept under the carpet? Who thinks it is in the spirit of the constitution to wiretap all Americans?
If some tragedy happens here, isn't it possible for the profiling of this sort to get widely scoped out to "who belonged or belongs to X political party"?
In the fifties, that party was the Communist Party. In a decade, maybe that will be the Republican or Democratic party.
People are surely aware there is a strong possibility all of their "private" online conversations including political and religious opinions are likely being archived and not deleted. And maybe, in the future, that will be used against them.
"You would think that Scott Adams, of all people, would know that the sociopaths would be in charge of the algorithm."
Ronson's book Psychopathy Test well shows they are.
The guy who initially came up with the theory and the test used to use powerful electric shocks to test for sociopathy. Basically, if you were not afraid of receiving an increasingly bad shock, you were a sociopathy.
That was with prison inmates. Because you know, you can't actually look at their criminal behavior in prison to deduce who is the worst.
About the base rate fallacy, the point here is not to catch all sociopaths and never make a mistake. The point is to get them at a higher rate than you miss them, say, 50%.
I find all this talk on privacy prettty stupid. Of course, PRISM was too much, and was too stupid also. But what are you guys suggestions to prevent terrorism from happening(before it does happen, of course), without putting privacy ~JUST A LITTLE BIT~ aside?
IF there was a reliable way of doing it, it should be done. One's particular privacy is less important than everyone's safety, and saying otherwise is selfishness.
This is easily my favorite math-based fallacy. Right when I had the light bulb moment I was like, "Eureka!". Though it's not entirely intuitive at first (hence it being a common fallacy), it's frustratingly stupid for a large organization to fall prey to this :(
"One's particular privacy is less important than everyone's safety, and saying otherwise is selfishness."...
I hope your kidding us, Johnes :/
"They who can give up essential liberty to obtain a little temporary safety, deserve neither liberty nor safety."
And... Franklin concurs that you do not deserve to live in the USA.
Another potential problem is that the terrorist might not be a sociopath. He might be an ordinary person whose family was killed by a drone attack.
One's particular privacy is less important than everyone's safety, and saying otherwise is selfishness
Err no you are working on a significantly flawed assumption that privacy and safety are at either end of a one dimensional spectrum, they most definatly are not.
It can be easily shown (and has been) that giving up privacy does not increase safety, infact the very oposit is more usually the case.
Further it can be shown (and has been) that changing individuals or everyones safety for the better does not have to decrease privacy, it can and often does actually increase it.
A simple example of the latter is the increase in car safety due to the standard fitting of airbags and Side Impact Bars. Prior to them being fitted road deaths and injuries were measurably higher, the fact that the police recorded the victims details and forwarded them for statistical use lowered the victimes privacy compared to those who were not victims. Thus decreasing the annual number of road victims by increasing safety reduces the number of disclosures of personal details, and thus privacy for individuals and the whole population has increased marginaly.
The conflation of "safety" and "privacy" has been deliberately done to justify an otherwise indefensible series of measures.
The recent Boston Bombings show that even when significant invasion of peoples privacy had occured by both Russian and US agencies it most certainly did not prevent the individuals concerned going on to commit acts of significant violence (some might justifiably argue that the invasion of their privacy actually caused or increased their radicalisation and thus caused in part their violent act).
I thought regular use of Facebook and/or Twitter was sociopathic behaviour.
"I find all this talk on privacy prettty stupid. Of course, PRISM was too much, and was too stupid also. But what are you guys suggestions to prevent terrorism from happening(before it does happen, of course), without putting privacy ~JUST A LITTLE BIT~ aside?
IF there was a reliable way of doing it, it should be done. One's particular privacy is less important than everyone's safety, and saying otherwise is selfishness."
So you are in an upper government position and a senator goes against you. What do you do when you are secretly wiretapping them (and all of their peers). One option is to disclose their dirt and downfall their career. Another option is to use their peers against them by threatening to expose their dirt. Or simply by arming their peers with all of the plans of their rivals.
All of that can be easily done without getting caught. We know this because Hoover did it for decades. The records for that are impeccable. It is not a conspiracy theory but documented by Presidents and Senators. Why they did not fire the guy or come forward during their tenures is obvious.
Hoover and the FBI was not alone in this behavior. The Gestapo and KGB did this. The Stasi did this. On and on and on one can go. What is hard to find is a system where this did not and does not ever happen.
All of these moves are very much like the colonialist powers the US revolted against to create the country and the founding documents in the first place.
So, it starts at undermining the entire government, then the corporations, and slowly - eventually - gets down to mass population control. Though, there is already a blanket of intimidation across everyone. It is hard for people to express opinions which might - even decades later - be construed as "against the powers that be on earth".
I strongly doubt most of the posters here have anything they are scared of "The Government" having.
All of these moves are also highly illegal. People can lamely and transparently argue otherwise, but you know it is bullshit ways to worm around the law. So, we have the world's primary "superpower" showing its' self as lawless.
The ramifications of that is severe. It justifies all crime and criminality.
As for safety, does it help safety to continue to create more assassin drones? Does it help to continue to prolong occupation in many countries through the world? Does it help to say, "We can break the law, you just can not"? Does it help to continue the occupations of Iraq and Afghanistan? Does it help to elevate "terrorism" light years above violent crime on the streets which far outnumbers terrorism and is a persistent, daily threat the majority of Americans have to deal with? Does it make friends or enemies to be arguing you can hack into any foreign company or politician without any manner of legal repercussion?
Was there good in the founding documents? If those are made immaterial and shown to be an enemy of the actions of the Government, is this still the same country at all?
That book, Confessions of a Sociopath (by M.E. Thomas), is suspect. Despite getting reviewed in the NYT, most people seem to think that, while the author is narcissistic and might have borderline personality disorder, it's not clear that she's a sociopath/psychopath.
There's also no agreement on how sociopathy and psychopathy should be defined. Only ASPD -- anti-social personality disorder -- is recognized in the DSM-IV; psychopath and sociopath are not clinical terms. Some people use them to mean someone diagnosed with ASPD. Other people use them differently, possibly as a strict subset of ASPD.
The state *probably* should not "follow them around" etc. on that basis. But we need a brain scan at birth or in childhood that can identify psychopaths (distinct? whatever) before they become a problem. And we certainly need to bad such children from ever becoming Wall St bankers who crash the world for their own sadistic amusement and enrichment. We know where this leads. If the bad guys keep having all the fun, seriously, where is my incentive not to become one?
I have lived and worked with sociopaths. They are challenging to identify even in real life. They demonstrate "extreme nervous breakdowns" at times with a sincere need for power over other individuals. This desire for power is often based on just the "need" for that. IMHO.
Arthur Me • June 20, 2013 11:49 AM
I have lived and worked with sociopaths. They are challenging to identify even in real life. They demonstrate "extreme nervous breakdowns" at times with a sincere need for power over other individuals. This desire for power is often based on just the "need" for that. IMHO.
You worked in congress? :-)
"But we need a brain scan at birth or in childhood that can identify psychopaths (distinct? whatever) before they become a problem."
A... do you sincerely consider those studies truly mature? Are you aware of any studies which have accurately predicted children's future behavior from childhood brain scans?
Have you ever thought about how young, relatively speaking this technology is, in the scheme of things? Are you aware that in the 1920s a very popular idea was that society could be perfected through these sorts of ideas?
The idea grew unpopular after Hitler was defeated and his policies became exposed to the light of day.
I think if this facebook thing was the only test used, then Bruce's concern would be valid. However, using multiple tests would probably solve it, or greatly ameliorate the issue. Ultimately, it's a conditional probability thing, if allowing for false positives, one 'facebook' test says there's a 50% chance, then applying a second test which says that there's a 90% chance then you probably are a sociopath.
So you could use it for screening, even if it's not proof.
"Rule 34" by Charles Stross is about a
FUTURE female Scottish police office
whose job is internet pre-crime by trolling
unusal sites to collect intel, find trends
patterns and memes.
The book is written from the POV of an AI
who "nudges" various characters including
There's a nasty but common way of getting rid of the "false positive" problem. Just declare all "false positives" to be "true positives." If you fail the test, then you are "bad."
@Jack I meant a brain scan that works. I did not particularly mean "We need to do this". We should do it when we can. And while there is no point in pretending that I have no sympathy for any Nazi policies, I would never claim that such a scan would "perfect" society. But I do think it would improve it a lot. I simply regard that the parts of the brain that light up in certain situations or with certain stimuli indicate whether the person is a psychopath or not: I say quite facetiously that it worked on House MD, which proves nothing. But can we at least stop the kids who are observed to torture and kill animals while enjoying it from working at big banks? Hitler did indeed discredit certain radical approaches. In some cases, that is a pity. If he had won and those policies had been in place in most of Europe we would not have the problems we do - but of course we would have different ones that are possibly worse.
Schneier.com is a personal website. Opinions expressed are not necessarily those of Co3 Systems, Inc.