Fake Facts on Twitter - Schneier on Security

rekres • April 24, 2009 7:29 AM

A clever hack? Not really. What he got was NOT conservatives, but rather tweet-heads.

The majority of conservatives that I know aren’t shallow, brain-dead people and generally aren’t attracted to Twitter at all.

Travis • April 24, 2009 7:39 AM

rekres : No societal group is immune from idiots, those were most definately conservatives. Us liberals have our own share of idiots too, don’t feel bad….

Paul Crowley • April 24, 2009 7:40 AM

@rekres: and that would be why the salt marsh mouse myth was so quickly debunked in conservative circles?

More seriously, I went Googling for some of the InTheStimulus inventions to find people who fell for them. I can find lots of people discussing them, but I’m having a hard time finding the tweets or blogs of the fooled. In fact, so far I’ve found exactly one.

rekres • April 24, 2009 7:51 AM

You’re talking about Twitter here. Yes its easy to fool people when you go looking in the shallow end of the pool.

You’ve only got 140 characters to express your thoughts and back them up with logical arguments? HA!

Jim Lillicotch • April 24, 2009 8:06 AM

Paul: Try Googling “RT@InTheStimulus”, you’ll get a lot more than one.

rekres • April 24, 2009 8:35 AM

One question I have about all of this. How does the prankster know the ideology or political party of those who were following him?

He went to a Republican twitter feed, but no where does he ask what party or ideology those followers who got ‘suckered’ by him claimed. How many of those were moderates or liberals masquerading as monkey wrenches? How many of those were un-affiliated?

It was just assumed that those following a Republican twitter group were conservatives. If you make a general assumption that half of the followers were actually conservative, then wouldn’t that mean that half were not? And of those that followed, only a portion actually responded…

My point is that it seems he is drawing conclusions from an incomplete set of data, a set that was slanted in the first place. Did he repeat the experiment by going into a Democrat Twitter group and posting blatant falsehoods about Republican politicians?

HJohn • April 24, 2009 8:37 AM

@Travis: “No societal group is immune from idiots, those were most definately conservatives. Us liberals have our own share of idiots too, don’t feel bad….”

My dad always said, “there is an idiot born every 10 seconds… most of them live, most of them drive, most of them reproduce, and every single one of them talks.”

I had lunch with a delightful and otherwise intelligent friend of mine last week. But there is one thing that makes him inexplicably irrational: George W. Bush. A year ago, he was really worried (literally, physically ill) about something he read on the internet that said Bush was going to prove he really did lose in 2000, and therefore was legitimately only on his first term, and could therefore run for reelection in 2008. I just said to him “you believe that nonsense?” I use this example because this is an otherwise intelligent and well informed person, but he has a blind spot.

I’m not surprised that so many people believe lies about Obama, just as my friend (and many others) did about Bush. And I would suspect that, like my friend, many people who fall for nonsense are usually intelligent people. It’s very easy to deny the obvious or believe the obviously wrong when one has feelings that reinforce them. That’s why a lie about Obama is more likely to be believed by conservatives, and lies about Bush are more likely believed by liberals. (I’m talking psychology and bias here, please note I have not said anything good or bad about either president and will not debate the good or bad either.)

Bruce has written a lot about the “Psychology of Security” and why we downplay and exagerrate certain things when all logic is points the other way. Perhaps this one would be similar, like the “Psychology of Logic.”

Have a nice weekend. I hope everyone’s day is as beautiful and sunny as the one I’m enjoying right now.

Bruce Schneier • April 24, 2009 8:45 AM

I don’t want this to turn into a political discussion. What’s interesting here is not the political affiliation of any of the players, but the limited bandwidth of Twitter and how it affects out ability to authenticate.

yabbadabbadoo • April 24, 2009 8:50 AM

rekres: “The majority of conservatives that I know aren’t shallow, brain-dead people and generally aren’t attracted to Twitter at all.”

@rekres: You wish! The venerable conservative think tank American Enterprise Institute is now on Twitter and Facebook: http://www.kstreetcafe.com/aei-jumps-on-the-social-media-bandwagon/

Bruce: “Twitter is an ideal medium to inject fake facts into society for precisely this reason.”

@Bruce: You might also want to see the website http://www.gullible.info/ which has served the same purpose for entertainment for a few years now. By now it has raised it to an art form.

dmc • April 24, 2009 9:00 AM

The problem, of course, is that once a meme like a Twit is injected, it sticks around forever and if it’s a really good one gets amplified into something we all talk about. Eventually it finds its way onto CNN and Fox. It becomes absolutely impossible to debunk at some point. I imagine that in 20 years, there will be some people who truly believe that there was $2 Million for ShamWows in the bill.

Carlo Graziani • April 24, 2009 9:28 AM

Funny, this connects oddly to something that fell into place for me while contemplating Cheney’s passionate defense of torture-based interrogation: the modern American conservative movement appears to be dominated by people who think that “Reason” means “Marshalling one’s beliefs in support of one’s evidence”. To these people, evidence is something that you use to persuade others of what you know already, not something you use to evolve and tune your own beliefs.

This epistemic stance dominated the thinking at the senior ranks of the Bush administration, and I’m starting to understand that it was what I found most appalling about the way we were governed during the past eight years. Apparently, however, it’s a broader conservative pathology, not confined to just the Bush people. I can’t help wondering whether it helps or hurts the GOP, politically.

Awesome Robot • April 24, 2009 9:28 AM

I don’t know if you can blame it on limited bandwidth. Most people do not bother to authenticate information when it comes from people they know or admire. Look at all of the terribly false email chain letters out there, and before that, mail chain letters, and before that, gossip and “old-wives tales”. The difference is speed. People aren’t any more gullible, it’s just getting faster and faster for false information to spread amongst the gullible.

BF Skinner • April 24, 2009 9:34 AM

@Bruce: “Twitter is an ideal medium to inject fake facts into society for precisely this reason.”

So what is needed is a way to regenerate the signal;

Woesinger • April 24, 2009 9:42 AM

When the Mumbai attack happened and there was a lot of press about how Twitter was becoming a conduit for live information on the situation, it occurred to me that it would be incredibly easy for terrorists (or whoever) to poison the information well with false Tweets.

Jeff Dege • April 24, 2009 9:43 AM

http://www.mercurynews.com/peninsula/ci_11696283

“Then where did the $30 million figure come from, if it’s not in the bill? It turns out that $30 million is the total amount that the California Coastal Conservancy, a state agency, recommended more than a month ago to numerous federal agencies, looking for lists of “shovel ready” projects as part of the stimulus bill planning.

The conservancy’s wish list included five major ongoing wetlands restoration projects totaling nearly 4,000 acres, said civil engineer Steve Ritchie, a Coastal Conservancy staff member who helped draw it up. And the federal Army Corps of Engineers included all five projects on its own list of possible ways to spend stimulus money.

The projects, which range from Napa County to Silicon Valley, involve moving levees, creating islands and converting former industrial salt ponds back to marshes. Each could begin by year’s end and would benefit dozens of species, including salmon, steelhead trout, ducks, egrets, and yes, the endangered mouse, Ritchie said.”

Obi-w00t • April 24, 2009 9:54 AM

You could certainly have seen something like this coming. Twitter appears to have associated itself with everything and everyone, and Twitter seems to be the ultimate rumour-mill.

You could say this is associated with the concept of “The Big Lie”. It just means that no matter how ludicrous a lie is, someone will repeat it in the media as fact. Then when it is uncovered as a lie, the retraction is never given as much coverage as the initial story – people only remember the lie.

News organisations have stated that Twitter is important in covering complicated news events – such as G20 – so in a way Twitter has become its own news organisation, with its own inflated sense of authority. The problem is that, even though the news is supposed to be more “interactive”, the news is still controlled by people who are more informed about current events than the general public. Problem is, Twitter is the general public. So any falsehood can quickly be spread by people with absolutely no accountability whatsoever. At least before somebody might have gotten in trouble for spreading the lie in the first place, now there is no way to hold the perpetrators of the lie responsible.

Or maybe I’m just overreacting 🙂

Richard Cypher • April 24, 2009 9:58 AM

The Wizard’s First Rule, according to the book by the same title, written by Terry Goodkind:

[paraphrased] People are stupid. They will beleive things because they are afraid they are true or because they want to beleive they are true. People think they can tell the diffence between the truth and a lie, but really can’t making them all the more easy to fool.

Sam Bowne • April 24, 2009 9:58 AM

I don’t think Twitter makes it harder to present reasoned arguments than any other primarily social medium, such as a chat room. You just link to an article or blog entry–that’s what most people I follow do. Newspapers work the same way. The tweet is just a headline, the link included is for people who care about details, evidence, etc.

I think humans have a strong herd instinct, and a deep emotional need to locate a popular leader, so they can feel safe and supported by a tribe. The purpose of their conversation is then just to demonstrate adoration and loyalty to the leader. The actual truth of assertions is of very little importance. I think 90% of human communication serves this purpose, and it’s the same thing birds do in a flock clucking at one another.

So a fake fact that supports one political agenda or another is almost as useful as a true fact. Its purpose is to inspire anger at the “enemy” in another, and thus to create a stronger bond with them emotionally. The actual fact doesn’t matter, what matters is the “team spirit” created.

Joining tribes isn’t brain-dead, illogical, or stupid, either. It has real survival value, and is built into us for good reasons.

HJohn • April 24, 2009 10:05 AM

@Bruce: “I don’t want this to turn into a political discussion. What’s interesting here is not the political affiliation of any of the players, but the limited bandwidth of Twitter and how it affects out ability to authenticate.”

True, that is fascinating. My apologies if my fascination with how bias affected the psychology of one’s judgment is what caused your comment. Affiliation was never my concern (the bias the impaired judgment was simply based on it).

I’m kind of at a loss for what the best solution for Twitter would be. I’m guessing upping their bandwidth wouldn’t be very cost effective (not just in money, but I’m guessing it would be disruptive in the interim as well), yet leaving it unchanged poses this obvious risk (their information, use, as well as credibility). Anyone more adept than I have advice for them?

Awesome Robot • April 24, 2009 10:21 AM

The ONLY reason twitter is limited to 140 characters per message is so that the messages fit into a standard cell-phone text message. Cell phone compatibility is what limits lots of other things like threaded replies, though some Twitter clients have ways of solving those kinds of usability problems.

SenseiTim • April 24, 2009 10:22 AM

Twitter should be renamed “Wacko Party Line”! Seriously, the service is drowning in it’s own success, jammed beyond capacity most of the time, and 99.99999% of ‘tweets’ are utter nonsense.

Fun to read sometimes, tho!

sooth sayer • April 24, 2009 10:36 AM

Only repulicans?

Let’s keep it a bit technical – we know your affiliation but ignore it for science

But are you sure that those things AREN’T in the bill?

$700B is a lot of money and even Democrats couldn’t figure out how to spend it all in 1 year — so they have squirreled it away for the next 5.

Tim • April 24, 2009 11:08 AM

The problem with the stimulus package was the fact that is was written in relative secrecy. No one person could have known what the package included, even the legislators who contributed to the bill. The fact that a conservative (or liberal for that matter) would be starving to know what the bill contained is the entire reason that any single individual would tend to trust someone who claims to have authoritative knowledge.

Had the bill been vetted with the promised 48 hours of public review, this could never have happened.

HJohn • April 24, 2009 11:32 AM

The actual quality and value of the stimulus bill (and, for full disclosure, i’ll admit I think it was a mistake and a sham), is wholly irrelevant as to the security weaknesses in Twitter that allowed the hack to occur.

Someone else mentioned the point that the size of the messages is designed for cell phone text accommodations (whether or not this was in the original article said this is not known to me, I can’t link to the KOS at work). Basically, Twitter made a business decision to accommodate a certain technology and has to decide if the trade off of the authentication problems is worth it.

dohbama • April 24, 2009 11:33 AM

The flipside is also true. A negative truth about Obama and the stimulus will be under reported and downplayed by ABC, CBS, NBC, MSNBC, CNN, the newspapers, and the faculty at most schools. The shell game of cutting a token few million dollars from the budget on camera while ballooning the federal deficit is ridiculous, yet the news media somehow manages to keep a straight face while reporting this as real progress.

Lev Rickards • April 24, 2009 11:46 AM

Not convinced that limited bandwidth leads to authentication difficulty in this case. As Sam Browne mentioned, placing a citation link into your tweet is facile. Link-shorteners like bit.ly and is.gd make it easy to reference your statement.

Are there better ways to authenticate in this case? It seems clear that Twitter is an effective method for “[injecting] fake facts into society.” But you need a community of people committed to investigating truth in an independent fashion.

Pat Cahalan • April 24, 2009 11:57 AM

@ Lev

Not convinced that limited bandwidth leads to authentication
difficulty in this case.

Me either. A bunch of scientists have spent the last couple of days dismantling Jim Carrey’s recent op-ed on Huff Pro (check out Phil Plait’s “Bad Astronomy” blog to start). Giving people more bandwidth doesn’t solve the problem of clearing out garbage input to begin with 🙂

I think Bruce’s point about confirmation bias is spot-on, but I’m not terribly convinced that Twitter is any more perfect of a medium to inject false facts into society than any other Internet-enabled anything… except for one thing, it’s currently the rage to use.

It’s simple networking theory that the value of the network goes up as you add nodes. I think there’s a definite point in the adoption curve of any Internet-enabled technology where you reach a most pessimum point; the barrier to add to the network is so small that the signal to noise ratio starts decreasing exponentially. Twitter seems to have reached that point already.

Now, maybe there might be something to the theory that the size of the messages alters that curve (Twitter got there faster than usenet, for example), but I suspect the difference is a constant. 🙂

Team America • April 24, 2009 12:08 PM

Looks like a network of rating agents could solve this problem. If you trust the source there really isn’t any need to do the verification every time.

Every announcement on Reuters fits well below 140 characters, and we have no problem trusting them.

@nincompoops

Voters can’t tell the difference between:

every vote counts
every vote is counted

and fail to spot the equivalence of:

30 million subsidy for breeding rodents
30 million subsidy for XYZ

Pat Cahalan • April 24, 2009 12:16 PM

@ Team America

Looks like a network of rating agents could solve this problem.
If you trust the source there really isn’t any need to do the
verification every time.

This is a problem in and of itself. Do you get to pick who the experts are? Well, then you wind up with two large competing groups, each labeled as “experts” by themselves, each spewing out garbage. If you’re new to the network, how do you know which group actually contains the experts?

Roboticus • April 24, 2009 12:40 PM

I personally think the fact that bills are rushed through congress so quickly is itself the source of many of the problems. Most congress people do not know exactly what they are voting on otherwise they would have known immediately about the “salt marsh mouse” being fake. As far as limited bandwith on Twitter, most people dont want to actually research things anyway and let people like Mr. Schneier, the EFF, Glen Beck, Bush or Obama basically tell them what they should think. (I’m not saying the above people do it on purpose and I happen to like 3 of them anyway.) People dont want to confirm facts unfortunately. For the people who do, maybe a law like the one proposed by http://www.readthebill.org would help as it would require congress to post the text of a bill online for 72 hours before debate begins. Who knows? Maybe even some congress people would read the bills before they voted on them.

bob • April 24, 2009 1:04 PM

“two large competing groups”

Why would you think there will only be two?

G-man • April 24, 2009 1:14 PM

I don’t understand the popularity of Twitter. All it amounts to is the equivalent of the status line on someone’s Facebook page. Why bother?

n • April 24, 2009 1:19 PM

Congress doesn’t read or care what’s in a bill, so long as theres something in that that gets money to their campaign contributors.

The entire budget process needs an overhaul.

Paul • April 24, 2009 2:56 PM

I’m not so sure it’s the message length which matters here. Don’t forget that plenty of people were forwarding emails with untrue or factually distorted content during election season. There’s plenty of room in an email to reinforce or refute an argument, but few bothered to question what they read.

The medium involved is what makes this work so well. Consider how easy it is to forward an email, re-tweet on Twitter, or post a blog comment. Then compare that to how hard it is to verify the authenticity of something and how easy it is to believe something which fits neatly alongside your point of view, especially when you’re feeling outraged by what you just read and want to tell everyone who agrees with you “so we can do something about it”.

Greg • April 24, 2009 3:39 PM

Fake Facts on Twitter?

No surprise there considering we’ve been getting fake facts from the news media ever since the advent of the 24 hour news cycle.

Not one media outlet has the time to research their facts in their rush to be the first to publish, and very few publish retractions and of those that do, even less readers actually ever see them and nobody certainly talks about them because we are already on to the next set of fake facts.

The 15 second sound byte has disseminated more misinformation than Twitter ever will.

Bob Stratton • April 24, 2009 3:49 PM

Greg is right on the mark. I don’t subscribe to everything Chomsky has to say, but his indictments of the unwillingness to entertain cogent articulations of anything on television news is well stated.

Exceptions seem to include the real BBC World News channel (which I’ve never seen available on US satellite/cable, though Canada has it), and Al Jazeera.

Subhankar Ray • April 24, 2009 5:12 PM

That’s why Real-time news from Twitter has a credibility problem. It needs a Wikipedia like devoted editorial board, but then Twitter has to become a non-profit.

Regards,
Subhankar Ray

Tosk59 • April 24, 2009 7:34 PM

For all the “Twitter full of lies and crap…” group, yup indeed. Prime example (not!) is @schneierblog with the article headlines and links to this blog!!

It’s no different (i.e. 99% crap & 1% great) than Usenet, IRC, chat, blogs, web, etc.

http://twitter.com/schneierblog

Davi Ottenheimer • April 24, 2009 8:07 PM

The BBC had a story a little while ago about militias in Somalia that use cellphone and SMS command structures.

I wondered about the same authentication issues. Supposedly they rely on voice recognition, but that seems hardly reliable on cell signal/speakers. Perhaps it’s only a matter of time perhaps before they gravitate to tweets.

http://davi.poetry.org/blog/?p=3969

Joe • April 24, 2009 8:33 PM

This is exactly why I only trust comments on message boards.

Roger • April 24, 2009 11:06 PM

So, any wagers on how long it takes for Twitter to be used to depress / inflate a stock price for a fraudulent speculator?

WarLord • April 25, 2009 12:32 AM

This focused hack worked because of hashtagging I would venture #tcot is the right wing calling card for tweets.

See Hashtag.org for explanation of hashtags on twitter

Don’t twitter? so sad, your loss. #justsayin

not who you think • April 25, 2009 11:56 AM

@Roger

Already happened. Twittering about Steve Jobs health from unreliable sources caused a brief drop in Apple’s price last fall. Was intentionally devised to degrde the price? Probably not. Did it prove the principle? Yes.

nostromo • April 25, 2009 4:12 PM

twitter works as meme spread because the size limit turns each message into a motto. No propaganda expert will convey long explanatory messages, but sequences of slogans sewn together. twitter makes easy to invent mottos.

After the bombings in Madrid, March 11, 2004, there was a strong movement using SMS to join forces mobilising people to demonstrate and vote against the government, and I think it worked from the same reasons: “mottos coming from close friends and spreading exponentially and very fast” == “distributed-mutiny” 🙂

behind the 8 ball • April 26, 2009 2:14 AM

what are you all on about… the medium has not changed the possibilities… just made them more or less likely… but in thise case
“fake facts” … when i meet you, i’ll tell you how big the fish i caught was, and you can tell me if technology is helping promote lies, or if they existed before technology…

Redfox • April 26, 2009 2:47 PM

Here is an interesting an highly relevant Article about how misinformations about Swine Flu spread on Twitter:

http://neteffect.foreignpolicy.com/posts/2009/04/25/swine_flu_twitters_power_to_misinform

Redfox • April 27, 2009 5:14 AM

And (just like ordered 😉 ), xkcd on Swine Flu and Twitter:

http://xkcd.com/574

RH • April 27, 2009 11:35 AM

I am a longtime follower of the theory of Memetics (not really a theory per se, but I digress). Memetics is the idea that ‘ideas’ can reproduce, just like living beings. While we have our genes, ideas have ‘memes,’ the smallest reproducing part of an idea. The classic example is that song you hate to hear on the radio, but catch yourself humming anyways.

Twitter is the extreme example of what a Times editor (I long forget the name) described as “bad information getting cheaper, while good information still costs the same.” With twitter, you can get 100% unverified facts the INSTANT someone writes them.

What links these ideas… what are the chances of a meme eventually learning to do nothing but reproduce itself on twitter… an uuber Rickrolling meme or something of its kind. It would be a parasite on society, yet by social evolution it could be incredibly difficult to stop. Perhaps something like penny stocks, but tweaked in such a way that a scarce few marks actually DO make money.

We could become mere hosts for the next great lifeform… our own thoughts!

jman • April 28, 2009 2:56 AM

Could Twitter be the perfect platform to spread viruses and worms. Imagine a news twitter that reports correct news for a couple of hours. Once a few thousand people have subscribed, a new news is spread with a link, e.g.

swine flu more dangerous than expected and some tinyurl.com/foo/bar

The link than refers to some malware site that contains the necessary browser exploits. Since there is no way to find out what’s behind the url, it should work quite well.

Outfoxed • May 1, 2009 3:56 AM

Brevity may be the soul of wit, but concision is the death of truth.

WDB • May 15, 2009 8:38 AM

I briefly followed Twitter and one of the things that struck me most was just how off base most everything was. 140 characters written in stream of consciousness by the semi-illiterate products of texting on cellphones is pretty thin stuff.

At any rate a friend of mine who is very left leaning and Muslim posted tweeted about a ‘poison gas attack on a Muslim pre-school’. I followed that to its source on the Dailey KOS and from there to a newspaper article from Ohio. The truth seems to have been that some children found an empty cannister for dog repellent spray, took it to one of there mothers who was doing something at a Mosque at the time and she called 911. This was followed by a small scale panic.

Sid Viscous • May 15, 2009 12:53 PM

Interesting. So I go to the link at the top of the story

http://www.congressmatters.com/storyonly/2009/2/13/102933/725/384/627

So lets see the source of his claims.
First where he calls the cclaim of wetlands money in the stimulus package is complete “horseshit” is here

http://theplumline.whorunsgov.com/stimulus-package/pelosi-staff-conservative-talking-point-about-30-million-for-mice-is-fabrication/

Which is just Nancy Pelosi saying it’s not true, or at least she wasn’t briefed on it right. Hardly a strong argument, we are a the “He said she said” stage.

The second link he says is a “thorough debunking and summary” but is in fact a dead link.

So how exactly is this story about twitter any different than his blog page, and why would either be credible.

Interestingly I can no longer find the searchable stimulus package link and I aint reading 1075 pages to see if in fact the wetlands package is or is not in the stimulus package. Remember the “Mouse” is actually a red herring. Teh complaint is about the wetlands project, any wetlands project is going to be implemented to same some endangered critter, mouse or not, the main complain is the existence of the wetlands project.

So what we have is an article about a blog site that doesn’t have (at least currently) verification of it’s claims complaining about twitter not having verification of their claims.

Schneier on Security

Comments

Leave a comment Cancel reply