Alex Stamos on Content Moderation and Security

Former Facebook CISO Alex Stamos argues that increasing political pressure on social media platforms to moderate content will give them a pretext to turn all end-to-end crypto off -- which would be more profitable for them and bad for society.

If we ask tech companies to fix ancient societal ills that are now reflected online with moderation, then we will end up with huge, democratically-unaccountable organizations controlling our lives in ways we never intended. And those ills will still exist below the surface.

Posted on January 15, 2019 at 5:55 AM • 26 Comments

Comments

AndersJanuary 15, 2019 9:38 AM

Sorry, but i don't buy that Alex Stamos DID NOT KNEW
about all those data schmingling and privacy violation
schemes FB was and is so fond of. It's little bit too late to
pretend to be innocent.

BradJanuary 15, 2019 9:51 AM

From what I know about Alex he seems like a well intentioned, sincere guy often put in a tough position (Facebook, Yahoo!). However, he's obviously really hurt his credibility. I think being where he was he has a unique perspective to consider, but will anyone listen? Probably not.

Impossibly StupidJanuary 15, 2019 10:01 AM

People can still vote with their feet and their wallets, so social media and other tech companies are still very accountable. What's really bad for society is that the profitability of many companies comes from advertising rather than from people directly paying for a valued service. That inherently means you need users who are most easily influenced by that shallow level of thinking. You can't (unless you're just as idiotic) cater to that kind of audience and then in the next breath complain that giving them what they want is an awful thing to do.

People posting the minutiae of their lives online don't want security. They don't care about encryption. They don't care that companies are run by sociopaths. Stop acting like you care, Alex; you're not fooling anybody.

PhaeteJanuary 15, 2019 10:05 AM

"Providing that privacy means that normal means of content moderation are now impossible, which leads to both organized abuses (by terrorists and child abusers) and societal-level challenges (like in India)."

I stopped taking people serious who play the terrorist/child abuse card against privacy.
Especially the above wording, providing security...leads to abuses of terrorist and child abusers.

It sounds to me like every office managers temper tantrum when i read those tweets.
Just because he wants to use targeted ads which is kinda difficult with full end-to-end encryption.

"Both of those guys are gone, as are a number of executives and engineers who were big proponents of mathematically guaranteed privacy."

Now things are starting to make sense, monetisation gained top priority due to power balance in company.
At least i never used their stuff, so no need to adjust.

Humma hummaJanuary 15, 2019 11:03 AM

"In ways we never intended..."

Your honor, it is true that I took an ax and clove my husband's head in two, but I never intended to kill him! That was just an unfortunate and unforseeable consequence. You can't convict me of murder on an inference.

Clive RobinsonJanuary 15, 2019 12:02 PM

@ Bruce,

Your interpretation of,

that increasing political pressure on social media platforms to moderate content will give them a pretext to turn all end-to-end crypto off -- which would be more profitable for them and bad for society.

Is correct as far as it goes and not difficult to reason out.

However I feel that "Alex Stamos" is aping the behaviour of a number of Federal directors of the likes of the DHS. Who play the political game all through their tenure, then on leaving turn around and recant what they have done.

It reminds me of people saying they are atheists then praying to god(s) when their lives change.

Proving not only can a turd be polished, it can also polish it's self when it suits...

Clive RobinsonJanuary 15, 2019 12:07 PM

@ Bruce,

Happy Birthday and congratulations on being fifty five years young. I hope that you have many more to come.

AlJanuary 15, 2019 12:10 PM

"Could Moderating Your Website Invalidate Your “Safe Harbor”? "
http://copyright.nova.edu/safe-harbor/
Yep.

And I don't think it's just safe harbor for copyright infringement, but could include things like libel. In a way, fake news is all right, because, where did you copy it from? 😜

I think this area is going to lead to a balkanization of the internet. Under U.S. law, government doesn't regulate speech. Other countries want to regulate speech.

In recent news, it is old people who disproportionately pass along fake news.

WaelJanuary 15, 2019 12:12 PM

@Bruce,

Happy Birthday and congratulations on being fifty five years young.

Ditto!

@Clive Robinson,

Happy birthday to you
Happy ___ dear ____

Start the Security related lyrics. I'm having a mental block!

MikeAJanuary 15, 2019 1:02 PM

@Al -- two things:

1) Is this the LiveJournal that is/was an early social network run by vkontakt?

2) How does "where did you copy it from" come in? Years of media sites blithely switching ad revenue from Indy bands playing their own compositions to shady "publishing companies" who claimed copyright violations should have made clear that copyright is right up there with "She's a witch!" from a neighboring landowner looking to expand. Plus there's the whole "Back date an article stolen from some news outlet, post on a newly created domain, and get the actual article taken down (because copyright)" now being used to removed "unfortunate" coverage of corruption.

bigmacbearJanuary 15, 2019 2:21 PM

@MikeA 1) LiveJournal started in Seattle, moved to San Francisco, and was then purchased by a Russian firm who were more or less competitive with VKontakte.

I left LJ when the Russian government stuck their nose in, required the servers be physically moved to Russia, and made them subject to Russian law instead of US by forcing agreement to a new TOS document.

Clive RobinsonJanuary 15, 2019 3:03 PM

@ Wael,

Start the Security related lyrics. I'm having a mental block!

It would be so much easier if it was the rack mount server with a birthday rather than the owner/operator.

You could start with a simple,

    Happy bootday 2U

The problem with the song is in the way it's sung in the UK it's just two lines,

1) Happy birthday to you.
2) Happy birthday dear Xxxx

Where Xxxx is the persons prefered social name. That said the first line is sung twice befor the second then once after. So around two thirds (52:24) is just two words repeated... So not much room for a little fun.

No wonder you bowling ball feels like it's stuffed with something less usefull than a bucket of wet concrete ;-)

WaelJanuary 15, 2019 3:29 PM

@Clive Robinson,

No wonder you bowling ball feels like it's stuffed with something less usefull than a bucket of wet concrete ;-)

Then lend me your brain (and walk barefooted.) lol ;)

Ross SniderJanuary 15, 2019 3:41 PM

From my perspective, this has already happened.

Facebook has politically censored content I've posted numerous times (e.g. Snowden documents). Twitter and Facebook have recently given way to Congressional requirements to turn them into perception and content management systems for the US. Facebook even worked with the US military to study how to manipulate foreign elections. That - and the promise of these platforms to advertisers (including political advertisers) is that they will be able to "drive engagement with targeted demographic subgroups". Facebook was originally _funded_ by CIA.

The mundane dystopia Alex Stamos describes has already dawned for us.

WaelJanuary 15, 2019 3:41 PM

@Clive Robinson,

It would be so much easier if it was the rack mount server with a birthday rather than the owner/operator.

True! Skipping this one -- too difficult and dangerous.

Explanation of the exchange between Clive Robinson and I:

He replied to my:

I'm having a mental block!

by saying:

No wonder you bowling ball feels like it's stuffed with something less usefull than a bucket of wet concrete ;-)

He is saying my skull (which I called a bowling ball earlier) is stuffed with something less useful than wet concrete (block.) He's calling me a blockhead, in a way. lol

I replied with:

Then lend me your brain (and walk barefooted.)

I'm calling his brain a shoe (jokingly, of course.)

That's how we talk. Good luck to LE deciphering what we say. Huge room for plausible deniability. Encryption, my foot. When we talk poems and Shakespeare, none of the top tear spook agencies will have a clue.

Clive RobinsonJanuary 15, 2019 3:54 PM

@ Wael,

Then lend me your brain (and walk barefooted.)

But are you going to walk the mile?

Or go crazy and damage my sole.

AlJanuary 15, 2019 4:12 PM

@MikeA

(1) Whatever the site was, was the first site that showed up to make my point. Here's a site with another aspect.
https://en.wikipedia.org/wiki/Section_230_of_the_Communications_Decency_Act
" The act was passed in part in reaction to the 1995 New York state court decision Stratton Oakmont, Inc. v. Prodigy Services Co.,[3] which suggested that service providers who assumed an editorial role with regard to customer content, thus became publishers, and legally responsible for libel and other torts committed by customers."
This perhaps more directly addresses the point, which is, moderation, particularly "before publishing" moderation creates a liability that, say, taking down a post after being flagged by a poster would not.

(2.) In addition to liability for things like libel, there are issue with copyright, to the extent something is pasted from someone else content that exceeds "fair use". Sometimes I see this issue from posters on this blog.
https://en.wikipedia.org/wiki/Online_Copyright_Infringement_Liability_Limitation_Act
My point about fake news is, it might be made up on the spot, and hence, original content as opposed to being copied. Of course, fake news could also be copied from a publisher in the fake news generation business. In that case, "fair use" would still apply.

I'm not going to research this exhaustively. But, I would check out liability if I was to engage in moderation.

jonesJanuary 15, 2019 6:23 PM

I'm more concerned that the elimination of net neutrality will create content liability, which will lead to widespread corporate censorship and legal abuse, as with the DMCA.

Nothing corporations control is under democratic control: that's the existential threat posed by the privatization of public services. At work, you don't get to vote on who your boss is, or what the dress code is, or what your hourly rate is, etc.

Net neutrality is "common carrier" status in telecommunications. "Common carrier" status is why an individual postal worker is not personally liable if he or she unknowingly delivers a mailbomb that later explodes.

The EFF has a great write-up on different ways the DMCA went sideways:

https://www.eff.org/wp/unintended-consequences-16-years-under-dmca

Lord TalksalotJanuary 15, 2019 8:36 PM

"Proving not only can a turd be polished, it can also polish it's self when it suits..."

Kind of like your assertions about the bitcoin theft motive, what with attribution being "so very difficult" and all.

Phoning it in.

IggyJanuary 16, 2019 9:34 AM

Raise the minimum age limit in all the TOS from 13 to 18, the age of majority in the US, and watch a lot of the demand for moderation fall away. Of course, cheating will continue, but most parents will enforce it. Mature adults know to ignore or discount spurious and outlandish comments scrawled on the internet fence. Children are easily manipulated by ads and graffiti. Sadly, we have far too many humans with rights and responsibilities stuck in a persistent state of neoteny, and they rival sub-adults in refusing to learn and use critical thinking.

And:

Hippo Birdie Two Ewe
Hippo Birdie Two Ewe
Hippo Birdie Deer Bruce
Hippo Birdie Two Ewe

MikeAJanuary 16, 2019 10:20 AM

@Iggy I'm not so sure raising the age will help. Wasn't there a recent study that found over-60's (present company excepted, of course :-) are the most likely to forward/re-tweet/upvote/whatever fake news?

Maybe it will be like driving; over some age limit and involved with more than some arbitrary count of violations, and "It's the bus for you, Gramps"

Hm, being past the probable age limit and a bit of a jerk on UseNet back in the day, I might not want TPTB to hear that...

Impossibly StupidJanuary 16, 2019 10:42 AM

@jones

Nothing corporations control is under democratic control: that's the existential threat posed by the privatization of public services. At work, you don't get to vote on who your boss is, or what the dress code is, or what your hourly rate is, etc.

Rubbish. If you don't like a work environment, you are free to simply leave, or to never take the job in the first place, or not follow the rules you think are silly. Under the control of a government, though, democratic or otherwise, those rules are called laws, and there can be pretty serious freedom-depriving consequences for disobeying them. And most governments are pretty steadfast in asserting ownership of citizens and their labors. Just look at the USA: "democratic control" resulted in the election of a President that is allowed to run roughshod for (at least) 4 years, and the people who don't like it can't just decide to follow a different President or pick up citizenship in another country. The simple fact is that jobs are more democratic than governments, but another simple fact is that most people choose not to exercise their democratic powers.

That's the problem here. Alex chose to work at Facebook in 2015. Everyone with a brain already knew they were deeply toxic by then; the "then something happened" was not by accident. It would be incredibly naive on his part if he thought he could change any of that from a CSO position. He doesn't get to pretend that he's looking to stand on some moral high ground just because he eventually left. He isn't offering any insights when he says Zuckerberg (and other sociopathic CEOs) will do bad things if politicians decided to do X, because Zuckerberg is always going to continue to do bad things no matter what X, Y, or Z thing gets decided.

AlJanuary 16, 2019 12:09 PM

Google declines to moderate search results for Russia.
https://www.theinquirer.net/inquirer/news/3069441/google-faces-ban-in-russia-after-ignoring-online-censorship-demands
"GOOGLE HAS SHRUGGED OFF requests from Russia's Roskomnadzor to censor its search engine in accordance with local laws ... A law passed in the country last year requires search engines to be connected to the federal state information system (FGIS) that allows the Kremlin to filter search results"

This is an example of why I think the internet will become increasingly balkanized, because one internet isn't going to be able to comply with multiple jurisdictions with conflicting laws and agendas.

IggyJanuary 16, 2019 2:28 PM

@MikeA, I heard about the study, out of the NYT - a bastion of scrupulous accuracy (remember this? https://www.nytimes.com/2017/07/04/world/asia/north-korea-missile-test-icbm.html), complete dedication to being fraud free (Jayson Blair) and a leader in championing the defeat of racism (oh wait, but not against whites: Sarah Jeong). You'll forgive me if I pass on that study.

But for argument's sake, let say it's true, that over 60 year olds are more likely to forward/RT fake news than anyone else, even everyone under the age of 17.

Do you think someone who's been on the planet for 60 years is too stupid or too wiley?

RealFakeNewsJanuary 16, 2019 11:08 PM

The reason Google refuses absolutely to censor results in Russia is two-fold:

1) It assists the foreign policy goals of the USA/West in destabilization of Russia

2) It makes Google appear not to censor in the USA/Western societies, which they absolutely do at the direction of various Governments.

The problem with propaganda is realizing that you're the subject of it.

Clive RobinsonJanuary 17, 2019 12:43 AM

@ RealFakeNews,

Google refuses absolutely to censor results in Russia is two-fold

Whilst the second is an internal choice for Google and their owners there could be a lot more behind the first.

If you remember back some time ago Google were trying to setup in China and ran into all sorts of problems which included from what leaked out some of the localy employed staff not acting in their employers interests... Also there was the issue with the Dali Lama and others from Tibet having their electronic mail plundered. We now have reason to think it started in 2002 but this article from nine years ago indicates a small part of what was going on,

https://www.telegraph.co.uk/news/worldnews/asia/china/6996906/Chinese-human-rights-activists-claim-their-Google-emails-were-hacked.html

But Google are still "apparently" salivating at the Chinese market with their project "Dragonfly". Then just over a month ago we got this,

https://tibetnetwork.org/tibet-uyghur-activists-urge-google-drop-dragonfly/

With Google also pursuing military type government contracts a whole bunch of questions get thrown up. Google apparently can not keep the Chinese out of their customer facing systems but potentially their development systems as well.

Many have considered the Chinese publicaly to be "inept" at hacking, whilst also publicly considering Russia to be better[1]. Reality is obviously different which indicates "The dead hand of politics" is very much in play.

I'm guessing politics is very much behind Google and their owners decision not to play the Russian Way.

However give Russia their due, they do tend to get the horse before the cart, and put legislation in place first before taking what is then "lawfull action".

At the end of the day it is upto the Russian government and those that vote for them who get to decide what goes on in their Sovereign Nation. We may not like it but then, have we done any differently in the West?

[1] My view is as I've said any nation who can will be at it one way or another. Thus not only would you consider a range of skills to be normal around the world you would expect it in individual nations as well. Both China and Russia have co-runing power structures based on "favours". The center rarely says exactly what it wants but will accept gifts from those who are trying to gain favour, if it's what is wanted and those doing it don't cause embarrassment then not only do the favours improve for them so does their political capital. Further I find it odd that the US "existential threat" in Cyber-Space is almost always just one at a time from the usuall group of four "China, Iran, North Korea, Russia". It's also noticable that the companies involved with seaking out Cyber-Threats have a history of reflecting the US Gov choice current existential threat... People should be asking why we rarerly if ever hear of any other nation doing State Sponsored Cyber-Attacks, it's only common sense to realise that various other Wester Nations as a mininum must be at it and have the US in their target list...

Leave a comment

Allowed HTML: <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre>

Photo of Bruce Schneier by Per Ervland.

Schneier on Security is a personal website. Opinions expressed are not necessarily those of IBM Resilient.