The Security Risks of Unregulated Google Search

Someday I need to write an essay on the security risks of secret algorithms that become part of our infrastructure. This paper gives one example of that. Could Google tip an election by manipulating what comes up from search results on the candidates?

The study’s participants, selected to resemble the US voting population, viewed the results for two candidates on a mock search engine called Kadoodle. By front-loading Kadoodle’s results with articles favoring one of the candidates, Epstein shifted enough of his participants’ voter preferences toward the favored candidate to simulate the swing of a close election. But here’s the kicker: in one round of the study, Epstein configured Kadoodle so that it hid the manipulation from 100 percent of the participants.

Turns out that it could. And, it wouldn’t even be illegal for Google to do it.

The author thinks that government regulation is the only reasonable solution.

Epstein believes that the mere existence of the power to fix election outcomes, wielded or not, is a threat to democracy, and he asserts that search engines should be regulated accordingly. But regulatory analogies for a many-armed, ever-shifting company like Google are tough to pin down. For those who see search results as a mere passive relaying of information, like a library index or a phone book, there is precedent for regulation. In the past, phone books—with a monopoly on the flow of certain information to the public—were prevented from not listing businesses even when paid to do so. In the 1990s, similar reasoning led to the “must carry” rule, which required cable companies to carry certain channels to communities where they were the only providers of those channels.

As I said, I need to write an essay on the broader issue.

Tags: , , ,

Posted on June 4, 2013 at 6:19 AM • 54 Comments

Comments

Steve Wildstrom • June 4, 2013 7:14 AM

Is there a link to the paper itself?

Was this a real election or a contrived one? In a real election, it is very unlikely that the only information voters would have on the candidates is what turns up in a search engine, so the effect would be much smaller if it existed at all.

In any event, government regulation of search algorithms sounds like a cure much worse than the disease.

Someone • June 4, 2013 7:16 AM

Google keeps people in a search bubble of sorts. Conservatives will get results favoring conservative candidates, liberals get results favoring liberal candidates, libertarians get results favoring libertarian candidates, and so on.

I admit, what Google could do with the resources and position they have does concern me. I am not particularly worried about it currently, but Google will not always be run by the same people.

Matt Drew • June 4, 2013 7:21 AM

That’s ridiculous. The search engine is not fixing the election, it is influencing it – through First Amendment protected speech! No one is forcing people to use the search engine in the real world, and there are numerous other sources of information for comparison, literally a click of the mouse away.

You could make his same argument for regulation versus TV news stations that publicly claim objectivity but have a covert agenda, or book authors who write books with a particular political slant. In fact, this very argument has been made recently, and enforced – in Venezuela, as the country slides into totalitarianism and shuts down any media outlet that is critical of the government.

Steve Wildstrom • June 4, 2013 7:22 AM

Found the original paper after some searching. It seems subjects in San Diego were given rigged results for an Australian election. Since the subjects, unlike voters in a real elections, likely had no information on either the candidates or the issues before seeing the rigged search results, it is hardly surprising the the search swayed their views. But would this happen in a real election where voters have many other sources of information?

DanCarey_404 • June 4, 2013 7:34 AM

I agree with Matt Drew’s comment, and would extend it to say that Google’s position to influence an election is no different than that of a publisher in a one-newspaper town (and there are

    lots

of those). Such publishers enjoy 1st Amendment protections, and so should any search engine or social media company.

Really? • June 4, 2013 7:36 AM

Providing transparency seems reckless. If more people understand the way the search ranks are generated, the more people can manipulate the outcome. Google and others must constantly alter the algorithm to prevent the gaming of the system. Regulating predictability and slow changes into the system seems to invite more rampant abuse.

paranoia destroys ya • June 4, 2013 7:51 AM

@ Steve – If a search engine was used to find the original paper, how do we know that the search engine directed us to the right one and is not trying to influence our opinion?

chuckb • June 4, 2013 7:55 AM

Google has no monopoly on search. Anyone with access to the web has access to other search engines.

Chris • June 4, 2013 8:22 AM

Even if there are sources of information out there, other search engines available, and everybody can choose to listen to whatever source they prefer, the study still present a valid point.
It doesn’t matter if you choose to look for the right information on a variety of sources, what matters is what the majority of people do – in today’s Internet, it is searching on Google.

As a comparison, try watching nothing but the insert TV news channel with views opposing your own for two weeks then imagine what would happen if most people would willingly watch that on a regular basis in your voting district. How much will it matter if you get your information from a balanced diet of different sources?

The free market argument doesn’t hold when defending democracy. Democracy doesn’t depend on a free market. The free market depends on democracy.

Alan • June 4, 2013 8:24 AM

Cable companies were granted the right to crawl through our cities digging trenches and stringing wire. They were often granted local monopolies or duopolies. In exchange, they had to accept certain regulations, including providing 100% service coverage in the territory, price controls, channel carry requirement, and other provisions. This sort of arrangement has no comparison to Google.

bcs • June 4, 2013 8:31 AM

This is an unavoidable consequence of the universal franchise. Some of the voters have no informed opinion and as such it is trivial to sway them by manipulation of “irrelevant” aspects of their environment.

I’m not sure anything can (or should) be done about that.

paul • June 4, 2013 8:37 AM

The first amendment situation, I think, is not entirely clear. It’s well established that search results have monetary value: see both “sponsored” results and the controversy over whether Google and other search engines preferentially return results for their owners’ other properties.

So although a search engine is perfectly free to announce its support for a particular candidate, preferentially returning stories favorable to that candidate could well be regarded as an in-kind contribution to their campaign, which would be unlawful per se, or an uncoordinated campaign expenditure, which would have to be valued and disclosed.

It’s not clear what kind of regulation you could impose that would actually work. But some kind of heightened transparency does seem in order.

Obi-Wan Kenobi • June 4, 2013 8:54 AM

“The author thinks that government regulation is the only reasonable solution.”

I felt a great disturbance in the Force, as if millions of libertarians suddenly cried out in terror.

Chelloveck • June 4, 2013 9:06 AM

Another agreement with Matt Drew. I think this is no different from having a trusted “neutral” news source turn out to have a hidden agenda. I know of several news sources which I consider politically neutral, but which others consider strongly biased. And I know of many more news sources which I consider biased despite their claiming to be neutral. This is no different.

There’s no way to force complete neutrality in reporting or in a search engine. I’m not even sure it’s possible to define. You just have to accept that yes, there are biases. Sometimes intentional, sometimes not. If you think a source is biased, by all means warn others about it. But that’s the best that can be done.

johnp271 • June 4, 2013 9:07 AM

Obi-Wan: you are right, I found this “only reasonable solution” to be abhorrent. Who doubts that the regulations would be tuned to insure searches favor incumbents, one party’s darlings (the party in power) at the expense of the other’s? The real solution would be for people to think for themselves, and/or for a free and open society to expose such biases and offer alternatives.

vas pup • June 4, 2013 9:14 AM

“Epstein believes that the mere existence of the power to fix election outcomes, wielded or not, is a threat to democracy”, not threat to democracy, but to republican form of representative Government which is not equal to democracy.
Republic does not guarantee real democracy, i.e. representation of interests of majority population AFTER election till next election, in particular when there is close election. It just provide mechanism of transition of power based on criteria other then blood relation.
Is good King/Queen better than elected bad President?
Democracy is about political regime regradless of formation of highest authority: freedoms, human rights, including protecting rights of minority; law-guided state, privacy, etc.
Conclusion: republic <> democracy by default, it may or may not become a tool for democracy. It depends whom serve elected.

Steve • June 4, 2013 9:14 AM

Google and others already influence elections in disproportionate ways through unmonitored and unregulated campaign contributions, thanks to the Citizens United decision which equates money with speech.

While they didn’t do so well in the last election (Romney lost — sort of), they will have learned their lessons and will be better at it next time, the time after that, and the time after that, ad infinitum.

I suspect our form of government will evolve toward an overt corporatocracy of some sort, where national and regional loyalties eventually fade away and corporations hold political power.

Since corporations have been held to have civil rights, I envision them someday having direct representation in the legislatures.

And I now yield the balance of my time to the Senator from Google.

Michiel • June 4, 2013 9:18 AM

Agree with Matt Drew.

Google is a medium just like any other.
All media influence elections, deliberately or not. It used to be the papers, then it was tv, now it’s The Internet. Nothing wrong with that.

The only problem here is that a lot, and I mean a lot, of people have no clue that Google is just one of many commercial companies providing internet search. To many, Google is The Internet. It’s an education problem.

Alex • June 4, 2013 9:19 AM

I’m not sure this would apply in a real election. I don’t think there is enough of a “swing” vote. Most people are staunch supporters of their party and would dismiss information contrary to their belief. Same way you don’t see mass changes in religious beliefs. Perhaps over time (i.e. decades) people will change votes but I’ve only seen that when it hits people in their pocketbooks (i.e. lost their job and health insurance with pre-existing conditions and finally realized the Republican party wasn’t looking out for them)

Anon • June 4, 2013 9:24 AM

“Access to” is not the same as “Uses”.

When you break it down Most people use Google to search either directly or through some sort of “powered by Google” proxy.

Google through it’s page ranking system has a ton of influence on what people ‘know’

And knowing is half the battle…

More... • June 4, 2013 9:55 AM

Google is just a news aggregator in this context – the provider of a curated list.

A curated (and potentially exclusionary) list of associated articles compiled by humans = free speech and is found on millions of blogs.

A curated (and potentially exclusionary) list of stories or topics compiled and presented by humans = free speech and is found on countless TV stations and in newspapers.

A curated (and potentially exclusionary) list of articles compiled by machine is somehow so vastly different that it requires government regulation??? I don’t see it.

Anon Techie • June 4, 2013 10:38 AM

Why do some people think that the answer to any problem is more regulation or government intervention ?? Reminds me of an old saying – For a person who only has a hammer, every problem looks like a nail !!

Nick P • June 4, 2013 10:58 AM

@ oldie

“https://addons.mozilla.org/en-us/firefox/addon/google-search-link-fix/”

And it works! Thanks for the addon! I was having to do some annoying tricks to get the right result, esp for pdf’s and such.

nonegiven • June 4, 2013 11:00 AM

I don’t know of any news source I consider neutral. I have to assume they are all tainted.

Hugh No • June 4, 2013 11:37 AM

Does the author think that Fox “News” should be regulated as well? Of course, they’re fair and balanced.

Johnston • June 4, 2013 11:54 AM

Government regulation would favor whoever is currently in power.

Use and advocate DuckDuckGo or IxQuick as alternatives to google search. Promote decentralization. Centralized power is what enables corruption and abuse.

Bo Vine • June 4, 2013 12:24 PM

“@ Steve – If a search engine was used to find the original paper, how do we know that the search engine directed us to the right one and is not trying to influence our opinion?”

  • by switching on our ‘bullshit’ filters.

tz • June 4, 2013 1:39 PM

If people are so shallow to be very easily manipulated, the problem is with them, not the search results. They already are going to watch either MSNBC or FoxNews to get their own tunnel vision talking points. Ought we bring back “The Fairness Doctrine” on steroids? I would rather first regulate all these infotanment entities before imposing something like that on the search engines.

The Press also is VERY selective on which stories they cover, and how they cover them (e.g. the very sensational Kermit Gosnel trial because he performed abortions instead of any of the less sensational trials).

There is no cure when people refuse to think and question and check the sources.

The problem is we want to give the responsibility for vetting whom we trust to a ministry of truth – and worse, instead of labeling them heretics and dissenters, the ministry should torture, censor, imprison, and kill.

It is not that truth is impossible to determine, it is that “the certificate chain” is too long so is easily compromised, or really honest people – those with integrity – are saying things we don’t want to accept as truth (compare Ron Paul’s words about economics and policies in the campaign, how he always votes the constitution, and hasn’t changed position – he’s the most honest but lost).

Those who don’t want honor and integrity will find someone who promises to lie, cheat, and steal, but do it to “those other people”. But everyone is “those other people” to someone.

So we need honor, integrity, prudence, temperance, fortitude, justice, patience, and reason – the old virtues. But the cost is too high and it is too hard – until the current system crashes.

chuckb • June 4, 2013 1:45 PM

“Could Google results tip an election? Just ask Rick Santorum.”

Uh huh, also the Gov of Texas? Michelle Bachmann? Newt Gingrich? Mitt Romney?

What do they share in common?

Dirk Praet • June 4, 2013 5:52 PM

Google has been slapped on the hand before over alleged “search bias”, as in the case of the “Rival Links” Antitrust Settlement with the EU. Microsoft has equally had its fair share of run-ins with EU watchdogs over abuse of its dominant position in the market.

In this particular case, I don’t think additional regulation, but more transparancy on behalf of Google is the solution. I agree with most commenters that in the end Google is just a medium as any other, with the power to influence its audience just like NewsCorp and many others do on a daily basis. There is however a subtle difference between manipulating and distorting information on one hand, and rigging an election on the other.

That said, my personal advice to the Google Board of Directors would be to stay far away from any such hypothetical scenarios. The world is bigger than the US alone and any real or perceived abuse of power, monopoly and the like can have far-reaching consequences unless you’re in the financial sector. Even Rupert Murdoch knows that by now.

@ oldie

“https://addons.mozilla.org/en-us/firefox/addon/google-search-link-fix/”

Thanks for that!

Simon • June 4, 2013 6:34 PM

One word – Santorum. Mind you, he had it coming. I see the top result for him is now a discussion of the neologism, rather than the neologism itself. I guess that’s just the natural evolution of the usage of his name on the web, and not a deliberate manipulation of results?

Julien Couvreur • June 5, 2013 2:42 AM

This is silly.

First, search engines may have biases, just like newspapers. So this is not necessarily bad.

Second, if Google tried to subvertly change its results, it would likely get detected (results different than competitors, or whistleblowers) and it would be a big PR hit. Bye bye customers, bye bye advertisers, hello lawsuits.

Competition and regulation keep companies in check better than voting keeps government in check.

Bruce Schneier • June 5, 2013 9:58 AM

“Second, if Google tried to subvertly change its results, it would likely get detected (results different than competitors, or whistleblowers) and it would be a big PR hit. Bye bye customers, bye bye advertisers, hello lawsuits.”

I wish this were an adequate security mechanism in all cases, but again and again we see evidence that it isn’t.

“Competition and regulation keep companies in check better than voting keeps government in check.”

I’m not sure that’s the correct comparison. The question is what is the better vehicle for society to keep corporations in check. There is the market mechanism, which works for some things. There is the government regulatory mechanism, which works for other things. I don’t know which one will work better in this case — and by that I mean the general case of private algorithms that control our lives — but I’m not optimistic about market mechanisms.

Yes, I understand that there’s a knee-jerk Libertarian markets-good governments-bad instinct, but I think it’s much more complicated than that.

Bruce Schneier • June 5, 2013 10:09 AM

“Providing transparency seems reckless. If more people understand the way the search ranks are generated, the more people can manipulate the outcome. Google and others must constantly alter the algorithm to prevent the gaming of the system. Regulating predictability and slow changes into the system seems to invite more rampant abuse.”

This is an important consideration, not just in this example but in the general case as well. There are many examples of algorithms that need to be verified in some way, but whose public disclosure would be detrimental. Google’s search algorithms need to be kept secret to prevent people from gaming them, and also because it’s a company trade secret. The obvious way to solve this is to have a trusted team of verifiers act as a proxy for society. There might also be some clever technical solution that we haven’t thought of yet. It’s an interesting research question.

Bruce Schneier • June 5, 2013 10:16 AM

“The first amendment situation, I think, is not entirely clear. It’s well established that search results have monetary value: see both “sponsored” results and the controversy over whether Google and other search engines preferentially return results for their owners’ other properties. So although a search engine is perfectly free to announce its support for a particular candidate, preferentially returning stories favorable to that candidate could well be regarded as an in-kind contribution to their campaign, which would be unlawful per se, or an uncoordinated campaign expenditure, which would have to be valued and disclosed. It’s not clear what kind of regulation you could impose that would actually work. But some kind of heightened transparency does seem in order.”

I agree that the issues are not at all clear. I also think that the default “markets will magically fix everything” solution isn’t always going to work.

I expect that this will become a real issue over the next years, as more algorithms start controlling more of our lives. Maybe it’s worth making a list. Credit scoring algorithms. TSA algorithms determining what sort of airport security we are subjected to. Google search result algorithms. The algorithms that determine what prices we’re shown on retailer websites. Heath insurance premium algorithms. A lot of these have been around for a long time, and there has been minimal — if any — public outcry. I believe that will change.

Bruce Schneier • June 5, 2013 10:19 AM

“If people are so shallow to be very easily manipulated, the problem is with them, not the search results.”

This is the same sort of blame-the-user thinking that plagues so much of security. The “problem” is almost always with the user; that doesn’t mean that we as security people get to ignore it.

Carpe • June 5, 2013 10:40 AM

The main issue here is the “filter bubble”. For example, I have been assigned an SEO task even though I haven’t done SEO since the early 2000’s. I tried to explain some of the latest Penguin and Panda updates Google was applying to the boss, and how one previous method we were using is no longer valid and if anything would hurt our SEO. He insisted “it works for me”, typed in a key search in Google (chrome) and beamed when our sites dominated the results page. Typing the same results into duckduckgo, or in a tor browser showed our sites ranking off the page!

The problem is that google et al are tailoring results, which has very complicated effects on the consumption and processing of information by people. The other problem is that people are not aware of the filter bubble, and feel they are getting delivered the same news… they are stuck imaging news in the “newspaper edition” style, where everyone sees the same thing.

Beyond that, after doing much research to catch up to modern SEO after being out of it for so long, I have come to the conclusion that despite their adamant denial, Google constantly manipulates search results in order to do one thing and one thing only. Maximize profits. Not to deliver information (they are touting that content is king, but to them content is only king because of the associated profits!)

I highly suggest the use of engines like duckduckgo and tor.

askme • June 5, 2013 11:43 AM

3 posts by Bruce on a single post? Must be important.

Surprising because the “paper” is a hit piece by a nutjob mad at Google (see Sean’s link above, and then read the comments).

The “secret” algorithms topic is an interesting one however. working in Financial Services, there are core algorithms all over the place that not only affect people’s lives, but drive whole global economic shifts. Trillions of dollars are moved around to hedge or profit from perceived discrepancies between algorithmic models and the market daily.

Me • June 5, 2013 1:01 PM

I am unsure that having the government controlling the information is much better than letting a corporation do so. I think the fact that there are multiple corps trying to woo eye-balls/ears puts them as better guardians of the truth than the single government (which is supposed to be modified at the whim of the people, not modifying the whim of the people).

Neither is ideal.

Bruce Schneier • June 5, 2013 6:40 PM

“3 posts by Bruce on a single post? Must be important.

“Surprising because the ‘paper’ is a hit piece by a nutjob mad at Google (see Sean’s link above, and then read the comments).”

It’s less the specific paper and more the general issue of secret algorithms increasingly controlling our lives.

Related questions: How much public scrutiny should we require — none is a reasonable answer, if you can justify it — of the software used to control self-driving cars on public roads? How about the software in breathalyzers whose results are used as evidence in court? Could we make the IRS more fair if we required them to publish the algorithms used to decide who to audit? Could we achieve the same level of fairness with some other audit mechanism that does not result in the algorithms being made public?

Figureitout • June 5, 2013 7:33 PM

Bruce
–Simply asking what type of radar (I assume officers of the law get proper training and knowledge on the tools they use, ha), let alone algorithm guts, an officer was using during a speed ticket got the response similar to “Shove it and ask in court”. There’s always going to be the question of “is this just what’s being fed to the public..?” So something like a Brian Krebs is needed.

“3 posts by Bruce..”
–Wonder what algorithm he’s using…

Joe Buck • June 5, 2013 8:56 PM

Publishing the IRS’s audit algorithm would just result in a brisk business for teaching rich people and corporations how to game their tax returns to pay almost no taxes without being audited, shifting even more of the tax burden to people whose income just comes from a paycheck and have no room to cheat.

Clive Robinson • June 6, 2013 4:13 AM

@ Bruce,

I’ve mentioned in the past that you should include the UK Satirical magazine “Private Eye” in your reading list….

If you look inside the current issue you will find that as far as Google is concerned their relationship with both of the UK Major Political Parties (Conservatives, Labour) is very intimate including senior members being relatives by both blood and personal partnerships (ie living together be it “in sin”, with “civil” recognition or that of one or more Gods).

There is also appears to be a “revolving door” policy where senior “party officials” have found senior managment positions with Google and the other way.

Part of the reason Private Eye have been looking at Google is their (probably) criminal tax avoidance that they appear to lead a “charmed life” over this with regards the UK Revenue service turning a blind eye and political influance been excerted over the Civil Servants working their and at the Audit Office. Also Google’s overly close relationship with various News Paper Organisations and the fact that they are a little more fleet of foot than the likes of the Murdoch Empire.

In short it would be amazing if Google were not distorting their search results for political reasons.

Tyler Larson • June 7, 2013 12:03 AM

To be clear: search results, like every other importance scoring mechanism, represents an OPINION. In this case, it’s the opinion of the Google search quality team expressed as an ever-changing algorithm. But it’s still an opinion subject to bias, error, and manipulation. If the algorithm were public or regulated, the problem would still exist. The realproblem isn’t secrecy or regulation, it’s trust. Too much trust.The real problem is that we all unflagginly trust the same opinion.

If news media were as trustworthy as search engines, they too would pose a significant risk to society.

As it is, the frequent errors, misreporting, and flagrant bias that we are used to seeing helps us maintain a healthy distrust of the news. The almost comical opposition of Fox News to CNN helps keep present in our minds the fact that news reporting need not be accurate, sensible, or unbiased. No matter what you believe, you will find a source to disagree with.

What we need with search engines isn’t greater control, greater transparency, greater accuracy. What we need is more dissent, more inaccuracy, more obvious bias. We need a reason to not trust search engines the way we don’t trust news reporting.

anon • June 12, 2013 2:35 AM

To those arguing that this is no different from a newspaper influencing voting behaviour: That’s why in some juristictions the newspaper market is regulated to ensure not only competetiveness but also plurality (i.e. to ensure that a single company doesn’t dominate the newspaper market).

To those claming that this study should be disregarded because the author (supposedly) has an axe to grind with Google: Can you point to any flaws in the original paper (summary)? With scientific studies, the whole point is that the methodology should be transparent and the results reproducible, such that one doesn’t have to trust the author.

To those saying that we would notice if Google were to manipulate the search results for political ends: The study suggests otherwise:

In a third experiment, a more aggressive mask was used to hide the manipulation, and no subjects showed any awareness of it, even though voter preferences still shifted in the predicted directions.

Julien Couvreur • June 16, 2013 11:49 AM

What is worse, Google’s behavior only checked by market incentives (mainly customers, reputation and competitors), or regulated by government with large discretion on when and what to enforce?

What has the most negative impact on political speech, Google’s potentially biased results (a hypothetical risk) or the IRS putting its audit power on selected groups (as it did the last few years)?

Leave a comment

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.