Schneier on Security: Data Privacy as a Prisoner's Dilemma

← Cryptography and Wiretapping ShareMeNot →

Data Privacy as a Prisoner's Dilemma

Companies would be better off if they all provided meaningful privacy protections for consumers, but privacy is a collective action problem for them: many companies would love to see the ecosystem fixed, but no one wants to put themselves at a competitive disadvantage by imposing unilateral limitations on what they can do with user data.

The solution -- and one endorsed by the essay -- is a comprehensive privacy law. That reduces the incentive to defect.

Tags: game theory, laws, privacy

Posted on July 28, 2011 at 6:27 AM • 22 Comments

Comments

Carlo Graziani • July 28, 2011 7:08 AM

Prisoner Dilemma is the wrong game-theoretic model here -- no additional defection reward, just compliance penalty.

This is The Tragedy Of The Commons.

Dr. I. Needtob Athe • July 28, 2011 7:36 AM

Going by Wikipedia definitions, Tragedy of the Commons doesn't really fit because it refers to depletion of shared limited resources.

Wikipedia says this about Prisoner's Dilemma:

"In casual usage, the label 'prisoner's dilemma' may be applied to situations not strictly matching the formal criteria of the classic or iterative games, for instance, those in which two entities could gain important benefits from cooperating or suffer from the failure to do so, but find it merely difficult or expensive, not necessarily impossible, to coordinate their activities to achieve cooperation."

http://en.wikipedia.org/wiki/Prisoner%27s_dilemma

http://en.wikipedia.org/wiki/Tragedy_of_the_commons

Clive Robinson • July 28, 2011 8:12 AM

I would very much like to see a comprehesive law on PII and other related data.

However ignoring for the moment "vested industry intrests", privacy is soething "we know when we see it, but cannot describe it".

Thus there is the danger of "The Law of Unintended Consequences" getting in and "spiking the gun".

Now I'm going to go out on a limb a bit here and mention SabOx...

From what I remember the US had a few monumentaly embarising and costly failings of control and audit, just one of which was Enron.

The legislators went cap in hand to the auditors and said in effect "here is blank paper put down every thing there should be including your wish list and well legislate for it".

The result was SabOx, which many will say turned out to be a real waste of effort as it achived nothing (not quite true but the sentiment is not misplaced).

So how do you stop similar happening with privacy legislation...?

S • July 28, 2011 8:42 AM

Clive:

"The result was SabOx, which many will say turned out to be a real waste of effort as it achived nothing (not quite true but the sentiment is not misplaced)"

LOL, I detect the bitter voice of experience there.

"So how do you stop similar happening with privacy legislation...?"

Do you consider the EU implementation to have similar problems, or are you purely speaking to the case of the US?

Personally, I've long found it pretty shocking that the US has effectively no data protection laws at all. More interesting factors come into play these days given the explosion of cloud-based services, certain provisions in the 'PATRIOT' ACT, and incompatibilities with other nations' laws.

Don • July 28, 2011 9:04 AM

I just took a cursory look at Rep. Rush's bill, and it's weak tea indeed. Data collection is opt-out, not opt-in, and they can make data collection a condition of service. You can sue companies only for "willful" violations and for a maximum of $1000, unless they "self-regulate" in which case they're immune. And they get to disclose "deidentified" information, which we know is in fact easy to connect right back to particular persons.

There's nothing in the bill that would give me the right to say "delete all the information in your files that pertains to me." There's no presumption under the law that the company is liable for data breaches. There's really nothing that would require a change in data-mining business as usual, except some paperwork.

Douglas Knight • July 28, 2011 9:11 AM

That article was short on detail. It looked to me mainly like wishful thinking. Yes, a few companies pushing for regulation is evidence that it's a prisoner's dilemma, but pretty weak evidence.

The cross-border issues are partly that the patriot act is anti-privacy and partly that the laws simply ban moving data across borders. Both parts are *caused by regulation*, the first US and the second EU.

Yes, consumers say that they want privacy guarantees, but apparently not enough to choose services with better privacy. Regulation would make them happier, but would effective regulation make them happier than empty regulation? Yes, privacy is complicated, making it hard to compete on the details, but companies could advertise "EU-strength privacy" (with patriotic exemptions, of course).

wiredog • July 28, 2011 9:53 AM

Privacy law, in the US, is going to have problems with the First Amendment.

ZG • July 28, 2011 11:28 AM

I am constantly surprised at how effective our (Canadian) laws are at creating cultural change in business both at a Federal (PIPEDA) and at a Provincial level. There is a definite shift towards privacy awareness in Canada that stems from legislative change.

John David Galt • July 28, 2011 12:24 PM

It's neither Prisoner's Dilemma nor the Tragedy of the Commons -- just the public good problem of economics, of which PD and TotC are also examples.

Richard Gadsden • July 28, 2011 3:02 PM

How much difference is the EU Data Protection Directive going to make?

If a US company is prosecuted in the EU (all the big ones have operations here) for storing data in the US without Safe Harbor, then would they pull out of the EU and refuse customers there, or will they comply with the DPD for all customers?

Is the DPD strong enough?

JimFive • July 28, 2011 3:21 PM

@wiredog: "Privacy law, in the US, is going to have problems with the First Amendment. "
Not as much as you'd think. If it becomes accepted that private data is "owned" by the subject of the data then it becomes pretty easy to say that a business relationship does not give a company the right to capitalize on something that they don't own. Exceptions would certainly be carved out for law enforcement, and the first amendment would be (rightly) invoked by journalists, but in the main it would be fairly straightforward to tell businesses that the fact that "Bruce bought The Communist Manifesto" is Bruce's secret to tell, not theirs. In the same way, we already tell health care providers that the fact that "Mary went to Dr. Bob" is not something that the provider is allowed to tell everyone.
--
JimFive

Dirty Davey • July 28, 2011 3:32 PM

Additionally -- even the greatest voluntary privacy policy can be changed, or a bankrupt company's customer data can get transfered to a company with a different policy. Only a legal protection gives you something more substantial than "trust us".

Don • July 28, 2011 3:35 PM

Dirty Davey, it's worse than "trust us," it's "trust us to do something that costs us money."

IA • July 28, 2011 4:37 PM

Maybe another interesting thought:

if date security really IS a prisoner's dilemma, being played by, let's call them "web companies", then an increasing concentration of web services in the hands of ever fewer companies (as it appears to be happening right now) could lead to a solution of the dilemma. Some small companies could keep violating consumer privacy, but the big ones like Google or Facebook could benefit from not doing so anymore. (At least according to Mancur Olson)

But, honestly, I'm not convinced that data security really qualifies as a collective good to these companies.

P-E • July 28, 2011 4:56 PM

I am slightly more optimistic insofar as I believe that some (many?) companies can use "Security as a means of being competitive" i.e. as a means of reducing churn/losing customers and as a means of gaining business which they otherwise would not have had a chance of aquiring. Privacy is part of this. Compare the automotive industry which today needs to make safe cars in order to be successful. Not all companies can do this, but those that do start using "Security as a means of being competitive" effectively have the chance of improving their bottom line substantially.

David Harmon • July 28, 2011 11:12 PM

YA problem with using security as a competition point is that consumers are Really Bad at evaluating security. Consider:

Company A collects the minimum amount of data, regularly purges credit-card and other "transaction" data, and has done so for years... but last month they got hacked, and the hackers got a list of E-mail addresses with age/gender information. They willingly disclosed the incident, and notified their customers, while the New York Times, WSJ, etc. notified everyone else in creation.

Company B only got started last month, but they promise that they're using their proprietary, UNCRACKABLE OleoSerpentes encryption system, which is why you shouldn't worry about their having your phone number, SSN, home address, and credit cards on permanent file.....

AC2 • July 29, 2011 12:25 AM

@David re

"while the New York Times, WSJ, etc. notified everyone else in creation"

Well my dog can't read.. yet...

And yes, the US legislature supposedly will act to pass and implement laws that benefit people, possibly at a disadvantage to companies.. Right. I'm sure everyone is familiar with what's going on with the financial consumer protection act?

averros • July 29, 2011 4:11 AM

"There must be a law" is practically always the wrong solution. Simply because beating and shooting people (and that's what *any* law implies, just to be "enforceable") is usually way worse than the problem the law is supposed to solve.

The real solution to the privacy problem is not to pass laws, but to eliminate them. Like, permit use of anonymous electronic payments, so people who'd wish to maintain their privacy could do so.

The reality is, though, most people simply don't care about their privacy. "Protecting" them is both wasteful and patronizing.

Richard Steven Hack • July 29, 2011 10:47 AM

JimFive: "If it becomes accepted that private data is "owned" by the subject of the data then it becomes pretty easy to say that a business relationship does not give a company the right to capitalize on something that they don't own. Exceptions would certainly be carved out for law enforcement, and the first amendment would be (rightly) invoked by journalists..."

And if I "own" some random data about myself, and the law says you can't "have" or "use" it, then I can invoke the law, civil or criminal, to shut down anyone who says anything "bad" - or that I randomly declare is "private" - about me.

Basically the same problem as intellectual property laws which are misapplied on a daily basis to shut down Web sites, critics, and a variety of other pointless harassments.

Averros is correct about one thing. On the one hand, we have people agitating to "get rid of anonymity" on the Net, and on the other we have people agitating to protect "privacy". You can't have it both ways - and passing laws for either is going to end up being worse than not, because laws invariably end up being used to oppress rather than "protect".

Personally, I couldn't care less about my "privacy". Being on the Internet is like being out in public - you have no privacy.

Must I add another meme?

"There is no privacy outside of your head and whatever four "walls" - technological or physical - you can put between yourself and others. Suck it up."

p-e • July 30, 2011 4:30 AM

@David Harmon, Granted, many customers are not good at understanding, nor for that matter evaluating security. The automotive industry is better off in this respect insofar as "look, my car just saved your life due to.../we got 5 stars in...". This does not mean however that it can not be done. It "just" takes persistent information/education... E.g. forget about the end consumers for now and concentrate on companies. Inform/educate the decisionmakers and help them. E.g. if a company wants to purchase x and you are the only one also informing them/helping them with security in the process. Who stands the better chance of impressing the customer in a positive way?

JimFive • August 1, 2011 10:14 AM

@Richard
re: "Being on the Internet is like being out in public - you have no privacy."
Except, of course, that you do have privacy in public places. Just not as much as you have locked in your basement. The idea that you have no privacy if you are not in your house is absurd. If I am walking down the street talking with my friend I have a reasonable expectation that we are not being recorded (assuming I don't live in London, I guess). Note: The expectation isn't that I *could not* be recorded, but that I *am not* being recorded.
--
JimFive

Dirk Praet • August 1, 2011 6:34 PM

Without a proper framework of do's and don'ts set in legislation/regulation, nothing happens. How these materialise (or don't) depends on politicians and lobbyists. The Belgian approach to the issue would be to actually tax collection of data by companies unable to present proper argumentation why they should be exempt. This way, politicians could avoid having to think through the issue while still being able to offer some sort of solution to the general public. Anybody questioning the proposal would be accused of intellectual nitpicking and refusing to compromise.

Photo of Bruce Schneier by Per Ervland.

Schneier on Security is a personal website. Opinions expressed are not necessarily those of Co3 Systems, Inc..

Data Privacy as a Prisoner's Dilemma

Comments

Leave a comment