Helen Nissenbaum on Regulating Data Collection and Use

NYU professor Helen Nissenbaum gave an excellent lecture at Brown University last month, where she rebutted those who think that we should not regulate data collection, only data use: something she calls “big data exceptionalism.” Basically, this is the idea that collecting the “haystack” isn’t the problem; it what is done with it that is. (I discuss this same topic in Data and Goliath, on pages 197-9.)

In her talk, she makes a very strong argument that the problem is one of domination. Contemporary political philosopher Philip Pettit has written extensively about a republican conception of liberty. He defines domination as the extent one person has the ability to interfere with the affairs of another.

Under this framework, the problem with wholesale data collection is not that it is used to curtail your freedom; the problem is that the collector has the power to curtail your freedom. Whether they use it or not, the fact that they have that power over us is itself a harm.

Tags: data collection, databases, power

Posted on April 20, 2016 at 6:27 AM • 51 Comments

Comments

Old Bull Lee • April 20, 2016 7:26 AM

But in the case of private entities, they only have that power if you give it to them. And most people are willing to give it to them in exchange for “free” services.

Dr. I. Needtob Athe • April 20, 2016 7:29 AM

That prospectmagazine link will spoil your day if you click it. It’s a trap. It lets you read a few paragraphs and then suddenly blocks your view with an irritating window that tries to make you to jump through some hoops before you’re allowed to continue.

Clive Robinson • April 20, 2016 7:56 AM

@ Bruce,

the problem is that the collector has the power to curtail your freedom. Whether they use it or not, the fact that they have that power over us is itself a harm.

I would disagree slightly and say the real harm is “the fact that we know they have that power over us”.

The reality of having collected the data does not mean that the collector can use it. They don’t have the resources to use it to curtail every ones freedom, at best just a few.

Thus people keep there heads below the parapet from the fear of knowing about the collection, not because of the collection. Thus they take a step or two back, hopping others are slower and thus appear in the spotlight and take the fall instead of them.

Thomas • April 20, 2016 8:07 AM

@Old Bull Lee

But in the case of private entities, they only have that power if you give it to them. And most people are willing to give it to them in exchange for “free” services.

Often times joining up is not voluntary.

I’ve lost count of the number of ‘free’ online services my kids have had to sign up to because they are part of the lesson plan at school. I’ve pushed back on a few, but you can only tilt at so many windmills before you get tired.

I’ve been pressured to join FaceBook because that’s what everyone uses to organise events. Met a job seeker lately who isn’t on LinkedIn? Which chat application do you use?

These private corporations have enormous power. With it should come responsibility.

B • April 20, 2016 8:41 AM

That surveillance and data collection give the snoops power over those being snooped on is something that has already played out in tangible ways in the USA. The NSA passes along information to law enforcement, which is used to catch people committing routine crimes (drugs, etc.). Once upon a time I used that as a hypothetical example of the danger of mass surveillance and data collection.

@clive — It is not necessarily the case that we are aware of the power data holders have over us, as we are often unaware of the collection. Think of the numerous, unregulated, largely unaccountable companies out there that do “background checks” for anyone willing to pay. Most people have no idea about the deployment of license plate scanners, cell site simulators, etc., but there is no question that such things have increased the power of police forces.

What really matters is how technology is changing power relationships. The pendulum swings both ways. Big organizations and traditional houses of power have gained the ability to collect mass amounts of data about individuals. The flip side is that individuals have never been more capable of communicating. The police have always beat up protesters, but now protesters can show videos of police abuse to the entire world. Smartphones have changed the balance of power between individuals and phone companies (various premium fees have become harder to push — tethering, for example — and when companies do misbehave people can use e.g. VPNs to fight back).

Unfortunately our politicians are stuck in the past and continue to think about the world as if it were the 1950s. The idea that individuals can render useless the copyrights, wiretaps, and various other regulations designed for a world of centralized services is alien to the most powerful people in America and Europe. Their frustration is obvious as they push for increasingly stronger versions of those laws, while failing to realize that their entire way of thinking is decades out of date. They do not even see a change in the balance of power, and they continue to pass laws that assume that society is structured exactly the same way it was structured back when they were starting their careers. If history is any lesson (i.e. the history of printing presses, which also upended the balance of power) we might be in for several more decades, maybe a century, of more painful fights before everything settles down and the new social order takes its final form (whatever that might be — again, the pendulum swings both ways!).

paul • April 20, 2016 9:44 AM

At this point, whether we know explicitly that a particular data collector/aggregator has the power to curtail our freedom is almost irrelevant. We all know that the data is out there, and that anyone who wants to spend the resources (time, money, skilz, connections) can get it and use it in unpleasant ways.

I would say, for what it’s worth, not just collection and use but also aggregation. Perhaps that really fits under one or the other of the first two, but that’s where a lot of the dangers get solidified. (But since aggregation and correlation are now relatively easy, it’s really collection and retention that do it.)

Impossibly Stupid • April 20, 2016 10:30 AM

@Old Bull Lee
“But in the case of private entities, they only have that power if you give it to them.”

Simply not true. Cameras are easily placed in ATMs or outside stores and homes, all surveilling the public without consent of any kind. The government has many ways to get at that data and aggregate it, but you do not. Who knows what the private entities themselves are doing with it.

albert • April 20, 2016 10:47 AM

@Dr. I. Needtob Athe,

Paywalls are one thing, but the BS you mentioned is quite another. ‘Annoying’ doesn’t even begin to cover it. Try running NoScipt sometime, and notice number of scripts needed to view the page correctly. Big media pages (TV networks and shows are the worst). I’ve counted 25 separate scripts on one site. I usually allow the sites own script, and perhaps others that I’m familiar with. After that, it’s ‘buh-bye’… If it’s really important, it’ll be somewhere else. If it’s MSM, it’s probably BS anyway.

@Thomas,
I feel your pain. I had an IT tech from a -major- citys assessors office tell me that you need Windoze to use their website (my work -required- access). In other words, eff you. Times like that make me wish I was a lawyer. Unfortunately, your kids are stuck in a Windows environment, for school and eventually, work. That doesn’t mean that they need not be taught about computer security and FOSS software and operating systems. The young pick up these things quickly.

Apologies if I’m preaching to the choir here.
. .. . .. — ….

Daniel • April 20, 2016 12:34 PM

I agree with the position that we should be limiting data collection. It is a point which I have harped on for many years in the comment section to this blog. But I disagree with her on WHY we should do that. I think “anti-domination” conceptions of Republican liberty are a fool’s errand. The problem isn’t conceptual, it’s practical.

In my view the problem lies not in the existence of power itself but in its lack of temporariness. Once the data is collected, it is never forgotten–computers don’t forget unless they are told to forget and with billions of computers on the planet it’s not possible for them all to forget. So once the data is collected, it becomes impossible to limit its use in any meaningful way. In other words, the problem with the line of thinking that says the problem isn’t the collection but the usage of data is that it is predicated on a fallacy viz. that it is possible to meaningfully limit the use of data. When that fallacy is recognized for what it is, then the only solution is to limit the collection of data.

Gweihir • April 20, 2016 3:28 PM

The collection alone already creates massive chilling effects, as most people do not know or understand the limits on using the collected data. Historically, these chilling effects were created by convincing people there was an all-seeing all-knowing “God” that dishes out punishment in the afterlife and hence people self-censored even what they dared to think. With smartphones and computers often used to hold thoughts, ideas, etc. the modern technological version is not much better. Sure, a few of us understand that all that surveillance data cannot actually be used except in a few targeted cases, but we are the exception.

Hence the primary problem is in the collection and it is an extreme danger to personal freedoms.

Steve • April 20, 2016 4:36 PM

“Under this framework, the problem with wholesale data collection is not that it is used to curtail your freedom; the problem is that the collector has the power to curtail your freedom. Whether they use it or not, the fact that they have that power over us is itself a harm.”

Living in any sort of civil society cedes the power, indeed, the monopoly on power to do harm to some sort of authority.

That’s why we have traffic cops and police of all sorts, because we have ceded the monopoly on force and violence to an authority. Even Libertarian theoreticians eventually posit some form of communal policing authority if only to prevent “free riders” from benefiting from private police.

Not that I find wholesale data collection of any sort to be particularly sanguine but when you think about it, the NSA is less likely to mess up your credit rating and cause you interminable hassle than, say, Experian, Equifax, or TransUnion. NSA has shown itself too busy collecting dots to ever get around to connecting them.

jones • April 20, 2016 4:55 PM

The definition of domination is adapted from C. Wright Mills’ definition of power, which is, the ability to make and enact plans regardless of opposition.

The notion that surveillance is power is inherent in the conception of the Panopticon, as described by Bentham & Foucault.

Incidentally, the Panopticon is the surveillance model in 1984: Winston Smith gets busted at the end due to police entrapment, not because his deviance was “detected.” The resale shop where he bought trinkets and eventually rented a room was run by the police.

Daniel • April 20, 2016 5:24 PM

@jones

The definition of domination is adapted from C. Wright Mills’ definition of power, which is, the ability to make and enact plans regardless of opposition.

Except no serious political philosopher believes such a thing is possible. After all, the idea of “non-domination” is itself an attempt at domination. It is the equivalent of saying that the only rule is that there are no other rules. (Rorty called such positions philosophical ironies.) In other words, non-domination is self-contradictory. This isn’t a philosophical problem if one believes that societies are inherently self-contradictory (or at least that constitutional democracies are inherently self-contradictory) but it is a problem if one believes that non-domination is supposed to have some moral force. Because there is no moral force whatsoever to criticizing others for doing the exact same thing one is doing oneself.

Anon10 • April 20, 2016 5:56 PM

Helen’s argument is nonsensical because it fails to distinguish between legal power and physical power. The government has the physical power to do a lot worse than read my e-mail. They lack the legal authority to arrest me, send me to a black site, and torture me, but they certainly have the technical capability to achieve that. If domination means that someone has the physical ability to control you, then domination occurs in every country in the world. A libertarian might argue that the people could fight back if the government ever became too tyrannical. So is Helen really against gun control or in favor of letting people buy fully automatic weapons? I doubt it, although it’s the logical end point of her argument. By that standard, domination is more extreme in Europe than the US. If you instead choose the more reasonable definition of domination as the legal authority to interfere with someone arbitrarily, then you can prevent domination by regulating use and not collection.

Jim • April 20, 2016 6:00 PM

I really like these comments:

He defines domination as the extent one person has the ability to interfere with the affairs of another.

Under this framework, the problem with wholesale data collection is not that it is used to curtail your freedom; the problem is that the collector has the power to curtail your freedom. Whether they use it or not, the fact that they have that power over us is itself a harm.

Very good definition of “domination”.

In the corporate world, data collection and cataloguing is heavily regulated. There are numerous main drivers for that: financial institutions, customer agreements, some governmental regulations, legal responsibilities, corporate potential for loss/liability (spans wide gamut, depends on industry and data, but from brand name damage to lawsuits, and so much more).

Much of that drive, if not all, is about potential unauthorized access and use of that data.

For examples: hacker gets your company healthcare records and posts to web; criminal group gets credit card numbers and uses them; proprietary technology is stolen and sabotaged while given to foreign competitor whose government “did it”.

Governments don’t have those sorts of stakes for losing data.

Look at OPM. One person got fired? Who hired her? Who managed her? Of all the government organizations trusting OPM with their confidential records, why did none of those check on how OPM was handling their trusted data?

That was not just intelligence agencies and military, either. That was any organization that has cleared employees. Parks and Reaction, the EPA, whatever.

This is even at a comical state of “no liability”. For instance, at one juncture, a lot of DoJ folks quit in public protest because of these very issues. What changed? What liability is there? Will anyone get fired or prosecuted? By who? The DoJ?

Get the fox to guard the henhouse.

And note, these are lawyers. Lawyers are hyper-sensitive to liability and caution. They literally have this much confidence.

The skeleton of the closed, covert judicial system was there, at least. It is amazing that its’ very existence made it to the public. But, there is nothing meaningful that will transpire from that being made public.

At worst, something might get shut down. Nobody will lose their job, nobody will get a pay decrease. Nobody will get personally sued. Maybe the government might lose some money, but not likely to be much.

If anything, having participated in such work, only would be a strong resume addition. To anyone meaningful they deal with.

Bad news is, everyone involved is also a citizen. Rarely do totalitarian programs not take as their victims their very founders.

That rule stays true in this case. They are creating pathways and doorways for rogue domestic intelligence to run free. Rogue domestic intelligence targets foremost leaders, judges, prosecutors, defense attorneys, large corporations. “Large corporations” include elite law firms they may wish to move on to.

That is their worst case scenario. And it is very far more probable of a threat they face, then your Average First Worlder. Nobody blackmails your average first worlder, because they have no power. You blackmail or anything else to manipulate those who have power. To control them.

So, for a bunch of lawyers and security folks, they really are exposing themselves to some very plausible risk, one of a very nasty variety.

Peter Gerdes • April 20, 2016 6:39 PM

The problem is any law which effectively restricts data collection endangers the rights of private citizens to both learn about and constrain the actions of the powerful.

Unsurprisingly laws are usually best employed by the powerful against the weak. Drafting a law which will stop big companies from collating giant amounts of (individually legally obtained information) and using it to expose personal facts about me but can’t be used against a grass roots organization which puts up a website to gather information about the wrong doings of some political candidate or controversial business leader seems all but impossible.

Moreover, it seems all but impossible to stop anyone with the money from downloading public data off the internet and analyzing it as they choose in some other jurisdiction.

I’m much less worried about a world in which big data is used to learn the people’s secrets but also used to expose hypocrisy and bad behavior of those in power. I’m very scared about a world where legitamate private companies and non-profits can’t collate information to expose the powerful but law enforcement exemptions and secret off shore analysis can be used to silence dissent

Talking about what privacy would be nice to protect without a proposed mechanism that avoids abuse is like talking about how it would be great if we could restrict only the really bad speech.

Craig McQueen • April 20, 2016 6:59 PM

Although we want freedom, doesn’t humanity also want to grant power to government, for certain benefits? By giving power to a king and his army to defend the nation and administer justice, a society obtains the benefits of security and rule of law. Even though the king’s powers are inherently capable of domination, the people are happy as long as the king uses the powers for the benefits of society as intended. In that way power contributes to liberty.

The question is, and always has been, how to keep reigns on abuse of power so granted. E.g. a bad king might be overthrown by a strong enough man. Democracy values the orderly limitation of government power that can be accomplished by the citizen collective within the normal rules of the process.

The power/domination problem with the haystack collection is not that the power exists, but there’s a feeling that the government has got out of hand, and is acting with secrecy that obstructs the workings of the democratic process, so that the citizens can’t reign in its power even if they see a need to. In this situation the ideals of democracy are in danger of failing. The solution under democracy is to ensure the essential principles of democracy are preserved, i.e. transparency, and that government must always be genuinely answerable to the majority vote of the citizens.

Jim • April 20, 2016 7:22 PM

@Peter Gerdes

I’m very scared about a world where legitamate private companies and non-profits can’t collate information to expose the powerful but law enforcement exemptions and secret off shore analysis can be used to silence dissent

You can’t trust “private companies” and “non-profits” to “collate [private, critical] information to “expose the powerful”.

That would be very bad. “Causes” run the gamut and themselves tend to be very biased, and so hypocritical. They would then have no accountability.

The fact is there is no meaningful regulation.

There is no meaningful enforcement.

There is simply nobody to regulate and enforce.

Not at all unlike how local police used to not have internal affairs departments.

The real threat is this creates a very strong breeding ground for rogue domestic intelligence.

Jim • April 20, 2016 7:27 PM

@Craig McQueen

The question is, and always has been, how to keep reigns on abuse of power so granted. E.g. a bad king might be overthrown by a strong enough man. Democracy values the orderly limitation of government power that can be accomplished by the citizen collective within the normal rules of the process.

Law enforcement dealt with this very similar problem by eventually implementing internal affairs soundly and across the board.

Federal agencies do not have that.

They do not police themselves, and when they do, it is biased and badly setup.

Fox guarding the henhouse.

At best, some intelligence agencies have counter-intelligence and security, but they are focused around external threats, even when focused on internal threats (moles).

The excuse of “just over zealousness” is just too strong of a cover for serious wrongdoing in those circles.

Dirk Praet • April 20, 2016 7:50 PM

@ Anon10

They lack the legal authority to arrest me, send me to a black site, and torture me, but they certainly have the technical capability to achieve that.

So what you are saying is that the CIA and the Chicago PD are in fact rogue organisations? And if that is true, there must have been quite some people accused and convicted of running secret detention facilities? What about the Disposition Matrix and Guantanamo?

I’m afraid most, if not all of that still appears to be very legal under secretive executive orders and secret interpretations of the law by secret courts. And even without resorting to these, you can also get prosecuted into poverty or suicide, as the DoJ did with Aaron Swartz. It would be ill-advised to underestimate the legal authority, public or secret, of any government, and the more data is being collected on you, the easier it will become to get at you if ever you tick off the wrong person. I refer to Richelieu’s famous quote “If one would give me six lines written by the hand of the most honest man, I would find something in them to have him hanged” and to Harvey Silverglates’ Three Felonies A Day: How the Feds Target the Innocent.

Anon10 • April 20, 2016 7:59 PM

@dirk

There were never any US citizens sent to CIA black sites because CIA lacks the legal authority to hold US citizens prisoner. I wouldn’t be surprised if some Chicago PD cops do in fact go to prison.

r • April 20, 2016 8:08 PM

@dirk,

Don’t forget the spin cycle on the washing machine. A little bit of slander can go a very long way: that’s what worries me about the collect it all program. You don’t even needed to exhaust money on a public trial before a crucifixion, just get the mob wriled.

Anon10 • April 20, 2016 8:18 PM

@dirk

I think it’s somewhat off point, but Aaron’s Swartz’s problems were largely of his own making. He used a computer that he wasn’t authorized to use to download illegally vast amounts of data with the intention to upload it all to the internet. When he got caught, he could have taken a plea deal for just 6 months in prison, but he chose suicide instead. It’s take a lot of media spin to show him as the victim.

what_you_get_when_dick_and_bush_rules • April 20, 2016 8:47 PM

Anon10
“There were never any US citizens sent to CIA black sites because CIA lacks the legal authority to hold US citizens prisoner.”

yea because they I guess were formally not classified as CIA black sites but e.g. Uzbekistan sites, cordially rented from countries that do not mind providing “services” to US of A.

r • April 20, 2016 9:03 PM

@anon10,

The way I understand it, the things he was downloading both currently are and were publicly available – just not for mirroring. It may have been legal to sip, but not chug the public waters in his case; prompting EULA/ToS violations, charges of obfuscation (network origin? this I’m certain is the actual hacking allegation), DoS (please see Connecticut for archaic laws(degradation of service (unwarranted ping or scan))), using a computer in commission of a crime (ToS violation)… If you don’t see where this is going then you don’t understand the whole forced suicide reference above.

I’m not saying what he did was right, but the man is dead for God’s sake after being crucified prior to a fair trial. Is it fair for the government to smeer someone in the public eye before a trial among his peers?

There’s no justice when there’s no trial.

r • April 20, 2016 9:19 PM

I forgot trespass, both digital and real: he was asked to stop correct?

Anon10 • April 20, 2016 9:42 PM

@dirk

In earlier posts, you’ve been pro gun control. That makes you pro-domination, if we’re to use Nissenbaum’s definition of domination. If you’re willing to give the state a monopoly on lethal force, then you’ve given physical power that dwarfs that of the people individually or collectively.

r • April 20, 2016 9:44 PM

@anon,

Are you keeping dossiers of everyone on this site?

Anon10 • April 20, 2016 9:53 PM

Ha, no, but there’s a few posters who seem to post in almost every thread.

r • April 20, 2016 10:11 PM

@anon10,

Well hands hands down your memory beats mine… I saw both of those 2015 dirk+gun+control threads and didn’t remember seeing any specific statement until I double checked your position and realized I’d seen them and you’re right. I can’t parse the XML of the second, but a little gun control good a long way: see Somalia.

Drone • April 20, 2016 11:28 PM

Not regulating data collection but attempting to regulate the use of the data is like giving a three-year-old a cookie, and telling him not to eat it.

discreditor • April 20, 2016 11:49 PM

@steve

Not that I find wholesale data collection of any sort to be particularly sanguine but when you think about it, the NSA is less likely to mess up your credit rating and cause you interminable hassle than, say, Experian, Equifax, or TransUnion. NSA has shown itself too busy collecting dots to ever get around to connecting them.

I’m reminded of Rumsfeld’s ruminations on “known unknowns” and “unknown unknowns”. This paragraph seems too confident in it’s knowledge of all things importantly knowable and relevant to this situation. How much intimate knowledge of the totality of NSA activities do you really believe you possess? Similarly, even if the NSA doesn’t maliciously and politically mess up your credit rating and cause you interminable hassle, if they do it to 50 of the most influential people on their 500 most influential people dossier, the secondary effects to you personally are likely going to be more significant than your concern level suggests.

orwellian fanatic • April 21, 2016 12:13 AM

@telesio jones

Incidentally, the Panopticon is the surveillance model in 1984: Winston Smith gets busted at the end due to police entrapment, not because his deviance was “detected.” The resale shop where he bought trinkets and eventually rented a room was run by the police.

That seems like an incorrect pigeon-holing to an either-or scenario. Certainly even (right thinking) philosophers that utterly oppose police/state entrapment don’t deny that entrapment can succeed (some percentage of the time) with deviance detection.

orwellian fanatic • April 21, 2016 1:37 AM

Since I’ve been given an opportunity to chat about 1984, I’ll go ahead and walk into that trap…

Upon further thought, I’m reminded of the keen relevance of ‘entrapment’ to 1984 (lets go with the movie which I’ve seen much more recently than having read the book). Specifically, continuing my prior comment- the key philosophical ponderance is what you hope to get by allowing/facilitating/engaging in state sponsored entrapment. 1984 demonstrates the psychology of the fact that (presumably in Orwell’s worldview) any human, persecuted enough by the state, can become guilty with willful intent of any crime. In fact, 1984 took it to biblical crucifixion level extremes. As I recall, the state had ‘succeeded’ when it had psychologically tortured the protaganists enough to the point where they had first discovered the intense feelings of human love and attachment and shared value/sacrifice, but then renounced them to the point of- “take that torture you have discovered that causes me the most excruciating terror (the rat face cage for this protaganist), and inflict it on my most special lover instead of me.”. It was at that point that the entrapping state had achieved its ultimate psychological goal. Dialing it down a dystopic notch, the general idea is that there isn’t anyone who couldn’t be made guilty of anything in an environment of total authoritarian power (ultimate state domination). So then the question becomes- what is your motive for allowing state entrapment in the first place? Is it to catch criminals? potential criminals (pre-crime)? Or to create criminals? To psychologically permanently damage your political opponents (Donald Trump – fire the person most likely to fire you, aka eliminate your enemies thoroughly and ruthlessly). State entrapment isn’t bad because it doesn’t work, it’s bad because it’s evil.

never say never • April 21, 2016 2:17 AM

@anon10

There were never any US citizens sent to CIA black sites because [they said so, and have never been known to lie]

It’s always so nice to read the comments here. Where else can you find people who have such total knowledge of alphabet agencies.

Philosophy of Complexity • April 21, 2016 2:33 AM

@Craig McQueen

By giving power to a king and his army to defend the nation and administer justice

I’m not sure you understand what a king is. A king isn’t given power (except perhaps by the previous, usually related paternally, king). Kings take power, and hold onto it. If they don’t take it, they aren’t king. If they don’t hold onto it, they stop being king.

The other glaring problem with your line of thought is that you are on all these discrete/binary things. It isn’t either or, it’s a vast spectrum of thousands/billions of individuals living their lives every day in countless varying ways. I know you are just passing things along the way you were taught. But the future of philosophy must include a greater emphasis on complexity and spectrums rather than ‘its this way or that way’. It’s actually a lot of different kinds of ways, all at the same time, continuously changing.

indominatable snowman • April 21, 2016 3:45 AM

@Daniel

This isn’t a philosophical problem if one believes that societies are inherently self-contradictory (or at least that constitutional democracies are inherently self-contradictory) but it is a problem if one believes that non-domination is supposed to have some moral force.

Not that I RTFA, but I think it wasn’t about the philosophical extreme end of the spectrum of non-domination that was being advocated, but rather anywhere in the middle of the spectrum, instead of the other extreme end of total-domination.

Somehow I feel like I’ve got a big pocket full of moral force right now, but I’m not quite sure what I’ll ever be able to do with it.

thirty one dimensional pendulum • April 21, 2016 4:28 AM

Unfortunately our politicians are stuck in the past and continue to think about the world as if it were the 1950s. The idea that individuals can render useless the copyrights, wiretaps, and various other regulations designed for a world of centralized services is alien to the most powerful people in America and Europe.

Most of your comment seemed good, but I’d be more hesitant to make such a sweeping assessment of ‘our politicians’. Also, I think basic one time pad knowledge was probably about the same in the 50s, thus the ‘rendering useless of wiretaps’ seems fundamentally about the same.

7 • April 21, 2016 4:59 AM

@drone

Not regulating data collection but attempting to regulate the use of the data is like giving a three-year-old a cookie, and telling him not to eat it.

It’s important to distinguish between regulation of data collection by the government, and by private individuals and corporations. It’s also important to question more skeptically how it is that for instance twitter, youtube, and facebook command such market dominance. They really aren’t that good.

Anselm • April 21, 2016 6:42 AM

The reason Twitter, YouTube, and Facebook are so dominant has less to do with their massive data collection but more with being in the right place at the right time and not sucking too much. (FWIW, Microsoft is what it is today because they were there in the right place at the right time – when IBM was looking for an operating system for the then-new PC – and were able to deliver something that didn’t suck too much, by the low, low standards of the era.)

Once you have achieved a certain size, massive data collection becomes useful because you can sell the data to advertisers, who generally aren’t really interested in two-bit mom-and-pop garage-type web sites that are just starting out. The other thing to remember is that Facebook etc. aren’t primarily in the business of making life blissful for end users. It is more appropriate to think of them as advertising agencies. If you’re Facebook or Twitter, you have to be useful enough to end users so they will stay with your service (being the dominant player helps there because of network effects), but the end users are not actually your customers – their attention is the product that you’re selling to your real customers (the advertisers).

If they wanted to, Facebook could easily do away with all the advertising and associated data collection by charging users, say, $1 per month. With an operating budget of over $12 billion per year you could do some pretty amazing things for your end users, who at that point have become your paying customers. The problem with that idea is that end users would much rather pay Facebook with their personal data than with real money, and would probably leave Facebook in droves for something like Google+ that is still “free”, but Facebook needs to make money somehow, hence advertising. (Just watch what happens whenever somebody on Facebook posts the “OMG Facebook will be $9.95/year from May 1st” meme.)

Dirk Praet • April 21, 2016 7:16 AM

@ Anon10

When he got caught, he could have taken a plea deal for just 6 months in prison

For the unspeakable crime of downloading subscription-only documents JSTOR didn’t even press charges for, Swartz was facing a maximum of up to 35 years in jail and $1 million in fines. Do you have even the foggiest idea what that can do to someone’s mind, even if he gets a plea bargain of “just six months” and having his *ss owned for the rest of his life? This is the very definition of prosecuting someone into suicide quite alright.

Needless to say that none of the people that caused the financial crisis in the US were even indicted.

In earlier posts, you’ve been pro gun control. That makes you pro-domination

Only in the eyes of US gun nuts, mate. Us over here in the EU have a completely different opinion on gun legislation, and there really is no point in trying to derail this thread in turning it into a gun control debate.

There were never any US citizens sent to CIA black sites because CIA lacks the legal authority to hold US citizens prisoner.

You’re dead wrong. The FBI and CIA seem to be well within their rights to secretly detain any US citizen abroad forever, and without remedy. The case of Amir Meshal comes to mind, to name just one. It’s not even that far a stretch given the fact that US citizens can even be killed abroad without trial, like what happened to Anwar al-Awlaki.

AlanS • April 21, 2016 8:55 AM

Pettit contrasts republican and liberal conceptions of liberty but a lot of people who one would normally label as classic liberals have a very similar conception of liberty to the one he proposes. And he says this in places in the essay you link to and elsewhere. But under liberalism (or whatever you want to call it) there is an assumption of a legal and institutional framework, rights, justice, etc.

I agree with the problem Nissenbaum defines and why it is a problem but the solution seems unlikely as we don’t live in that liberal world of rights any more. The solution is nostalgic. We still talk in this sort of language but it is not the practice of the world. We live in a world in which everything (and that would include government and the law) is increasing subservient to the rationality of the ‘market’. It’s all about information, efficiency, self-interest, competition, flexibility, maximization, and calculation. And surveillance, sucking up vast amounts of information and doing so to an ever greater degree, is a fundamental feature of this world. The solution can’t merely be passing a regulation because for that to work it depends on something of a Copernican Revolution in the way we conceptualize governance, social relationships and ourselves.

AlanS • April 21, 2016 10:46 AM

@Jones

C Wright Mill’s writings on the Power Elite are very important. They maybe a bit dated now. It is interesting that you then go on to mention Foucault as both Mills and Foucault can be seen as writers who take up and expand Weber’s critique of rationality, bureaucracy and expertise.

I don’t think we should over-emphasize the role of Panoptic surveillance in Foucault’s writings. He himself writes:

The central point of the panopticon still functions, as it were, as a perfect sovereign. On the other hand, what we now see is [not] the idea of a power that takes the form of an exhaustive surveillance of individuals so that they are all constantly under the eyes of the sovereign in everything they do, but the set of mechanisms that, for the government and those who govern, attach pertinence to quite specific phenomena that are not exactly individual phenomena, even if individuals do appear in a way, and there are specific processes of individualization (pp.93-94)

He then goes on to discuss a very different type of surveillance:

I think the population no longer appears as a collection of subjects of right, as a collection of subject wills who must obey the sovereign’s will through the intermediary of regulations, laws, edicts, and so on. It will be considered as a set of processes to be managed at the level and on the basis of what is natural in these processes. But what does this “naturalness” of the population signify? What is it that means that the population will henceforth be seen, not from the standpoint of the juridical-political notion of subject, but as a sort of technical-political object of management and government? What is this naturalness?…I think it appears in three ways…[1.] the population is not the simple sum of individuals inhabiting a territory….it is dependant on a series of variables…..the naturalness identified in the fact of population is constantly accessible to agents and techniques of transformation, on condition that these agents and techniques are at once enlightened, reflected, analytical, calculated, and calculating…[2] in the fact that this population is of course made up of individuals who are quite different from each other and whose behavior, within a certain limit at least, cannot be accurately predicted. Nevertheless…there is at least one invariant that means that the population taken as a whole has one and only one mainspring of action. This is desire….[I]t is here that this naturalness of desire thus marks the population and becomes accessible to governmental technique…[3]…Now it is enough to observe these phenomena that should be irregular, it is enough to look at them and count them, to realize that in actual fact they are regular….In other words, with the population we have something completely different from a collection of subjects of right differentiated by their status, localization, goods, responsibilities, and offices:[We have] a set of elements that, on one side, are immersed within the general regime of living beings and that, on another side, offer a surface on which authoritarian, but reflected and calculated transformations can get a hold….[P]opulation is therefore everything that extends from biological rootedness through the species up to the surface that gives one a hold provided by the public…(STP pp.98-105)

This is from lectures in 1977/78 but this is a description of a type of scientific or expert management that is not only essential to modern government but to corporations such as Google, Amazon etc. But he’s discussing practices that originate in response to problems back in the 18th C. And this population surveillance does not replace disciplinary power (or even sovereign power). They exist together.

Also note that this relates to liberalism and neoliberalism (which he takes up in the following year’s lectures). Power in this sort of population surveillance acts not through discipline and oppression in the way Panoptic power does, but through recording variables, calculation and acting on desire. (This is something taken up in Harcourt’s recent book, Exposed: Desire and Disobedience in the Digital Age). Liberalism involves the notion that the market operates best when there is limited/judicious regulation. Neoliberalism is a different animal. The market sphere colonizes everything –and at a certain level is imposed–and with it a very specific rationality. The moral component in classic liberalism (e.g. Adam Smith) is gutted. (For recent discussion see Davies The Limits of Neoliberalism: Authority, Sovereignty and the Logic of Competition and Amadae’s Prisoners of Reason: Game Theory and Neoliberal Political Economy).

This is why I think Nissenbaum’s solution is ineffective. The larger historical configuration of power/knowledge escapes her analysis.

jones • April 21, 2016 11:31 AM

@AlanS

You’re right to point out that aspects of Mills’ analysis are dated; for example, as he was discussing the rise of the permanent war economy over 50% of federal spending went to the Department of Defense. While this number is now smaller, the important point in Mills’ analysis that is still significant today is that, while we call our government a democracy, there are large portions of the government that are not under democratic control, but which are, rather, controlled by appointed officials.

The full title of Bentham’s Panopticon reads:

PANOPTICON; OR, THE INSPECTION-HOUSE:
CONTAINING THE IDEA OF A NEW PRINCIPLE OF CONSTRUCTION APPLICABLE TO ANY SORT OF ESTABLISHMENT, IN WHICH PERSONS OF ANY DESCRIPTION ARE TO BE KEPT UNDER INSPECTION; AND IN PARTICULAR TO PENITENTIARY-HOUSES,
PRISONS, POOR-HOUSES, LAZARETTOS,
HOUSES OF INDUSTRY, MANUFACTORIES, HOSPITALS,
WORK-HOUSES, MAD-HOUSES, AND SCHOOLS:

with A PLAN OF MANAGEMENT adapted to the principle

There is, in fact, only one democratic body in our government: the Congress. These days, Congressional representatives no longer write the laws or read them, but simply vote on them, making them not so much powerful, but instruments of the powerful. They are pseudo-celebrity figureheads, to borrow a phrase from Guy Debord.

The argument Mills makes about the rise of the “managerial society” is also something that I think warrants further attention today, as it relates to where power lies and how power operates. What enables the power elite’s power is that, as more realms of society come under bureaucratic control, the key criterion for entering and working effectively in those bureaucracies is not so much skill but grooming — that is, a lifelong familiarity with specific modes of bureaucratic administration. More than any one specific qualification, it’s what allows members of the power elite like Dick Cheney to move from a position in the Congress to the Cabinet to private industry and back to the Oval Office (though Cheney has offered conflicting views on whether he was actually a member of the executive branch https://www.youtube.com/watch?v=3bthCtlXedc ).

One consequence of the bureaucratic control of society is that it creates a de facto class system, where the wealthy get socialism and the rest of us get authoritarian coercion, though we often discuss these coercive pressures as “freedom.” There is little significant wealth that is not corporate wealth, or derived in some other way from structured finance: stocks, bonds, trust funds, expense accounts, corporate retreats, corporate limos, drivers, etc. Note in this connection that the “book within the book” in 1984 was titled “Theory and Practice of Oligarchical Collectivism.”

Your remarks about the importance of the Panopticon in Foucault’s writings are perfectly fair with respect to my comments, and your point about mass society as control are well taken; below is a comment I posted on this blog earlier that maybe makes more clear the main point I was getting at….

Cheers!

This issue is getting some attention lately:

http://news.sciencemag.org/archaeology/2015/08/birth-moralizing-gods

The idea is that when human societies are organized along tribal hunter-gatherer lines, everybody knows what everybody is doing, and there is no need for abstract deities to enforce social norms.

This is closely related to something Jeremy Bentham formulated with the idea of the panopticon. Here’s Foucault’s characterization of the idea:

“In view of this, Bentham laid down the principle that power should be visible and unverifiable. Visible: the inmate will constantly have before his eyes the tall outline of the central tower from which he is spied upon. Unverifiable: the inmate must never know whether he is being looked at at any one moment; but he must be sure that he may always be so.”

“Hence the major effect of the Panopticon: to induce in the inmate a state of conscious and permanent visibility that assures the automatic functioning of power…

“So… that the surveillance is permanent in its effects, even if it is discontinuous in its action; that the perfection of power should tend to render its actual exercise unnecessary; that this architectural apparatus should be a machine for creating and sustaining a power relation independent of the person who exercises it; in short, that the inmates should be caught up in a power situation of which they are themselves the bearers.”

This is precisely the concept of surveillence at play in 1984:

“There was of course no way of knowing whether you were being watched at any given moment. How often, or on what system, the Thought Police plugged in on any individual wire was guesswork. It was even conceivable that they watched everybody all the time. But at any rate they could plug in your wire whenever they wanted to.”

“You had to live–did live, from habit that became instinct–in the assumption that every sound you made was overheard, and, except in darkness, every movement scrutinized.”

This is one of the less well examined consequences of the surveillance society: surveillance is control, regardless of whether law enforcement gets its hands on the information collected.

Also note that in 1984, it’s not surveillance that gets Winston Smith busted, but police entrapment.

Steve • April 21, 2016 11:39 AM

@discreditor: My comments wrt NSA were somewhat tongue in cheek. However, human nature being what it is, I think we give NSA more credit for competence than they probably deserve.

Look around you, at your colleagues and your competitors. How clueful are they? How good at their jobs are they really? Even the stars and the standouts eventually regress to the mean.

Mostly they’re bumbling, semicompetent bozos, just like the rest of us.

Наркомюст USA • April 21, 2016 2:16 PM

Oh they use it, don’t you worry about that. Having shredded the legal right to privacy under the pretext of failing to stop terror, CIA’s FISA rubber stamp is now acting to maximize the repressive capacity of its overreach:

http://www.latimes.com/nation/la-na-fisa-court-records-20160419-story.html

The government never gave a rat’s ass about your privacy before – they forgot about it shortly after they made a binding legal pledge to respect it in 1992. We reminded the US government about privacy, and the beltway parasites looked at it, and sniffed it, and thought what they always think: “Gee, wouldn’t that make a great weapon against dissidents and political prisoners!” So now the government is using privacy to gag framed human sacrifice Jokar Tsarnaev:

http://whowhatwhy.org/2016/04/21/doj-cites-bizarre-reason-deny-whowhatwhy-access-tsarnaev-info/

Rights distortion is a standard US government trick for attacking foreign states – in this case Russia and non-compliant Moslem-majority states. It breaches the peremptory norms against coercive interference and bad-faith interpretation of conventional international law. When the Soviet Union expired, the US government brought their red-tape totalitarianism over here. You’d have to be an idiot to mistake the USA for a legitimate or lawful state. That’s the problem with academic political philosophy. It doesn’t know what to make of criminal states.

Anon10 • April 21, 2016 5:57 PM

@dirk

Are you claiming that Nissenbaum is a US gun nut? You might be right.

Anwar al-Awlaki was killed not captured or detained, so is irrelevant to my claim. Meshal is an interesting case, but it’s not clear that the FBI ever detained him except in the process of returning him to the US. He appeared to be in Kenyan or Ethiopian custody, while the FBI interrogated him.

Dirk Praet • April 22, 2016 8:17 AM

@ Anon10

Are you claiming that Nissenbaum is a US gun nut? You might be right.

What I’m saying is that equating gun control to supporting government domination is pretty much preposterous.

Meshal is an interesting case, but it’s not clear that the FBI ever detained him except in the process of returning him to the US

You should read Patrick G. Eddington’s “How the F.B.I. Can Detain, Render and Threaten Without Risk” some time. His conclusion was that “Mr. Meshal has fallen into a legal black hole, where the light of justice is extinguished in the name of national security. The appellate court decision means that American citizens have no means available to hold the government accountable for violating their constitutional rights, simply because the United States conveniently denied those rights in another country of its choosing.”

Anon10 • April 22, 2016 5:48 PM

@dirk

What I’m saying is that equating gun control to supporting government domination is pretty much preposterous.

It sounds like you’re in agreement with me that Nissenbaum’s definition of domination is preposterous, which was my original point. I’m still not sure how we got sidetracked to Aaron Swartz.

You should read Patrick G. Eddington’s “How the F.B.I. Can Detain, Render and Threaten Without Risk” some time.
You should read it as well, including this quote, One of your congressman’s constituents is being held in an Ethiopian intelligence service prison. If you’re in an Ethiopian prison, Ethiopia, not the FBI, is detaining you.

r • April 24, 2016 4:51 PM

From Slashdot, on topic

https://tech.slashdot.org/story/16/04/24/1629215/how-big-data-creates-false-confidence

Eppesuig • September 21, 2016 8:02 AM

Chess players use to say that “a threat is more powerful than it’s execution”.

We do not need actual blackmails cases, the mere existence of a potential for blackmail is harmful.

Physically forcing people to do things is inefficient, you may have to dedicate more resources to it than would be necessary to perform the desired task in the first place. The threat created by uncontrolled data collection, on the other hand, is a powerful persuasion tool and not likely to have the sort of unintended effects the use or threat of physical force has.

Massive data collection is harmful even if the threat is not there. We are left to decide whether that harm is balanced by some other benefit but this is made impossible as the data needed for rational cost/benefit analysis is kept secret.

A second risk of massive data collection is more subtle, computer driven finely tuned targeted advertising may look less threatening than brainwashing, but the same data and algorithms used by Amazon to suggest I buy one particular book could be used to “suggest” much more important decisions. When political parties start using them I believe we have crossed a line. I want to believe I’m voting for people who will be the best for my future, even if it’s often a choice of the lesser evil, not for the ones that have the best “persuasion software”, persuasion skills are important in a leader but picking the right algorithm is no indication of that.

The third risk is that it creates a huge unbalance between people with access to the databases and those who have not such access effectively turning a court system that relies on equal access to the evidence into a farce.

So the European approach of equating data collection with other high risk activities, to be avoided whenever possible, looks better than the US belief that all you need to prevent harm is to put some controls in place after the data was collected. Whether legislators will be able to come up with sensible rules is a different matter but at least the EU is looking at the problem from the right perspective.

Schneier on Security

Helen Nissenbaum on Regulating Data Collection and Use

Comments

Leave a comment Cancel reply