Schneier on Security
A blog covering security and security technology.
« Protecting E-Mail from Eavesdropping |
| Another Perspective on the Value of Privacy »
July 8, 2013
Big Data Surveillance Results in Bad Policy
Evgeny Morozov makes a point about surveillance and big data: it just looks for useful correlations without worrying about causes, and leads people to implement "fixes" based simply on those correlations -- rather than understanding and correcting the underlying causes.
As the media academic Mark Andrejevic points out in Infoglut, his new book on the political implications of information overload, there is an immense -- but mostly invisible -- cost to the embrace of Big Data by the intelligence community (and by just about everyone else in both the public and private sectors). That cost is the devaluation of individual and institutional comprehension, epitomized by our reluctance to investigate the causes of actions and jump straight to dealing with their consequences. But, argues Andrejevic, while Google can afford to be ignorant, public institutions cannot.
"If the imperative of data mining is to continue to gather more data about everything," he writes, "its promise is to put this data to work, not necessarily to make sense of it. Indeed, the goal of both data mining and predictive analytics is to generate useful patterns that are far beyond the ability of the human mind to detect or even explain." In other words, we don't need to inquire why things are the way they are as long as we can affect them to be the way we want them to be. This is rather unfortunate. The abandonment of comprehension as a useful public policy goal would make serious political reforms impossible.
Forget terrorism for a moment. Take more mundane crime. Why does crime happen? Well, you might say that it's because youths don't have jobs. Or you might say that's because the doors of our buildings are not fortified enough. Given some limited funds to spend, you can either create yet another national employment program or you can equip houses with even better cameras, sensors, and locks. What should you do?
If you're a technocratic manager, the answer is easy: Embrace the cheapest option. But what if you are that rare breed, a responsible politician? Just because some crimes have now become harder doesn't mean that the previously unemployed youths have finally found employment. Surveillance cameras might reduce crime -- even though the evidence here is mixed -- but no studies show that they result in greater happiness of everyone involved. The unemployed youths are still as stuck as they were before -- only that now, perhaps, they displace anger onto one another. On this reading, fortifying our streets without inquiring into the root causes of crime is a self-defeating strategy, at least in the long run.
Big Data is very much like the surveillance camera in this analogy: Yes, it can help us avoid occasional jolts and disturbances and, perhaps, even stop the bad guys. But it can also blind us to the fact that the problem at hand requires a more radical approach. Big Data buys us time, but it also gives us a false illusion of mastery.
Posted on July 8, 2013 at 11:50 AM
• 28 Comments
To receive these entries once a month by e-mail, sign up for the Crypto-Gram Newsletter.
Data, Information, Knowledge and Wisdom.
The first two are more widely available than arguably at any other time in recorded history.
The third item is more rare in examples of its beneficial application.
The fourth item seems a forgotten art by current political practitioners, to the detriment of us all.
As I alluded to in my post to the prior thread, when you have so many laws that everyone is a criminal, and so much data that you cannot possibly prosecute all the "crimes" that you have evidence of, then you can be choosy about who you do go after. It is a pretty convenient tool for a government that wants to silence its detractors.
And if you cannot get them for a crime, you can probably find a personal indiscretion to embarrass and/or discredit them, which is pretty effective at silencing them.
We are not interested in wisdom, we are interested in power. The greatest pleasure known to humanity is the arbitrary exercise of power. That is the point of making everything illegal: the person in power gets to pick and choose with total freedom and yet without public accountability.
The purpose of Big Data in this schema is two-fold. The first purpose of it is to cow people with the realization that somewhere somehow someone is always watching. The second one is to give the illusion of a system; that it all somehow makes sense but the average person is just too stupid to understand.
The ultimate goal is to allow the organs of the state to act arbitrary and capriciously under the color of a rational, efficient, benevolent administration. Heaven of earth, to a certain mind.
The subtle meanings of human behavior aren't very amenable to massive data crunching. Is it a wink or a twitch? Is it a terrorist threat or a sarcastic teenager's post? Hard to tell without lots of interpretive context. But some people aren't partial to subtlety. They prefer their data thin.
Daniel: We are not always interested in making specific things illegal or legal, we are interested in knowing everything about those around us so we can shame & manipulate people to get our way. Aren't we..?
The reverse is also true -- Bad Policy Results in Big Data.
So I'm wondering if this NSA Gizmo can predict rise and fall of any particular stocks over a defined period of time with any profitable accuracy based on prior and unique knowledge of the data. If it knows something that everyone else does that doesn't count. If it has some technical quickness that allows it to get data before everyone else that doesn't count either.
I think there are so many seductive arguments to deploy such a vacuum cleaner tool.
1. If we just had enough data we could see the pattern we need to see.
2. If nothing else we can roll back time and see the dots.
3. Why this here gizmo is the top of the line terrorist (whatever) detector!
4. National Security
I do agree though -- having big data might make politicians even more lazy/worthless! :)
"Big Data... can help us avoid occasional jolts and disturbances and, perhaps, even stop the bad guys."
There is no evidence for that. We gathered boatloads of overhead surveillance data during the Cold War. There was too much photographic data to analyze. Analysts guessed at which photos were important. They usually were wrong.
The current massive text, voice and location/travel information also cannot be reliably analyzed, even with custom-programmed software to identify patterns.
The 9/11 attack could have been prevented if the FBI had paid attention to a flight instructor's report of Arabs who wanted flight training but not landing training. The Boston Marathon bombing could have been prevented if federal agent paid attention to the wealth of data on the perpetrators, including visits to countries hostile to the USA and also written warnings from Russia and Saudi Arabia.
Big data in the USA today is not about preventing terrorist atrocities. It is about characterizing individuals and guesstimating their risk to the government and its politicians. The false positives are likely to be a thousand times more frequent than true positives, but all the individuals will receive equal scrutiny and harassment.
What's most frightening to me is that almost all elected Congresspersons support federal collection of big data. They apparently (and probably correctly) believe that the voters will not punish them for their support of Orwellian spying. That's not a surprise, since most voters are idiots.
Very well said - yet many people will embrace data mining, simply because it seems big, powerful and advanced - pretty much the same reason that many people (blindly) advocate space programs.
Actually this is the exact same line of thinking that the well respected data visualization expert Stephen Few has about Big Data in general. http://www.perceptualedge.com/blog/?p=1671
This view of finding correlations without knowing or even caring about the causes is quite disheartening to me as someone who's career is about getting the best information from data. But it is absolutely terrifying when applied to criminal prosecution and government policy making.
"This view of finding correlations without knowing or even caring about the causes"
The proper name for this view is magical thinking.
"In clinical psychology, magical thinking is a condition that causes the NSA to experience irrational fear of missing certain data or not reading certain thoughts because they assume a correlation with such acts and threatening calamities."
Speaking of information overload, someone actually wrote a story on a monkey's blue scrotum in the popular stories section. F*cking stupid!
This sort of approach isn't constrained to surveillance. Any big data that is hard to understand is susceptible to it, and politicians love a good bandwagon for arbitrary policy making, to be seen to act decisively.
From graphs of weather relationships to mapping the patterns of deaths in insects - it is easier, cheaper, and gets your name in the headlines faster to jump on a correlation and say 'we must act' than to actually understand what is going on. Usually nobody even notices that it's just a correlation. Often, we as a species still don't know the causal relationships in science, and most laypersons treat it as 'well, they said we've got to act on this, surely case closed once we've done that?'.
If I could pick one thing to educate every human being on, it would be 'correlation does not imply causation', yet they bury it in high level statistics courses.
> it just looks for useful correlations without worrying about causes, and leads people to implement "fixes" based simply on those correlations -- rather than understanding and correcting the underlying causes.
This is not a new problem, ever since they invented statistics you have those who use statistics wisely, and those who make bad inference from available data. I don't know if big data introduces a trend of excessive reliance on statistical inference; or a worsening of the quality of such inference. In the end it is a tool that can be used wisely or it can be used foolishly;
It will probably take a while to learn this 'new' tool, but they will learn, learning always happens from mistakes though, so the new benefit of the new toy is questionable.
"If I could pick one thing to educate every human being on, it would be 'correlation does not imply causation', yet they bury it in high level statistics courses."
+1, although I've always taken it a step further and suggested that people should undertake far more statistical education generally (myself included).
Here is a citation that applies:
"A good decision is based on knowledge and not on numbers."
Seems this kind of stupidity is very old indeed.
The "Big Data" approach to social control is the one which leads to the French or Russian revolutions. By abusively suppressing symptoms, the elite do not notice that they are creating a powderkeg where *everyone* wants to get rid of them, including their own security forces.
It's actually astoundingly stupid. Emperor Augustus would have rejected it.
"Big data in the USA today is not about preventing terrorist atrocities. It is about characterizing individuals and guesstimating their risk to the government and its politicians. The false positives are likely to be a thousand times more frequent than true positives, but all the individuals will receive equal scrutiny and harassment. "
The problem is that this sort of spectacularly abusive harassment of complete innocents is what *makes* people hostile to the government. Eventually they'll be so busy hunting down Quaker drumming circles that they won't notice that their own generals are ready to overthrow them for their evil behavior.
"'correlation does not imply causation', yet they bury it in high level statistics courses..."
I've taken introductory statistics and statistics for epidemiology. I taught laboratory statistics. Correlation and causation were prominent in all three.
Some taxidermist once told me that if someone came to him with an eagle that had been shot with a shotgun, the hunter would have to tell him what it was because out of the remains he could reconstruct a duck just as well as an eagle. Anyone see the analogy ?
I find it difficult to make the obscure--not. To many of the locals here I am a defector. But not the kind that results in a war or lossing 40 trillion dollars of worldwide pseudo assets. Never bought or sold CDS instruments, never gave a knuckle head in DC credit for being contemplative, and never stole a cookie from the cookie jar or an AWT jar for that matter. By now you've summized my position relatve to the politics of the day. My statements are to the point and my fidelity to principal and people must be obvious--even though I am a bit obscure.
I find it difficult to make the obscure--not. To many of the locals here I am a defector. But not the kind that results in a war or losing 40 trillion dollars of worldwide pseudo assets. Never bought or sold CDS instruments, never gave a knuckle head in DC credit for being contemplative, and never stole a cookie from the cookie jar or an AWT jar for that matter. By now you've summized my position relatve to the politics of the day. My statements are to the point and my fidelity to principal and people must be obvious--even though I am a bit obscure.
The DoD is writing law, the proof I offer is in the policy document CFR 32 Part 182. In the congressional record of 14 April, the policy directive from the Secretary of Defense is published and a final ruling is made--after two comments. Nothing could be so bleak. The department dismisses both comments; "No action required."
This has gone too far...the legislation (DoD's words) is operative absent the participation of congress. I keep saying, the rule of law is kaput. Gone, 86'd, asta la vesta baby.
Forgive me, fat fingered a buffer copy!
In the sense that "bad law" is CRIMINAL given intent, the fed has been engaged in a greater than a decade attack on its ability to prosecute citizens PERIOD. For what ever color of law the feds see righteous (John Luke Picard--make it so number one), the cuffs are remade irrespective (here's where it is criminal and conspiratorial) of formative law or restrictions on prosecutorial egis. What is obvious, how the proponents of these "rights" to do their job, their isn't an ignorance or blindness that the right lays with the people (the true sovereign ) and this is antithetical to prosecutorial thinking...here is the boil down.
Everytime a "query" is executed, the only assumption available is that EVERYONE is guilty. In a sense it is a federal investigation in which EVERYONE is a suspect. There is no analog in case law today! How do legal scholars get past this..there is no human prior to scope or ACCURACY.
Actually, the folk wisdom of the Fertile Crescent and the Central Asian Plateau has a comment to make on this, in the form of the Mullah Nasrudin aka Goha/Juha/Guiffa/Hoja:
The local bigwig was getting ready to go out for the evening, so he asked the Mullah Nasrudin, who was working as guard that day, to tell him what the weather was like. The Mullah Nasrudin observed the cat dash in, soaked. He prided himself on his deductive ability, and said immediately that it was raining.
He got fired. Someone had thrown a bowlful of dirty washing water on the cat.
And the allegation that this massive overcollection of data has any bearing on the prevention of terrorist attacks?
The Mullah Nasrudin had done his washing for the day and had spread his cloak on the parapet of his house to dry.
The wind blew up and deposited the cloak on the dirty dusty pavement.
The Mullah Nasrudin immediately knelt and began praising god.
His neighbour said, "But Mullah! Your cloak is dirty again! How can you be praising god for that?"
"Well," reasoned the Mullah, "I am praising god that I wasn't in it when it fell off the roof. I could've been hurt quite badly."
Case in point: UK runs a massive austerity program to cut social services, yet at the same time has recently increased the funding of its surveillance programs.
One issue here is the very fact that (as people have pointed out above) big data is never quite big enough. There will always be ambiguities and misinterpretations. So there's a strong temptation to imagine that if you just collected a bunch more data you would arrive at a solution that wouldn't involve disturbing the Powers That Be.
I wonder, now, whether part of what made the East German model work for as long as it did was the huge percentage of the population who were recruited as snitches. Not only did it help spread the wealth (such as it was) around, it also co-opted some substantial fraction of antisocial impulses. Along those lines, the problem in the US would be all those young people who aren't getting employed as camera-footage watchers. Technological unemployment rears its well-groomed head.
Speaking of bad policy....please pay attention to PART II.
The framers limited trespass on constitutional rights--only by means of changes to the constitution, meaning that congress cannot just pass a law nor can the executive issue an order that is in contravention to the rights secured to the sovereign. As stated in Article 1, section 9; Limits on Congress:
"The privilege of the Writ of Habeas Corpus shall not be suspended, unless when in the Cases of Rebellion or invasion the public Safety may require it."
The language is clear, "unless when in the Cases of Rebellion or invasion", it doesn't say "...in Cases of Rebellion or invasion, or some other reason, the public Safety may require it."
This is one clause, with two terms "Rebellion, invasion".
Also, as I read this, congress can only produce exceptions to Rights guaranteed under the Bill or Rights, by means of constitutional law. The house and senate cannot just pass a bill, such as the Patriot Act, and trample on those rights. It is clear, as the right of Habeas is expressly defined in the constitution, and thus any restriction that the government would seek must rise to the same level.
Additionally, the constitution specifies "Limits on Congress" including a statement concerning "Rebellion or invasion" are reserved for specific circumstances. Neither condition, rebellion or invasion, is active or relevant. Unless an amendment to the constitution that has re-defined these terms, a constitutional test would require measuring the use of excess congressional authority with the limits to its authority as stated in Article 1, section 9, and, the termination of authority in the affirmative held in the Bill of Rights under the 10th Amendment. So, the MIN/MAX thresholds are obvious, and congress is obviously acting beyond these measures.
The constitution restricts completely the executive from performing any activity that could be considered domestic. Article 2, section 2 defines the power that is inherent in the office. A quick read shows that the three paragraphs reserving to the executive appointing ambassadors, judges, and issuing pardons. Not a whole lot of leverage to affect domestic law and activity but this is the rationale under FISA (claiming autonomy):
"...enable, if not invite, measures on independent presidential responsibility."
(a)(1) ... "authorizes and electronic surveillance program to obtain foreign intelligence information or to protect against international terrorism.
"DISMISSAL--The Foreign Intelligence Surveillance Court of Review or a court that is an originating court under paragraph (1) may dismiss a challenge to the legality of an electronic surveillance program for any reason provided for under law."
PART II - HERE IS WHERE THEY F'd THEMSELVES:
THIS SECTION CLAIMS THE SAME AUTHORITY FOR PHYSICAL SEARCHES AS IT DOES FOR ELECTRONIC--AND LEAVES ROOM FOR DRONE SURVEILLANCE.
"(j)AUTHORIZATION DURING TIME OF WAR."
Striking section 111
(k) PHYSICAL SEARCHES--Title III of FISA (amended)
"(1) in section 301, by striking paragraph 5 and inserting the following:
"(5) 'Physical search' means any physical intrusion with the United States into premises or property (including examination of the interior of property by technical means) that is intended to result in a seizure, reproduction, inspection, or alteration of information, material, or property, under circumstances in which a person has a reasonable expectation of privacy and a warrant would be required for law enforcement purposes, but does not include activities conducted in accordance with section 102 or 105."
---BUT NOW IT IS CONSPIRATORIAL---
The language in the bill says it must describe means and methods (classic restriction for classification) and forcible makes the orders classified. And I argue:
HERE IS THE DELIBERATE UNDERMINING OF CONSTITUTIONAL LAW, THAT IS WHY THIS GOES TO CONSPIRACY--A CONSPIRACY AGAINST THE CITIZENS OF THE UNITED STATES:
"(6) include a statement of the means and operational procedures by which the electronic tracking will be executed and affected:"
Nah, go with
(c) Video games to keep unemployed youths at home, indoors, and occupied.
I honestly believe that is a large part of the reason we haven't seen the expected surge in crime from the poor economy. My town, with a smaller population, had a much higher rate of burglaries and vandalism back in the 1970s -- you know, when teens roamed around because it was boring at home.
They seldom roam anymore.
After a once through read, understanding that what had been drafted over the life of the Patriot Act--I find it impossible not to understand the objective(s) and motive(s) of this legal abomination. The single, highest order concomitant identifier that might be associated with the decision process must be that the number of senators that are prosecutors or AG's and find little to object to when it affords LEO's what they want. Isn't that what you're are supposed to do...
As a non-cognitive dissent (there appears to be no need for a mind, at least one that could be quantified as cognitive), anyone having read the bill and responsible for signing it--did exactly the opposite of what the framers did at the Philadelphia hall in July of 1776. My hope is that we are not too late to turn this around, and won't be easy--making an 800 pound gorilla sit for a cup of tea.
My suggestion is to;
1.) stop creating victims.
2.) build coalitions, alliances, citizens' groups, and strange bed fellows,
3.) provide answers to the locus of friction:
a.) The classic--"I'm not doing anything wrong, so what's the big deal?"
b.) answer the force that will be pushing the a 100,000 cars behind 100 locomotives.
c.) assert the foundational requirements of a constitutional republic to move forward,
"Sure, you can spy on people, pass a constitutional amendment." There was a reason that the constitution was only mutable by co-opting populace
4.) Need to recognize what is wrong, it's not a single issue--I find it ironic that the NRA has been silent during this debacle as an example of why answering an issue like this is so difficult--people usually only serve their own narrow interests--well people--here is the result.
5.) Hold strong, forcible address (I mean with constructs in logic) the intelligentsia occupying D.C. needs removing. This is not to create victims, but we need to rid ourselves of the problem(s). We are not being served by our process, representatives, government, corporations, and our own ignorance. WE, THE PEOPLE, NEED TO BE JUST THAT...
6.) HERE IS A LIST OF ACTION ITEMS RELATED TO OUR REPUBLIC
a.) Break the parties, the DNC and the RNC have managed to focus power that subverts the representative principles of our governance. With modern tools, the compression of Geo-physical
boundaries allows small groups to affect systems that are not operative at scale. For example,
the house of representatives should be over a 1000 seats. The congress has managed not to add a seat for over a hundred years, that is purposeful--and it describes the issue that is what happens to power
when it finds a comfortable place to sit.
b.) Election reform--period. Money is not speech, money is power.
c.) The supreme court may need to stand--there may need to be a real reckoning
d.) The disbanding of the whole military industrial complex (this is repleat with cronism)
e.) Seizing all the national assets--not like FDR. We have plutocrats paying favors to their true
constituents--you're not one of them.
f,) Convene a "Constitutional, Continental Congress, just as they did in the 18th century.
Schneier.com is a personal website. Opinions expressed are not necessarily those of Co3 Systems, Inc.