Schneier on Security
A blog covering security and security technology.
« New RC4 Attack |
| Friday Squid Blogging: Bomb Discovered in Squid at Market »
March 29, 2013
The Dangers of Surveillance
Interesting article, "The Dangers of Surveillance," by Neil M. Richards, Harvard Law Review, 2013. From the abstract:
....We need a better account of the dangers of surveillance.
This article offers such an account. Drawing on law, history, literature, and the work of scholars in the emerging interdisciplinary field of "surveillance studies," I explain what those harms are and why they matter. At the level of theory, I explain when surveillance is particularly dangerous, and when it is not. Surveillance is harmful because it can chill the exercise of our civil liberties, especially our intellectual privacy. It is also gives the watcher power over the watched, creating the the risk of a variety of other harms, such as discrimination, coercion, and the threat of selective enforcement, where critics of the government can be prosecuted or blackmailed for wrongdoing unrelated to the purpose of the surveillance.
At a practical level, I propose a set of four principles that should guide the future development of surveillance law, allowing for a more appropriate balance between the costs and benefits of government surveillance. First, we must recognize that surveillance transcends the public-private divide. Even if we are ultimately more concerned with government surveillance, any solution must grapple with the complex relationships between government and corporate watchers. Second, we must recognize that secret surveillance is illegitimate, and prohibit the creation of any domestic surveillance programs whose existence is secret. Third, we should recognize that total surveillance is illegitimate and reject the idea that it is acceptable for the government to record all Internet activity without authorization. Fourth, we must recognize that surveillance is harmful. Surveillance menaces intellectual privacy and increases the risk of blackmail, coercion, and discrimination; accordingly, we must recognize surveillance as a harm in constitutional standing doctrine.
EDITED TO ADD (4/12): Reply to the article.
Posted on March 29, 2013 at 12:25 PM
• 27 Comments
To receive these entries once a month by e-mail, sign up for the Crypto-Gram Newsletter.
As long as those who control the surveillance have no downside risk, and only benefits to be gained, it will expand and continue.
With the current legal environment, it might take a surveillance "Stand Your Ground" law to turn it around.
Surveillance, more specifically acts carried out by officials on persons without proper legal standing, is an illegal act. I, as a private citizen, cannot endlessly trail behind someone day and night, I'd be guilty of stalking. There is no inherent right of the government to stalk citizens (and quite possible persons) just because the government has the capability. There is another issue regarding prima facia, evidence or data collected by "authorities" must be testable, and not just by a judge, but by a jury as well. If the government is the accuser and the prosecuted then the balance and subjective nature of the evidence comes into question. The United States government has lost the rationale basis for prosecution, not just by tepid reasoning but by the false assumption that it is the government that must protect itself from the I consenting governed. It is by virtue of the people, the suspect, that the government is given any weight in respecting the person/individual. It's asking the rape victim to consent to being guilty of inducing the act and denying the production of evidence at trail. "Just trust us, you're guilty of involuntarily F'ing yourself."
Lord, but law professors can be naive. Short version: Surveillance is bad for a bunch of obvious reasons, so the law ought to like, minimize these harms.
That's worked out pretty well for the last 250 years, hasn't it? We might as well just scream "read the constitution" even louder. New laws aren't going to help if the old ones haven't.
One problem is that there is a widespread, uncritically accepted assertion that those performing the surveillance (police and intelligence operatives) are trustworthy and would not abuse their power, at least most of the time (we all know there are exceptions, but we deal with those on a case by case basis when forced to, and it never occurs to us that they might indicate a fundamental problem with what we're doing or how we go about it).
Jo - my point in writing this article is what while the dangers of surveillance are obvious to many people, they aren't obvious to others. In fact, courts reviewing challenges to surveillance frequently hold that the harms or dangers of surveillance are too vague to succeed in court. The paper tries to show clearly and in a way that our law can comprehend exactly what those dangers are, and why they matter. If we can't do that, they maybe we don't deserve privacy. I think we deserve privacy, and try to show why.
Any chance you might condemn BT's secret deployment of Phorm's mass surveillance technology in 2006, 2007, and 2008?
Or is that still a taboo topic for BT directors with an interest in telecom security topics?
Since you haven't mentioned this yet, here is one where your "favorite" purveyors of security theater scored an own goal
NY Post article
Quick summary - TSA agent picks up object from floor. Claims he thought it was a laser pointer. Pointed it at fellow officers, and pushed the button. It was actually a can of pepper spray. "Brain" and 5 fellow officers to the local hospital. Kennedy screening shut down for 15 minutes. At least no passengers were hit.
Society took a very very wrong turn when we saw 'free' surveillance supported services appearing on the web and said "Oh Cool" instead of "Stop that, you disgusting fools".
So cheaply bought. It's sad.
The scary thing about all the data that is available about everyone today is that you have to worry not just about current governments but also any future government.
Any laws that are passed won't help here. Imagine what a totalitarian regime might do with all the information available on facebook and google.
In the US, anyway, there is enough evidence to suggest that there is plausibly a very wide net of illegal surveillance going on, on US citizens.
It is immoral and it is illegal. Even if there are loopholes in laws on the books that would suggest otherwise. This is just common sense. The reason why these guys would be so secretive is because they know it is highly wrong.
The issue is also that they know how to use that surveillance. They know how to use it so those "in the know" won't talk. This is typically maintained by a very strong understanding of functional blackmail.
If you mess up with blackmail and do it wrongly, the victim may go public. If you do it right, it is a subtle message. It says "you have good reason to believe that the people you work for have information they could use against you which can harm you seriously but you can not prove this".
That manner of blackmail keeps people in check, it keeps them scared, but it is performed in a very subtle manner so they they can not prove they are being threatened with blackmail. They are just left with a gawdawful fear.
Surveillance, of course, enables that capacity for blackmail. Just as it also enables a network of people in authority to work together in secret while doing Very Bad Things.
And it is all about power. Human beings are power hungry creatures. Money is power for them. Knowledge is power for them. Useful knowledge about businesses is knowledge that translates into money. Money buys people and buys more power.
And it is all about power. Human beings are power hungry creatures. Money is power for them Knowledge is power for them. Useful knowledge about businesses is knowledge that translates into about businesses is knowledge that translates into money. Money buys people and buys more power. money. Money buys people and buys more power
Actualy not quite...
The ultimate goal is status, and it is this that power buys.
It's important to understand this because it leads to some quite perverse behaviour.
When you earn more than 20million USD a year or have assets that provide an after tax profit in the tens of millions per annum, there is little you can do with it in the ordiinary sense to diistinquishh your level of wealth above somebody else in that income range. You've probably bought a hoouse or two in the hamptons got yourself various uniformed servents etc who Kow-Tow and tuch tthe forelock etc.
The kind of Status these people crave starts when the sittiing US President come to their home cap in hand for campaign funds and sits there politely looking fawningly at you for an hour whilst you tell him and his advisors how he is not doing enough for the wealth creators and thus destroying Americans and their way of life....
But where do you go on the status scale after that?
Well as in the Koch brothers you then buy up regional politicians to enforce your political view point, you start and fund patriotic sounding organisations the sole intent of which is to brainwash the gullible into signing away their collective rights and thus their money, power and status in the name of the American Dream.
But even this is not enough. You find there is a mental divide which you could call "the status gap" where by those with status will do what they can to not only prevent others from gaining status, but will push for legislation to ensure the sstatus gap, with the ultimate goal of "surfdom" or a "cast system" for the population.
Thus if you come up with a scheme that would make thhe ultra wealthy 400 US citizens ten times richer but in the process increase the status of the average US citizen they would shun it and you with vengence. If however you came up with a system that halved their wealth but made the average US citizen from comfortably off to the bread line or worse thus significantly opening the status gap you would have a very very atentive audience in the 400.
These people crave the ancient view of the Baronial estate with vilains and surfs tied to the land in a position of utter subservience where the status of individuals can be seen at a glance by the cloths they are forced to wear and how they stand and bow their heads etc.
"The ultimate goal is status, and it is this that power buys."
Well, I see status as power, though. But, yes, your definition is very good and important.
And statements like "money is power" and "knowledge is power" do irk me.
House of Cards: "Such a waste of talent. He chose money over power, in this town a mistake nearly everyone makes. Money is the McMansion in Sarasota that starts falling apart after ten years, power is the old stone building that stands for centuries. I can not respect someone who does not see the difference."
I did not finish reading the article when posting, but now that I have, the article does go much more into "how surveillance is power". I did not even read the part about extortion, and was surprised to see how they found many excellent examples.
I was reading in Wired the other day how a researcher was able to profile top far left extremists and rightists. While I dislike those extremes, I did come away feeling, "Am I the only one creeped out about this? But who can say they are, because they would then appear to be an extremist."
However, the article well points out how surveillance can severely dampen intellectual freedom. Basically, you take the promise of the open internet, open society, open government, open economy and use it all against its' self to destroy it.
Right now, in the "free" world, we are in a quandry. We have secret surveillance on citizens and we know it and there is nothing we can do about it because it is secret. Same thing in Europe.
We do not know how much. We get some numbers that would seem to indicate it is very limited. But, then some scare stories come out that seem to say it is not limited at all. Like with the NSA plant in Utah, or the Verizon bugging issue.
Only value add I have there is: just because a lot of traffic is encrypted, if the NSA is scooping up all data from the telecommunications networks does not mean all of it is.
A substantial amount is not, enough to get into those encrypted accounts, or to mitm and get a trusted root certificate on any target system. Not that there are not countless other ways to get around encryption.
So encryption its' self makes people feel complacent. More free to talk. Because, as once well said by Captain Flapjack, "you don't want the person you are spying on know you are spying on them -- that way you can see them doing things".
Though many levels of extortion with such systems involve rearing that ugly head and making people see they are being spied on. Instant control. They stop doing things. Including talking or making rival power plays.
Or, as that article points out, extortion from surveillance data can be used to control politics. Business. Anything and anyone just about.
I find, then, that these people find themselves in a state of true, deep, utter unbelief. They are the Watchers. They are the gods. They are the masters of the universe with no God, no angels, no Heaven, no Hell. Just them. Masters of the universe.
Prime, Grade A villains ripe for the takedown. :-)
The theory of pervasive surveillance and its effects has been long known. Most famously described by Jeremy Bentham and by Michel Foucault. As in his theory of Utilitarianism, Bentham was negligent in the details.. in particular, who gets to define "rogue" and "honest".
As the Western Hemisphere Institute for Security Cooperation (formerly US Army School of the Americas) has taught for generations, you don't need to go out of your way to find Communists or Muslims to murder and torture. You just need to find somebody, anybody, as long as you leave the body in a public place and arrange for a proper denouncement in the six o'clock infotainment. Not all of the graduates are foreigners.
The theory of pervasive surveillance and its effects has been long known. Most famously described by Jeremy Bentham ...
It is interesting that you should mention Jeremy Bentham, and his Panopticon both Bruce and I have met him in person as it were .
And I have on my wall at home a line drawing of what the Panopticon would have looked like, and beneath it a plaque with Benthams own words,
"The Panopticon a new mode of obtaining power of mind over mind, in a quantity hitherto without example."
Just to remind me that in all the best of intentions there is often true evil at work.
 Although Jermy Bentham died he was never buried his will instructed that he was to be disected and turned into an Auto-Icon. His skeleton and head were to form the core of the Icon with the flesh replaced with straw to pad out to a body. Unfortunatly the process of preserving the head produced a somewhat macabre sight so a replacment wax head was made. The Icon was aquired by UCL where it is still on display and for some time the original head was kept with it. Unfortunatly it became subject to student pranks and such like and is now kept locked up in a safe and is brought out occasionaly (not sure if Bruce has actualy been shown the real head).
Why do I get the feeling that the next time surveillance laws are going to be tightened in the US will be when there are enough google glasses in public to begin to record events the powers that be would remain unseen?
Gill: Any laws that are passed won't help here. Imagine what a totalitarian regime might do with all the information available on facebook and google.
Totalitarian regimes tend to self-destruct and work badly with technology. I'm not too worried about North Korea or the Taliban interacting with technology. The real worries are China and Singapore.
Also I think it's pretty likely that Chinese Ministry of State Security does have most of what they want to know from facebook and google. They have been extremely active at phishing and cracking, and given that most people are pretty lax about passwords, I think its likely that if you have a facebook or gmail account and what you have is of interest to the Chinese government that you've been hacked already (and I write my e-mail with the assumption that both the Chinese government and the NSA have access to it.)
The problem with the network world is that you are dependent on not only your privacy but those of your friends. If you are part of a dissident group, then you might have good password security, but all you need is a friend on facebook or gmail that has a weak password, and the Chinese government can read what you've sent to them. This makes it impossible to create a dissident group of any size which is the point.
I think that people worry too much about a totalitarian hell...... That doesn't mean that there isn't something to worry about, but it's that the worry is not totalitarianism.....
The thing about China is that it's really not a nightmare to live in. Yes, the police can read all of your e-mail, but they've figured out that if they arrest everyone that's annoyed at the government, they this means that there are less likely to stay in power. So the authorities allow and even encourage people to have some amount of freedom online.
The trouble is not that it's a hell, it's that for someone that has been able to work outside the firewall, it's annoying, and what's worse, most people inside the firewall don't notice why it's a bad thing.
So what I'm worried about is not so much "1984" but "Brave New World." It's not that all of this surveillance and privacy will lead us to some horrible hell, but rather that it will lead to a world in which I would find annoying, but in which people who live in that world would find "normal."
I'm old enough to remember a world in which if you didn't like your situation, you could move to a new city in which no one knew you, and you could start a life in which no one from your past could reach you easily. This was the way things were in 1985. Today, you can't "disappear.' However people who have grown up with facebook and google would find that "weird."
So it's not that I'm worried that people will to horrible things with the loss of privacy. I'm more worried that we will end up in a "surveillance society" in which most people just consider it normal.
One other thing is that I think there is too much of a focus on state surveillance and not the consequences of private surveillance. I'm not worried about the NSA having access to my gmail account or reading what I say on facebook. I am worried about my manager or present and future employers having access to my gmail account and on reading what I say on facebook.
In some ways that makes "secret surveillance" better. If the NSA has a secret survelliance program then they can't let my boss know any of the information without divulging the secret and that's fine by me.
The other thing is that having some experience with China makes me less worried about the United States. There are things that the Ministry of State Security can do with the information that they have that the CIA or NSA just can't. Part of it is that the MSS has no restrictions on letting people know how they got the information. If the MSS arrests you on a charge of state subversion, they will not have any legal problems saying that they got that information as a result of their internet monitoring. The fact that the NSA can't publicly reveal that they have a monitoring system greatly restricts what they can do with that information.
The fact that the NSA can't publicly reveal that they have a monitoring system greatly restricts what they can do with that information.
Have a look at what the UK Gov are currently trying to bring in in the way of secret courts.
Put overly simply neither the defendent or the defendents chosen representative are alowed to see the evidence against them.
Only a specialy trained and apointed and aproved judge and legal personel can see it.
Thus you have the Cardinal Richelieu attack of "If one would give me six lines written by the hand of the most honest man, I would find something in them to have him hanged" with the extra problem of you not being able to know what those six lines are so niether you or your chosen representative can either put them in context or refute them.
Good luck on not getting hanged with that sort of system, because there are no checks and balances and no accountable (to the accused) over sight.
There is a classical capability/intentions strategic argument in this area.
As citizens, we not only have to worry about the intentions of the current government, but also any future government. Given that our knowledge of the intentions of any current administration is limited - look to the ample evidence of governmental misbehavior in history - this should worry anyone.
On that basis, we should logically resist at all costs the creation of surveillance infrastructures, and where it is absolutely unavoidable, put strong governance controls in place backed by well studied incentive analysis to ensure that collected data is minimal and destroyed as soon as possible. Incentive structures should ensure that the systems fail open, as opposed to fail closed, as almost all police and intelligence watchdog systems currently do now.
What would that look like? It would look like a system where another government department tasked and incentivized with revealing intelligence operations should, within six months, reveal the intelligence gathering and content gathered to the target. The process for preventing this should be onerous, judicially reviewed (publicly visible via statistics), increasing in complexity the more times utilized, and with criminal penalties for abuse or incompetence. Subverting these processes should be a crime which carries a 10-15 year sentence for all police or intelligence officers involved, and their immediate management.
FYI, something similar to this was proposed in the late 90's in the Australian Attorney-General's department by Orlowski. It didn't carry the penalties, but it did notify the targets of surveillance within six months unless a judge authorized another six months. I am sure it will surprise no-one that this sank without a trace.
Anyone have one of those new-fangled URI thingies for the actual paper? That contraption of a "web application" seems to have the primary purpose of preventing access to the actual content ... at least my browser seems to be incapable of extracting the paper from their site.
Y'all might also want to look at an old story of Isaac Asimov's, The Dead Past. It takes on some of these issues as well.
"The fact that the NSA can't publicly reveal that they have a monitoring system greatly restricts what they can do with that information."
This has *a* limiting factor, but it also forces them to operate in a sophisticated manner and become better users of surveillance.
A study of now open source material on surveillance and counter-surveillance during WWII between Allies and Axis powers radio communications well highlights the many options available where you want to keep surveillance secret.
There are simply an enormous number of tactics available to the surveillor even if they must keep the fact that they are surveilling a target secret.
For citizen surveillance, the tactics are even wider because the surveilling agency *can* reveal to the citizen they are being surveilled if they find it useful for them to do so: they simply have to do so in a manner the citizen can not prove they are doing it.
There are many ways to do this.
We must remember Eric Schmidt response to the Google Street views privacy invasion: "Complete privacy does not exist in this world except in a desert, and anyone who is not a hermit must expect and endure the ordinary incidents of the community life of which he (or she) is a part."
At some point he said that if you are keeping things private you are doing something wrong- he was talking about you being morally wrong.
Along with Mark Zuckenberg, Eric, and our own blissful ignorance is our biggest worries today.
@joequant • March 31, 2013 6:45 AM
"One other thing is that I think there is too much of a focus on state surveillance and not the consequences of private surveillance. I'm not worried about the NSA having access to my gmail account or reading what I say on facebook. I am worried about my manager or present and future employers having access to my gmail account and on reading what I say on facebook".
Good point, but I am worried about Government secretly providing managers, present and future employers, landlords, universities, banks, insurance companies, etc. with information obtained by State surveillance which may negatively affect your life or just make it very difficult.
You'll never ever find out where bad wind is blowing from, but they could make your life miserable because of their own definition (contrary to Constitution and Bill of Rights) of disloyalty (which is basically in many cases is just dissent) or patriotism (they may consider any critical statement even constaructive with non-viloent suggestions how to fix obvious problem as extrimism) or in general passing Government illegal blacklisting down to the non-Government sector.
Schneier.com is a personal website. Opinions expressed are not necessarily those of Co3 Systems, Inc.