XKeyscore

The Guardian discusses a new secret NSA program: XKeyscore. It’s the desktop system that allows NSA agents to spy on anyone over the Internet in real time. It searches existing NSA databases—presumably including PRISM—and can create fingerprints to search for all future data collections from systems like TRAFFIC THIEF. This seems to be what Edward Snowden meant when he said that he had the ability to spy on any American, in real time, from his deck.

In related news, this essay explains how “three-hop” analysis of the communications of suspected terrorists means that everyone in the US is spied on.

EDITED TO ADD (8/3): The math is wrong in that three-hop analysis essay. Apologies.

Tags: , , , , ,

Posted on August 2, 2013 at 3:20 PM • 31 Comments

Comments

John Hardin • August 2, 2013 5:33 PM

@Dave Kearns:

the limited ability to search a database is neither spying, nor “real time”.

…that depends on where data is captured from, and how quickly captured data is added to the database…

Figureitout • August 2, 2013 5:56 PM

from his deck.
Bruce/Dave Aronson
–The NSA likes to have “Surveillance Cookouts”. Only those w/ Top Secret security clearances can attend and everyone just eats burgers in silence w/ headphones on and communicates w/ sign language.

Peter • August 2, 2013 8:17 PM

As far as we know at the moment, PRISM is not a database, but a data collection operation, and the collected data go in other, longer extisting databases, like Mainway, Marina, and Pinwale. These can be searched by XKeyscore.

Indeed it seems that Snowden based his claims on these programs, but it also seems to be he did this only after reading the slides without much knowledge about how these programs are actually used (remember he was a sysadmin, not a sigint analyst). The use of these programs is limited to people who need them for their assigned tasks, within approved collection plans.

Also searching NSA databases is limited with a need to know and for that, access to the databases is strictly limited. This can be read for example in the NSA inspector general’s report about the President’s Surveillance Program, which was also published by The Guardian.

Tim! • August 2, 2013 8:30 PM

in real time, from his deck.

Clearly Edward Snowden is a Neuromancer and I am shocked (shocked!) anyone here thought of the wooden thing attached to a house first.

Shachar Shemesh • August 2, 2013 8:33 PM

@peter

It is possible that everything you say is true. I see no reason to doubt you. Then again, it is possible it is not.

The thing is, and this is the pivot of the entire criticism against these programs, it should not have been left for more-or-less anonymous commentators to defend these programs. Even if you cannot disclose the specifics of what’s been done with these databases, their mere existence, as well as general idea of what’s been done with them, should have been known to the public from day 1.

If everything is mandated and regulated, how come the NSA directors felt the need to lie about it? There are two options I can think of, and neither one of them bodes well for the NSA.

  1. Either this is still not the whole story, and there are yet more intrusive systems and/or uses the public has still not heard about, in which case we still don’t have enough of the picture to form an informed opinion, or
  2. The NSA directors believe that the American public, and in particular, congress, works for them, instead of the other way around. As such, they believe that they have the right to decide on whatever spying programs they want, effectively without supervision.

The problem with 2 is that we’ve seen it all too often in the civilian world where security is concerned. The general rule of thumb when doing, for example, pen-testing, is that the more sure a client is that their network is secure, the less secure it actually is.

When you see people talk about all of these checks on unauthorized access, I always ask the same question: how many people were prosecuted for violating the access restrictions? If none, then that, to me, is proof that the restrictions are not working.

Shachar

Lisa • August 3, 2013 8:19 AM

@Bruce, is the math in the essay wrong?

With approx. 40 people called per person, to be 3 hops away would be:

40^3=64,000

Not:

40^4=2,560,000 (for 4 hops away from terrorist)

Correct? After all one hop away from a terrorist would be 40=40^1 people, not 1,600=40^2 people, right?

unknown.soldier • August 3, 2013 10:30 AM

Senator Lindsey Graham noted that for those not talking to terrorists on the phone, We dont have anything to worry about.

These senators are all paid by the defense industry. They are also all under surveillance and know it.

No they don’t fear people who have no compunction with wiretapping everything they do and who also mass produce assassin drones. :/

Funny thing is they do not speak for the people. He has everything to worry about. He doesn’t want to end up like Patraeus or Clinton.

I also find it very often alarming and highly ironic when I hear these saps – be they politicians, or whatever other ass kissers – talk about how “we” “have nothing to worry about”. Like they have any clue. They have no idea who is behind the camera and gun. They have no idea on what the technology is. They can barely figure out how to make a decent tasting breakfast.

SparkyGSX • August 3, 2013 11:02 AM

@Lisa: I think the suspected terrorist would be the zero’th hop, so someone who was called by the suspected terrorist would be the first hop; on average, there would be 40 such first hops for every suspect.

The author is rather obviously “statistically challenged”; for starters, the link back to the previous hop or original suspect is counted twice. Besides that, it also implicitly assumes there won’t be any duplicated within all of the contacts from the different hops. For any group of closely connected people, there will be a lot of duplicate contacts (probably approaching 50%). He even assumes the entire population of the US could be neatly divided into groups of 2.5 million people each, where every single person is only connected to a single suspect through 3 hops or less, with no duplicate connections at all.

On the other hand, the list of “suspects” is sufficiently huge to cover the whole population anyway, especially if companies would be considered valids “hops”. A single suspect and single telemarketer plus 2 other hops would probably cover the entire population within the area of operation of the telemarketer.

I wonder what the definition of “suspected terrorist” would be. Apparently government protestors, journalists, 2-year old children who won’t share toys, or somebody they don’t particularly like for any reason whatsoever can be designated a “suspected terrorist”. Does anybody know of some kind of definition from the NSA?

Also, I’d like to thank the author of the article for obviously not giving a rat’s ass about NSA spying on the rest of the world.

Paul Waterman • August 3, 2013 12:26 PM

Regarding the math: Yes, it’s wrong. But the article Bruce quoted it from was quoting it from another article. So now we know that bad math, like the NSA, can propagate at least two hops. 🙂

Here’s how the math works:

Terrorist = 1 person monitored

First hop is to terrorist’s 40 unique contacts = 40 new people added. Now 41 people are being monitored.

Second hop is to the first hop people’s 40 unique contacts = 1600 new people added. Now 1641 people are being monitored.

Third hop is to to the second hop people’s 40 unique contacts = 64000 new people added. Now 65641 people are being monitored.

CallMeLateForSupper • August 3, 2013 2:33 PM

@unknown.soldier
Senator Graham said people who don’t talk to terrorists on the phone have nothing to worry about? (I don’t know because the darkpolitricks article doesn’t load for me. Maybe because I have cookies and Javascript turned off.) If Graham did say that, then he obviously overlooked the fact that talking to someone who has talked to a terrorist (2nd hop), and talking to someone who has talked to someone who has talked to a terrorist (3rd hop), puts one under NSA screw-tin-y.

Bruce Schneier • August 3, 2013 3:27 PM

“‘from his deck’? Desk? Or was he able to do this at home, like that former Director of Central Intelligence?”

Typo fixed. Thank you.

guruphil • August 3, 2013 3:38 PM

I sure hope Rachel Fromcardservices isn’t a terrorist or within 2 hops of a terrorist.

not-so-anonymous coward • August 3, 2013 6:00 PM

Also, I’d like to thank the author of the article for
obviously not giving a rat’s ass about NSA spying on the
rest of the world.

Yes, that bugs me too. Killing forigners is ok as long as they are brown, male between 14 and 85 years old, or muslim, and spying on anyone else on this planet as long as they aren’t US citizens.

Fuck ALL of you, yanks. Welcome to fascism.

Wesley Parish • August 3, 2013 6:49 PM

True, @not-so-anonymous coward. It bugs me that the NSA don’t appear to count themselves in the three-hop analysis.

But consider this: an NSA analyst gets through to the “French Tourist” he is tracking, planning some terrible act of “French Tourism” a la the Rainbow Warrior bombing, except these “French Tourists” happen to be “brown” ie “black“, not “white” as in the case of the Rainbow Warrior and therefore are as criminally liable as the US did not hold the French government … /digression: and once the NSA analyst has finished his work on the “French Tourist” he then goes to work on you.

At that point you are now well within three degrees of that “French Tourist“, as indeed the NSA analyst and his supervisor are … inadvertently, of course, but that has never stopped the dorks before.

unknown.soldier • August 3, 2013 8:28 PM

not-so-anonymous coward
Yes, that bugs me too. Killing forigners is ok as long as they are brown, male between 14 and 85 years old, or muslim, and spying on anyone else on this planet as long as they aren’t US citizens.
Fuck ALL of you, yanks. Welcome to fascism.

UK is as bad or worse then the US in all of this. In very many ways, worse. Though, the US is far more dangerous.

I have my camp in the US (and a few other countries, lol) — but like at least some Americans, we have nothing to do with this crap.

It is fascist, it is tyrannical, everything bad.

Clear betrayal of the principles of freedom and law.

“History” (I prefer the term “judgment day”) won’t view these people kindly.

unknown.soldier • August 3, 2013 8:41 PM

@CallMeLateForSupper

If Graham did say that, then he obviously overlooked the fact that talking to someone who has talked to a terrorist (2nd hop), and talking to someone who has talked to someone who has talked to a terrorist (3rd hop), puts one under NSA screw-tin-y.

Yeah.

Though I am sure there are more compartmentalized groups in the government who surveil the hell out of innocent Americans without any kind of safeguard or liability whatsoever.

Of course, as some have pointed out, if you are not American, you are open season. Terrorist, foreign spy, political VIP, of course, as it was intended. But, also anyone. If you have a view they disagree with. If you have a company or product they want. Doesn’t matter.

It is not like they have any kind of real compassion or moral grounding.

People who do wouldn’t set up these kinds of systems. Even a child can understand there is no way to control these systems, and even if everyone was on the ‘up and up’, they have zero guarantee of who will be in power tomorrow.

Peter • August 3, 2013 8:45 PM

@ Shachar

I fully agree with you that NSA should be much more open about things they are doing and what checks and balances the exactly have, etc. Most of the information we learned from the recent disclosures was hardly a real threat for the US national security, but gave us a far better understanding of what NSA is doing (if you read the material carefully by yourself, not relying on the papers).

The reason why government officials aren’t telling these things, is, I guess, because they simply think the less they tell, the better it is for doing their job (keeping the enemy as uninformed as possible). So I don’t think they deliberatly want to hide misbehaviour, but that it’s the way government officials have learned to think and speak. I think this isn’t a very smart way, as it undermines the public trust in their organisations and they could easily be more open without threatening national security.

I don’t know about people being prosecuted for violating internal NSA restrictions, but I also don’t know if such violations will lead to a prosecution, or that it only leads to internal punishment or the loss of the job. This of course has to be proportionate to the kind of violation.

Clive Robinson • August 4, 2013 6:33 AM

Hey guys cool it…

The use of four letter words might be expresive, but it’s going to trip many work place filters, which is not good.

As for who spys on who and when and how, it boils down to

1, Anyone who have got or can by the technical chops,
2, Will spy on everyone one way or another,
3, As frequently and as well as technology alows,
4, Which now nolonger needs to be bespoke but COTS.

As for the WASP Nations the big five they are,

1, America (USA)
2, Australia (AUZ)
3, Britain (GBR/UK)
4, Canada
5, New Zeland (NZ)

In all cases these “Special Relationship” countries politicians have absolutly no clue as to what the agencies concerned are upto and in most cases the relationship is “Don’t ask, won’t say what you don’t want to know”.

It is safe to assume any nation with any kind of technical ability (all of Europe, Asia) with more than 20million citizens is at it. Further it is safe to assume that any country that does not have the technical ability can buy it off the shelf “no questions asked”. Thus that covers just about all nations on this planet with any kind of international trade.

This has to my knowledge been the state of play since the 1970’s to today. What is new is the free flow of information and technology between States and Corporates, prior to the 1990’s it was with “mil companies” but nolonger. Currently the mil companies can’t deliver in the way that marketing and advertising companies can and due to competition can do it at better rates than gov agencies.

Also in most cases the gov agencies don’t have the monopoly on grey matter they used to have. It’s now common for “bright young things” to turn their maths etc PhD into a business and be taking very large chunks of money from either financial companies or the government.

To “get in on the act” the traditional Mil-Ind entities are in effect turning themselves into venture capatalists for these “bright young things”. However some of the “bright young things” are realising the relationship has unwelcom constraints, the cage might be guilded but you cann’t spread your wings as far and wide. Thus others are totaly “freelance” and will sell to who ever has the money to buy.

Hence we have “weaponized zero day” hacks, complex analytical engines munching “big data” and comms monitoring / control kit being sold near and far by companies with only a handfull or so of shareholders/founders with nation state only customers providing very high returns, and darn few morals or scruples in sight.

We passed the tipping point on mass surveilance back in the 1990’s and like pandoras box what has come out technicaly cannot be put back.

Technicaly none of this was in question prior to Ed Snowdens revelations, it was all comfortingly deniable as conspiracy or ravings when it came to “our government” because codewords and specific details were not known. Thus people could chose not to believe it or claim it was only foreign despots, but since the “Death by powerpoint” slide shows have been seen the codewords are now know as are a some of the specific details and now suddenly for the man in the street it’s “real” and the sleeping giant is stiring and a reconing is on the cards.

Jason • August 4, 2013 7:40 AM

While the math may be wrong, it’s still alarming. If a 3-hop analysis covers “only” 64,000 people for each potential terrorist, then if there are 5000 suspected terrorists, you still cover everybody.

The calling network contains mainly people over the age of 12, though, so that lowers the number of terrorists a fair bit too.

The usual caveats about assuming an even distribution of terrorists in the call network apply.

Doug Coulter • August 4, 2013 9:17 PM

+zillion, Clive. You hit that nail so squarely on the head I’m sure you drove it home in one blow. Anyone who is or has been in “the community” knows it, hell, it was a twinkle in their eye when I was, lo, these 4 decades ago.

The big deal (and a good thing IMO) is that now it’s not just us who knows it now. Maybe someone will now be kind enough to remove the tinfoil hat they put on me earlier.

Nathanael • August 5, 2013 5:16 AM

The math is more alarming than you think.

Social networks are actually scale-free networks, like the Internet. The first hop gets you 40 people. But one of those people is a social butterfly. That person calls 200 people. One of those people is even better connected — perhaps a politician. That person calls thousands of people.

And you see the problem.

It seems inevitable that the NSA is spying on all our Congressmen. And that’s a problem. Spying on me? Unless they just decide to harass me for no reason, why bother? I’m harmless.

Spying on my Congressman? That is another matter. And hopefully the Congressmen will begin to realize the danger involved in that, and will shut it all down with extreme prejudice, which they do still have the power to do.

Apparently just under half of them realize that they need to, so far, unfortunately.

Doug Coulter • August 5, 2013 10:28 AM

Nathanael…
Surely even the most brain-dead bureaucrat long ago figured out that the easiest to blackmail people on earth are our elected representatives. They are the ones who are dirty, and who care.
I don’t care what they know about me, in general – I’m not easy to blackmail as my own life is an open book and I have nothing to lose by people knowing what I do. But think about our 100% dirty “on the take” congress for a second.

The fact that cutting these agencies never, ever, comes up in a time when budgets must be cut is the loudest “dog that didn’t bark” of all time. A tap on the shoulder is all it takes – you want us to reveal (underage girl/boy)(bribe)(nepotism)? And they go along like good little doggies.

I don’t think most realize this – if we hadn’t lost our representative republic for other reasons, this would be the final nail in the coffin.

The frog is fully boiled, stick a fork in it.

Eivind Eklund • August 6, 2013 12:32 PM

Three hop analysis will cover approximately everybody. I’m a relatively normal programmer from Norway (though presently living in the US).

Excepting Ronald Reagan and George Bush Sr, I have multiple three hop connections that I know about to every US president since Jimmy Carter. I have a two hop connection to Ronald Reagen, and that’s the only connection I know for sure to George Bush Sr (though I think I might be able to dig up more).

I have less three hop connections to two known terrorists: Osama Bin Laden (through Ronald Reagan), and Anders Behring Breivik (through friend that was encountering Breivik in a business setting).

This is just from going about my life normally. I don’t fraternize with politicians or high end business people, or do much active network building.

It's in fucking des! • August 9, 2013 4:04 AM

Salted__‘∆çá∞%~õ—ܶ%WwO√Í %Æ«ÍU∫ùÙ@E‘(m»ÆSXدn’lªn

RSA (open ssl) public key • August 9, 2013 4:51 AM

—–BEGIN PUBLIC KEY—–
MIIBGzANBgkqhkiG9w0BAQEFAAOCAQgAMIIBAwKB+wDA7iXmORBMmxTxriXrIEum
VuHy8FdH5IXXVaAromVZXTEL7xJxwTsK6lnM54LQ4Zc/pSLf7de2NKqFuwhWR2od
JcXv54RUf1xa/3ByQ/y143sGMaspFOYSGbmIuXG/TQGLjv1vA1iKg+Z1upgP2j08
cMbPOl1E1RrGxszkoV0p0IyFSTodRBtnybaXayykRDlF4RHd63EN5ylhiYCcgYJ4
v8D9Wn6TFHrtCF9GGyVD6/y8t5sKGw6xIApKe2QkuifztaUrM0SsmsMcXN32m8FL
EJKu33uEsl+ohffzrg+xFoUjD24bP15ThlX8peSQpJyUAEiYKuNRhorfAgMBAAE=
—–END PUBLIC KEY—–

Leave a comment

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.