The NSA is Not Made of Magic

I am regularly asked what is the most surprising thing about the Snowden NSA documents. It’s this: the NSA is not made of magic. Its tools are no different from what we have in our world, it’s just better-funded. X-KEYSCORE is Bro plus memory. FOXACID is Metasploit with a budget. QUANTUM is AirPwn with a seriously privileged position on the backbone. The NSA breaks crypto not with super-secret cryptanalysis, but by using standard hacking tricks such as exploiting weak implementations and default keys. Its TAO implants are straightforward enhancements of attack tools developed by researchers, academics, and hackers; here’s a computer the size of a grain of rice, if you want to make your own such tools. The NSA’s collection and analysis tools are basically what you’d expect if you thought about it for a while.

That, fundamentally, is surprising. If you gave a super-secret Internet exploitation organization $10 billion annually, you’d expect some magic. And my guess is that there is some, around the edges, that has not become public yet. But that we haven’t seen any yet is cause for optimism.

Tags: , , , , , ,

Posted on May 21, 2014 at 3:29 PM • 64 Comments

Comments

CallMeLateForSupper • May 21, 2014 4:02 PM

@Bruce
“[…] that we haven’t seen any [magic] yet is cause for optimism.”

Yes, but cautious optimism. As a certain prickly Secretary liked to say, “Absence of evidence is not evidence of absence.” We should always keep in mind that many of the Snoden “trove” are dated years ago, thus they tell us what was, then, but not necessarily what is, now. We do not know the full depth and breadth of the capabilities of today’s NSA. Maybe Poitrus, Greenwald et al have proof of current magic; we can only wonder and wait for publication.

z • May 21, 2014 4:14 PM

The fact that they try so hard to backdoor cryptography is a sign that they don’t have any practical attacks on properly implemented, commonly used crypto. I think that’s a somewhat comforting thought. Of course, it immediately gets uncomfortable when you start wondering about what actually is backdoored.

Anura • May 21, 2014 4:20 PM

If the NSA had the capability to break AES, RSA, DH, 3DES, etc. then I don’t think they would be using it for mass surveillance. I think it would only be used for extremely sensitive matters and a much smaller group of people would know about it. The processing requirements would probably be too high anyway, like super-computer levels. The thing is, if you weaken the protocols and infiltrate the vendors, then super-secret cryptanalysis is simply not necessary for the bulk surveillance side of things.

Christian • May 21, 2014 4:26 PM

Nazi Germany also didn’t do magic. They just used well done logistics and economically best way to kill as many Jews, disabled ,Sinti, Romanies (…) as possible.

The shocking thing is not the way it is done. It is the size of the operation, the amount of people you need without any qualms doing immoral work and being able to justify it before themself!

Even you Bruce defended TAO as “the thing the NSA should do”, because it is targeted spying. But now with Greenwald’s book, we know that you can use this implants as well on the Cisco routers exported to many countries.

The outragous about the NSA is not the amount of tech! It is the amount of people behind it, working to change the world into surveillance nightmare.

Evan Harper • May 21, 2014 5:31 PM

It may be that magic simply isn’t necessary to accomplish the NSA’s missions. One of the key messages I’ve taken from your work over the years is that, as you said in your preface to Practical Cryptography, security is only as strong as the weakest link, and the mathematics of cryptography is almost never the weakest link. Would it be that surprising if the principle holds even for the highest-level top-secret government security?

Benni • May 21, 2014 5:40 PM

Somewhere during a discussion Michael Hayden said that fortunatly Snowden would not have access to the files describing “operation”. I wonder what Hayden meant by this. And I hope that Snowden was able to carry out some hacking….

Regarding the assertion of Bruce that this TAO would be a good thing: I think this is one of the worst statements Bruce has ever made. It all depends on your targets.

The TAO targets entire internet providers, or major internet services.

In these spiegel articles were sentences like:
http://www.spiegel.de/netzwelt/netzpolitik/neue-dokumente-der-geheime-werkzeugkasten-der-nsa-a-941153.html

NSA malware is used against international telecommunication providers.
or:

http://www.spiegel.de/international/world/nsa-secret-toolbox-ant-unit-offers-spy-gadgets-for-every-need-a-941006.html

Many of these digital tools are “remotely installable,” meaning they can be put in place over the Internet. Others, however, require direct intervention, known in NSA jargon as “interdiction.” This means that brand new products being delivered by mail are secretly intercepted, and hardware or software implants installed on them. The package is forwarded to its intended destination only after this has been done.

With these articles, sysadmins and telecommunication providers now have a better chance to find nsa malware and protect their customers from nsa bulk surveillance.

These details on the TAO catalogue were absolutlely vital, since Spiegel wrote TAO tools are used against entire internet providers.

4kjr3kjrn3kjn • May 21, 2014 6:02 PM

You don’t scout and typically acquire the most talented people produced by the human race(that mean the best international talent) and not have ‘magic’..

I think all their ‘magic’ is in big number theory and computer engineering.. The rest of the stuff is basically what the public has but on the backbone of the Internet..

I’m actually surprised they buy zero days though. I guess it’s less expensive than buying a lot of IDA licenses or making a better tool and having a building full of people grinding away at binary analysis..

Also, something is off with the entire Snowden saga. If you look at date references, it’s all dated data that is probably obsolete in implementation by now.. This all could EASILY be psyops or something else meant to misdirect..

Mr. Pragma • May 21, 2014 7:31 PM

I sometimes feel that this whole situation evolves in a rather unpleasant direction, in part by rabbit-confronted-by-snake fashion.

We have every reason to believe that cryptography — properly selected, implemented, and applied — still is very secure.
We also have reason to believe that cryptography isn’t helping a lot because usually it’s a very realiable door in a building riddled with open windows, unlocked or even open backdoors, and sometimes frighteningly thin cardboard walls.

Of course nsa basically has the same tools everyone has, just in an enhanced version.
After all they live in the same world we do. And being an agency it has — and uses — the classical state approach, a mixture of large budgets, legal means, secrecy – and an unhealthy dosis of “we are the good guys”.

Focussing the other side I wonder why the open-source people who, after all, are known to almost start internal wars for “the right cause” and to fervently create newer legal incarnations of “the right thing” aka gpl, never made a simple clause à la “This software must under no circumstances be used to in any way limit the freedom of humans to freely and securely communicate, exchange ideas, thoughts, information, and opinions of any kind”. (Forgive my lousy legalese. I’m not a lawer but you get the idea …).

Funny thing, isn’t it? The open-source Taliban wage wars on companies not properly linking to the gpl or keeping some source files closed — but they don’t seem to care batshit whether their code is bluntly and utterly abused to spy on people (incl themselves!), to imprison them, or to even kill them.

Their (gpl taliban) address is in the usa. Maybe even close to the nsa. Happenstance.

Mr. Pragma • May 21, 2014 7:43 PM

Christian (May 21, 2014 4:26 PM)

Kindly spare us the Nazi and jews crap. For one that is nothing to do with the current nsa/5eyes/bnd/whoever problem.

Moreover, while the americans do support a Neonazi junta in kiev (but certainly not because it’s a Neonazi junta but because it’s a complying slave junta), I find it gross to put americans next to Nazis.

There is indeed strong reason to critizise the americans but plainly insulting them or painting them as or similar to Nazis isn’t reasonable or helpful.

Jacob • May 21, 2014 8:28 PM

The power of the NSA is not in its tool set as much as in the legal infrastructure that they can use to club you senseless if you don’t comply.

And this is not only FISA – Federal courts, border guards, local police – any authority can and will be used against you until you and your information surrender.

Some months ago there was a story about a Canadian scholar who drove to the US, and on the way back the border guard demanded to let him examine the scholar’s computer. The Canadian asked him “by whose authority?” – and the guard pointed to his gun.

Much more alarming is yesterday’s Guardian story by Lavabit’s Levison. The legal straight jacket he was put in while trying to argue his case before the court was fit for a totalitarian regime more than a western democracy. When the state wields so much power, you don’t need sophisticated tools to break a comm channel.

http://www.theguardian.com/commentisfree/2014/may/20/why-did-lavabit-shut-down-snowden-email

Side note: In the Levison article there was a little noticed entry that, IMHO, can be used to legaly bypass the constitution:

“The government argued that, since the “inspection” of the data was to be carried out by a machine, they were exempt from the normal search-and-seizure protections of the Fourth Amendment”
Next we will see a machine that will inspect your stored private data, and based on an analysis by a clever algorithm will auto-command a drone to hit you. No constitutional protection. The Terminator hath cometh.

Magnus • May 21, 2014 9:23 PM

I would be very surprised if they didn’t have much better static source code analysis tools, and binary code disassemblers.

John Regehr did a couple of posts about whether static analysis could have caught Heartbleed, see blog.regehr.com/archives/1125 and 1128. Coverity

I expect that the NSA automatically monitors open source source control repositories (your githubs etc) and on New Year’s Day 2012 some analyst would have received a generated report that OpenSSL had just been updated and contains a ‘read from uninitialised memory’ bug (as well as many other issues). They would have had an exploit for it easily before anyone had integrated the update into their own systems.

Mr. Pragma • May 21, 2014 10:09 PM

DB

I do not know what the americans can become like, and I don’t need to because neither you nor me nor anyone else is to judge them for what they may or may not become.

Now they are not Nazis.

And, yes, there’s lots of outright criminal action by the us authorities and agencies. But that doesn’t make the americans Nazis.

Moreover, whatever the us government and agencies do — may they rot in hell for all that — that’s not “the americans”, that’s their government and agencies which is not the same.

Critizising is one thing. Putting 300 Mio individuals into one large bag and bashing them is a very different and unacceptable thing.

Mr. Pragma • May 21, 2014 11:22 PM

Harry Johnston (May 21, 2014 11:04 PM)

Well, choose your favorite or considered correct term.
I used the term junta in the often and widely used meaning of “unelected criminal gang”. And frankly, I’m more concerned about terrorized, maimed, and killed civilians than about political correctness and or perfect words.
As long as usa’s gangsters and murderers in kiev (like timochenko) end up with a bullet in their head, you may call that scum any way you please.

Mr Smegma • May 21, 2014 11:37 PM

I find the comparison between Nazis and NSA quite fair, both believe themselves above the law, above the rest of subhumans, thinking that they can do whatever they want with other people lives, overall an insult to humanity. Plus they have similar names, obviously not a coincidence.
I find unacceptable that I’m buying equipment and software intended to harm me one day if necessary, not to protect me as it should be.

Figureitout • May 21, 2014 11:59 PM

Jacob
–Yes, read that article when it came out. The story of Lavabit is one for the history books, and notice how he speaks of “agents” and how retarded the legal arguments become once you actually think about them… You should’ve posted it in the squid thread.

Mr. Pragma • May 22, 2014 12:14 AM

Mr Smegma (May 21, 2014 11:37 PM)

That’s not what he said. He said “the americans”.

While I wouldn’t use the term “Nazis”, I can quite well understand your point of view. Yes, the us government and many of its agencies are acting highly criminal, are known serial mass killers, and generally behave in abominable ways.

I’m not sure, however, that they treat their own citizens that much better …

yesme • May 22, 2014 12:40 AM

@Benni

“Somewhere during a discussion Michael Hayden said that fortunatly Snowden would not have access to the files describing “operation”.”

Wouldn’t that be the actual execution of the sheets that Snowden showed? For instance the “Belgacom job” where they monitored the EU in Bruxelles?

Or maybe the payment for the zero day bugs to some other organizations?

Just guessing here…

@Magnus

“I would be very surprised if they didn’t have much better static source code analysis tools, and binary code disassemblers.”

That’s what Bruce is talking about and the NSA probably just doesn’t have. Otherwise they wouldn’t have to buy these zero day bugs…

chrisl • May 22, 2014 1:11 AM

I’m not that surprised that they don’t have magic. I’ve worked on the civilian side of gov’t spaceflight for almost 20 years (and avoided needing a clearance that whole time) and have pretty high confidence that the laws of physics are the same for the both the civil side and the black world. I’ve worked civil programs with plenty of people who are cleared and also do black world stuff, and they’re all just as mortal as the rest of us. They may (or may not) have better funding, so they might get to some cool stuff sooner, but that world is also much more compartmentalized so it can lead to a lot more expense just working the interfaces because far fewer people on a project are allowed to know the overall goal. In some cases a project and its mission may be classified while much of the tech development isn’t, partly so they can take advantage of non-cleared experts. So it’s not surprising that it doesn’t appear to be any different for the NSA.

Nick P • May 22, 2014 1:43 AM

@ Bruce Schneier

“But that we haven’t seen any yet is cause for optimism.”

I see where you’re coming from. I wanted to think that, too. Yet, smart attackers almost always go for the most resource-effective attacks. That NSA uses well-funded versions of common attacks is actually a cause for pessimism: their continued success using (mostly) well-understood methods shows just how horribly wrong are mainstream approaches to security. That alternatives, even some on market, exist without such weaknesses adds an extra pessimistic element whereby most entities in the markets continue to invest in their own insecurity.

The NSA might not have magic powers, but the market ensures they meet their goals anyway.

Winter • May 22, 2014 2:51 AM

That, fundamentally, is surprising. If you gave a super-secret Internet exploitation organization $10 billion annually, you’d expect some magic. And my guess is that there is some, around the edges, that has not become public yet. But that we haven’t seen any yet is cause for optimism.

If you compare large research efforts, e.g., CERN, the Human Genome project, the Hubble telescope, or Japan’s Fifth generation computer program, you see that just throwing money at a problem is not the answer.

This is like the “Mythical man month”: Putting extra people on a software project that is over-time can make it lag even more behind. I think something like that holds for all large R&D projects.

I suspect that the NSA dependents on the open market for its fundamental research and prime technological development. Both of which are extremely difficult to sustain in isolation of the wider community.

Which is to say, that the NSA only implement existing knowledge and technology and will have only a marginal lead in isolated fields.

Christian • May 22, 2014 3:38 AM

Comparing Americans to Nazis is totally unfair!

Ask yourself: would a Winston Churchil rather support Putin against the USA because of their doing. And the answer is a clear: No!

The problem is we see signs that had similarities in Nazi Germany.

  • Secretive Police/Court/law interpretations that should not happen in a lawful state.
  • Torture Camps like Guantanamo Bay
  • Invading other countries on false pretense.
  • Military service is seen as a good deed for society

Adding the NSA and their doings, basically violating human rights on the entire planet.
It is still not Nazi Germany, but there are already way to many similarities.
Way weaker all of them and by far not as bad. But we have become more sensible with the time!

The USA is alienating itself from Europe and the rest of the world by this actions. Pretty much the same way Putin did it with putting troops onto the Krim.

Mr. Pragma • May 22, 2014 3:50 AM

Winter

Yes!

Let me drive this a little further.

As soon as a project has more than a couple of designers and developers one also needs project management. That is a difficult task for companies but an almost unsurmountable barrier for agency projects.
Both because agencies usually don’t tick like that and because capable project manager almost always tick in an incompatible way.

The classical response? Throw more money and (incl. human) resources at it.

An lots of additional musts like patriotism, political and social reliability and mindset (which to evaluate is a complex task in itself), a high tolerance for bureaucracy, a strange mix of strongly believing in values like democracy and freedom and at the same time a high readiness to bluntly violate major principles of democracy and freedom make that task all but unachievable.

Looking closer one will often find that (particularly large) corporations and states share most of those problems, albeit in different versions. Not surprisingly the solution approaches chosen are quite similar. Throw more resources at it (and usually just trust in “the system” taking care of managing them …).

That’s why I’m not afraid of secret services cracking crypto or the famous “they might have the best cryptographers”.
What we should be afraid of is them undermining security.

Have a look at it. What a coincidence! It seems they have never (in time) cracked solid cryptography. What did do and succeed in, however, was to undermine and corrupt security.

They don’t need better cryptographers than those we have. A well placed student will do. All the Seggelmanns in a gazillion projects create way more damage than the smartest nsa cryptographers. Having some people in nist to corrupt standards is not a necessity but almost a luxury for nsa.

Combine that with the very dirty (il)legal maneuvers available to nsa, fbi, and the like and with the ways they’ve used since long, long before computers existed (bribing, blackmailing, etc.) and are very experienced in and you have the perfect nsa machinery.

Again: no matter all the talk about 1.000 highest level genius crypto guys in nsa and whatnot, what they actually used — and succeeded with it! — was to bribe or blackmail single programmers, students, technicians at carrier centers, etc. into cooperation.
Even heartbleed was not really a crypto problem. It was a potentially intentionally) lousy and careless implementation by a student and an equally lousy “checking” by an incompetent *##&% entrusted with openssl and by “the community” (whatever that may be).

Stewart • May 22, 2014 7:59 AM

So you are saying there is no ‘Setac Astronomy’?
Sorry, couldn’t resist.

A better comparison • May 22, 2014 8:26 AM

It is more apt to compare the NSA to the Stasi than the SS/Nazi.

I recently watched a German documentary “Das was der DDR”, or “That was the GDR” (available on Youtube) that chronicles the history and life of people living in the now defunct German Democratic Republic and the parallels between the crimes of the GDR and the present corporate rulership of the US, and the crimes of the Stasi and the NSA are far more striking than a comparison to Nazi Germany. The only obvious difference is the massive leap forward in technology from the Stasi wiring houses with big bulky microphones and tape recorders and the NSA using technology that is much more advanced as to be largely invisible.

I get that people like to push the Nazi button whenever they can, but it’s a bad comparison, especially in the face of a far more exemplar comparison.

Nobody in Europe could pretend the evils of nazism were not happening, it was an open and well publicised evil.

The stasi on the other hand was an evil hidden from public view, one that could easily be brushed off as hyperbole or rumors, and one that operated both with secret mechanisms and secret motives. I really doubt anyone could have claimed during WW2 that the nazi motives were a secret, they were trumpeted with the loudest propaganda.

Clive Robinson • May 22, 2014 9:49 AM

@Mr. Pragma,

Having some people in nist to corrupt standards s not a necessity but almost a luxury for nsa.

There are two ways to fritz a standard, the obvious and the non obvious.

The obvious ways are usualy requirments or protocols with identifiable “errors” in them that cause security issues. One such is the requirment embeded into telephone standards for “emergancy assistance” where a phone can become a bug under operator control, or as in mobiles have GPS position fixing that likewise can be enablex by a network operator. They are obvious but disguesed behind “Health&Safety” argument thus get waved through by commities.

A non obvious way is to apply certain types of complexity that encorages insecure behaviour in those making practical implementations. The most recent example of this was Heartbleed, the specification is not in error it’s lacking in details in a way that is ambiguous and thus opens the specification implementation upto errors and ommissions. Thus both an ommission (use of checking) and an error (use of malloc) gave rise to blocks of memory getting sent across the network.

The trick behind the non obvious fritzing is “interworking compatability” the first product on the market becomes the bench mark against which all others are tested. Thus having the first product under your control in some way causes others to repeat what you do so that they can interwork with it…

However both methods have a further sting in the tail with practical implementations, and that is “backwards compatability”. Even when something is known to be broken security wise, if it has a large market footprint new products will still need to work with the broken version for the forseable future. Thus carrying the security fault into the future.

Another sting in the tail is engineering order wires (EOW) and test harneses. These are put into products for good and proper reasons during development and for technical support.

All of which gives rise to another sting in the tail which is “auto negotiation” which causes “fallback attacks”. Users are assumed to be incapable of making choices, thus most “user friendly” systems not only do auto negotiation but hide it from the user, thus a very easily workable MITM attack will cause the end points to work in some broken security protocol or worse in a plaintext test protocol…

And as these little “treasures” will be in products with lives measured in decades paying people to sit on standards commities has an almost unimaginable ROI for the likes of the NSA, GCHQ, et al.

Which is why I’ve been pointing it out for quite a few years on this and other blogs and other places, in some cases going back quite a while into the last century. I’m not the only one the encryption recomended in the standards for GSM phones was “an in joke” in the industry at the time Bruce wrote his first edition of his most oft quoted and used book, and it was taught to students as an example of standards being downgraded/backdoored, sadly the industry did not learn the lessons they were being taught…

John Campbell • May 22, 2014 10:10 AM

Borrowing from Arthur C. Clarke, “Any sufficiently advanced technology is indistinguishable from magic”… and, in some ways, Snowden has functioned like Penn and Teller in showing that the technology isn’t really all that advanced when you take the covers off.

Cryptanalysis that cracks a code will look like magic… until you realize that the crypto system was working from an elliptical algorithm specified by the folks trying to crack the code.

Who would all of this have really served?

Benni • May 22, 2014 10:55 AM

@Yesme:

Yes, Snowden hasleaked some operations. The quote of Hayden suggests that they keep the most dirty ones on a separate network.

Slime Mold with Mustard • May 22, 2014 10:59 AM

@ Clive
“there are two ways to fritz a standard…”
That was funny as hell, although I hope we don’t hear back from that marketing guy at Der Spiegel or his compatriots.

Peter • May 22, 2014 11:05 AM

@ Benni
What gen. Hayden meant, was that Snowden apparently only had access to NSA’s administrative network, where the powerpoint presentations and the memoranda were, the things we have seen in the disclosures. These documents are about what NSA is actually doing, not the things NSA actually collects, nor the final intelligence reports based on that.

We haven’t seen any actual intercepted messages (except for some tiny fragments in a presentation). That kind of stuff is in separate compartments, which are under much more strict access control and only accessible for those who have a need-to-know. Snowden apparently wasn’t able to get access to those compartments, which shows that internal access control at NSA isn’t that bad.

Nick P • May 22, 2014 12:34 PM

I should add the the US govt groups that do all the magic engineering are National Science Foundation, DARPA, and Dept of Energy’s labs like Sandia. NSA just uses more typical engineering to put others’ magic to use. 😉

BleedingHeart • May 22, 2014 1:12 PM

I’ve always thought that it was common sense that NSA could break any and all digital encryption at will. As Bruce says, they likely don’t achieve this in an “Enigma machine” cryptanalysis sort of way, but they most certainly have ways to attack the weakest link, especially against specific targets. Of course, I suspect they are a decade or so ahead in the mathematical side too (and NSA insiders have said as much), but that probably isn’t enough of an advantage to conduct a practical attack against a modern cipher like AES. No, what they do is what many of us suspected they did before the Snowden leaks: they subvert and sabotage the whole process wherever they can. They insert subtle backdoors in standards, hardware and software that adhere to the NOBUS principle.

I honestly wouldn’t be surprised if Rijndael was selected because it was the weakest entry (at least out of the finalists). Some people have suggested this all along, but were usually discounted as conspiracy theorists. I can picture the conversation among the crypto-geeks at NSA back in 2000 now: “The Belgians think they are clever, but this is 1970’s technology for us. We have found a flaw that will be exploitable, but it’s so far ahead of the public research we feel sure that the cipher will be secure against anybody but us. Get on the horn with NIST and tell them to select Rijndael.”

I remember reading a declassified entry in NSA’s crypto-journal. The author (whose name was withheld) was attending Eurocrypt (or another similar conference) in the 1990’s. Keep in mind he was writing in a classified journal intended only for NSA eyes. He basically spent the entire time deriding and poking fun at public crypto research saying things like “they are on the wrong track, which is good for us.” He also pointed out that most of the talks were “interesting computer science and complexity theory, but not cryptography.” Basically I got the strong sense that the NSA wasn’t worried in the slightest what the academic community was working on, and I was quite surprised this entry was actually declassified.

Basically, I am not sure why anything in the documents surprised anyone who is educated on this topic (such as people who read this blog). I’ve always thought that it makes no sense at all to trust the NSA for security advice when their primary job is compromising security, conducting SIGINT, breaking codes, etc. It’s the quintessential “fox guarding the hen house” scenario. In other words, they are never going to give away their bag of tricks for breaking systems just so they can secure the banks or protect our Amazon transactions. Sure, they may help us secure some of the low hanging fruit, but they are never going to tell the public how to secure itself against an organization with their budget and technical expertise. Think about it, if they did, they would be telling China, Russia, and terrorist groups how to do the same. After all, everyone uses the same hardware and software and everyone has access to the same encryption technologies (which is enough to tell me that none of this stuff is secure at all).

I had never heard the term NOBUS until recently, but I have always known this is a philosophy the NSA likely employs when deciding how to balance security vs offensive capabilities. They are arrogant enough to believe no one but them will ever find the same flaw (and they are probably right in most cases). But the mere fact they use this acronym internally means they have actually employed it (and to think of the scenarios where they have actually employed NOBUS exploits is a fun exercise). Hayden gave an example of a basement full of Crays breaking crypto that no one else could break. I wouldn’t be surprised if this was more than just a hypothetical.

Ralph Haygood • May 22, 2014 1:12 PM

Magicians generally don’t care to work for police organizations. They don’t need the money, and they find bureaucratic environments stultifying.

aboniks • May 22, 2014 2:34 PM

As for why they buy zero-day exploits, it’s important to keep two things in mind:

The NSA is expected to be both the shield AND the pointy stick. They need to know where all the holes are.

Why pay? They’ve got far more money than personnel, for one. For another, it’s basic CYA; Would you want to be the one explaining to POTUS that someone got into something they shouldn’t have because your people didn’t pursue every available avenue to ensure security?

moo • May 22, 2014 4:20 PM

http://arstechnica.com/information-technology/2014/05/china-responds-to-nsa-tampering-with-network-gear-with-vetting-process

So here’s what I don’t understand. How does China expect to detect tampered-with network gear?

Its easy to embed “extra functionality” that is only activated under very specific conditions (port knocking etc.) in a way that is virtually undetectable either through testing or through RE / direct inspection.

In hardware, a few hundred gates added to a CPU of hundreds of millions can create a remotely-exploitable privilege escalation bug that is triggered by sending a specific IP datagram to the machine. The software equivalent, a few lines of code added to a TCB of hundreds of thousands of lines of code (kernel, drivers, firmware, various embedded controllers, ..) is just as hard to detect.

The only way you’ll ever know that you’re using compromised network gear is by watching its traffic and catching it doing something its not supposed to be doing.

Harry Johnston • May 22, 2014 6:27 PM

Mr. Pragma: the Yatsenyuk Government was appointed by the elected parliament. The parliament was certainly entitled to do that, that’s what parliamentary sovereignty is all about. So the new Government is legal. So was the previous one, of course, but the people are certainly entitled to recall a Government that no longer represents their interests.

Mr. Pragma • May 22, 2014 6:36 PM

Clive Robinson (May 22, 2014 9:49 AM)

You are perfectly right but notwithstanding I can still stick to my point.
There is, in fact, very little observed in terms of “high-tech attacks” by nsa but there is plenty in terms of old-school buy/bribe/blackmail’em attack vectors.
Similarly we have no reason to believe that nsa succeeded in cracking or even “just” gravely weakening crypto security. Putting it bluntly, at least most of what nsa does and achieved was/is based on a) old-school procedures, sometimes halfway adapted to current technology, and b) on others sad and lousy weakness.

Their main entrance isn’t any crypto issue. It’s hundreds of Mlocs of, uhm, less than perfect code, lousily crafted standards and de facto standards, and generally gross lack of even basic understanding, ignorance and carelessness by the vast majority (using windows or linucks, openssl, and a mass-produced, very poorly secured (and securable) router box).

Mr. Pragma • May 22, 2014 6:38 PM

BleedingHeart (May 22, 2014 1:12 PM)

I agree.

And: Nobus isn’t a strength. It’s a weakness. And it’s yet another weakness to not understand that.

Mr. Pragma • May 22, 2014 7:06 PM

Harry Johnston (May 22, 2014 6:27 PM)

How funny.

Fact is that many members of parliament have been blackmailed, threatened, and even beaten by the kiev thugs so as to make them vote the “right way”.
Fact is also that a number of MoPs has their MoP cards taken away. There is actually proof of kiev thugs voting multiple times (abusing the stolen MoP cards).

And yet they still failed to reach the needed 75% majority to properly oust yanukovich (who may be an asshole but that’s not the point here).

Today the ukraine criminal regime announced that they plan to militarily attack an eastern region on election day!

I’m afraid defending that crime gang as “democratic” is outright ridiculous and plain stupid.

Go and nuland yourself!

Harry Johnston • May 22, 2014 7:50 PM

@Mr Pragma: sorry, but the bottom line is that your claims aren’t credible. Might they conceivably be true? Certainly. Are they likely to be true? No.

Chris Abbott • May 22, 2014 9:13 PM

@BleedingHeart

You may be right.

Rijndael did have the lowest security margin of the top 3. It won because it worked best on primative smartcards. Twofish would have been a much better choice for two reasons, the 256 version is faster than Rijndael 256 and it’s much faster in general than Serpent. Serpent came in second because even though it had the highest security margin, it was also the slowest. I imagine that the NSA did think that Rijndael was secure enough at the time for general use, but thought that it would be the easiest to attack in the future if need be.

Nick P • May 22, 2014 11:00 PM

@ BleedingHeart and Chris

re AES

I doubt it was selected because it was the weakest. It was probably selected for the reasons stated with NSA knowing they’d have plenty workarounds. However, there is a middle point in this where they might have thought it would be the hardest to implement without bugs or side channels. Not being a cryptographer, I can’t say how likely this is. I just know most actual attacks on AES weren’t at the algorithm level.

And NSA allows it to be used for their own stuff. They mandated their Type 1 algorithms for highest secrecy stuff for obfuscation value. However, much of the rest can be encrypted with AES. Provides some indication they trust it.

BW • May 22, 2014 11:36 PM

Want to know about NSA’s decryption hardware?

Was talking with an acquaintance who worked in HPC (High Performance Computing) for quite some time. He worked for one of the big vendors and the NSA would come to them and say, “If you want us to buy from you, you system need to support these specific Instructions”. One of those Instructions would be POPCNT. Look for instructions that make decryption easier and that is the hardware the NSA is using.

Anura • May 22, 2014 11:58 PM

Keeping in mind that my opinions on cryptographic algorithms are worth absolutely nothing, I have to say that I find AES an odd choice from a security perspective, but I don’t think it seemed malicious.

The entire algorithm is based entirely on linear functions in GF(2) and a single nonlinear function: inverse multiplication in GF(2^8) (given x, solve for y where x*y = 1; x and y are in the range 1-255; when x = 0, y = 0); this makes it very small when implemented purely in hardware, and in terms of various properties, non-linearity, max value in the XOR table, bit independence, and strict avalanche criterion, the s-box is as close as you can get to perfect for an nxn bijective s-box. The problem is that given how different the algorithm is from what has been studied, and that it relies entirely on pretty basic functions in GF(2) and GF(2^8), it seemed like it would be prudent to choose something else.

The structure is also very different from what we have seen before it; matrix multiplication itself seems like a good choice for diffusion, but where each 32-bit word is composed of one byte of each word from the previous round? That just feels weak – if you change all four bytes of one input word, there is a relatively high probability (255*4/255^4) that after two rounds, only four bytes will change (although if that happens, it guarantees that after 3 rounds, all 16 bytes change, so there’s that). Of course, when you implement the algorithm, the structure makes perfect sense, because decryption is identical to encryption with a different key-schedule, the inverse s-boxes, and the inverse matrix – if you tried to add something like addition modulo 2^32, then either encryption or decryption would become significantly less efficient (unless it’s only for input and output whitening).

Personally, I think we’ve learned a lot since AES was chosen, and I would love to see us have a new competition to choose a successor – given that it will take decades to move away from it, it seems prudent to start as soon as possible, since the process will take years. There is a lot of promise in ARX design, and 64-bit processors are the norm for new systems.

Clive Robinson • May 23, 2014 3:40 AM

@Mr. Pragma,

There is, in fact, very little observed in terms of “high-tech attacks” by nsa but there is plenty in terms of old-school buy/ bribe/blackmail’em attack vectors.

Probably because the principle of “Low Hanging Fruit” works better for them than it does for most financial/criminal attacks.

However we do know from one or two observed peices of SLA generated code –Stuxnet / Flame / etc– that “they” can make attacks way way beyond that of criminals and a lot of securiry professionals, as it was shown that they had a method of breaking hashes that was unknown to the academic and security communities.

Thus I rather suspect that “they” –who ever the SLA(s) are– are ” keeping their powder dry”. Also as they say in all the financial adverts “previous performance is no indicator of future performance”.

So we know that atleast in one aspect one of the Five Eyes or their assosciates have crypto skills/luck ahead of the academic community and they were happy to risk it on what is in effect a low value target (contrary to political posturing a little reasoning shows that Iran is not realy that much of a threat). Unless those involved were doing political B’tkizz it would be unlikely that they would blow there best attack on Iran… so I suspect the likes of the NSA, GCHQ, et al do have crypto attacks as yet unseen.

Jhp • May 23, 2014 4:14 AM

Just for the record: It most likely wasn’t Snowden who disclosed these tailored access tools.

Mr. Pragma • May 23, 2014 4:47 AM

Clive Robinson (May 23, 2014 3:40 AM)

Do you have more specific info on hashes being cracked in the context of stuxnet (or other well known attacks)?
I only know of Stuxnet tricking a (very simple) CRC check.

Actually, I rather see stuxnet as an example for what I say (nsa doesn’t need high end crypto cracking abilities. They can rely on sloppyness, ignorance, lousy software, lousy OSs, lousy crypto, etc).

Just some examples: To work, stuxnet needed

  • Windows. In a security sensitive environment (provided by Siemens S7)
  • very poor OpSec, e.g. usb sticks being accessible to untrustworthy personel as well as the controller system carelessly accepting those usb sticks and even auto-executing
  • shared printers. In a security sensitive environment

Also, frankly, I think that stuxnet and its “incredible complexity and ingenuity” has been hyped. From what I’ve seen so far, I fail to see a high-end group of considerable size and will large budgets behind it (design, coding – not organization).
In fact those numbers hinted at do convince me that a state organization was behind stuxnet, possibly and probably usa. Generously wasting multiple 0-days, ~500KB payload, “incredibly complex code” – those are item I take indeed as a strong hint at a usa agency.
A professional black hat group, in particular a Russian or Chinese one, would almost certainly have created a way smaller, more efficient and professional attack (you don’t want to widely spread that kind of virus because e.g. it increases risk of discovery and such failure).

As for the rest, well, obviously good intelligence about the Iranian target was needed, someone had to either smuggle in the usb stick or to break in at some technician and poison his stick – all pretty classical fortes of intelligence services. No ICT or crypto high-tech needed.

Also importantly, the attack demonstrates once more how well intelligence services (or criminals) can rely on lousy software, lousy OpSec, lousy OSs, lousy crypto, etc. – even in highly sensitive areas.

Mr. Pragma • May 23, 2014 6:44 AM

Clive Robinson (May 23, 2014 6:21 AM)

O.K. looked at it. Thanks for the link.

Not wanting to be picky but I take that again more as an example of what I said.
Sure, that guy very smartly developed an academically interesting and intelligent crypto attack. But, uhm, on MD5.

So, the real problem, looking from a security standpoint, is that microsoft secured its updates by (quite poor) MD5 hashes in the first place.

Again, good crypto is important, no doubt. Similarly, it is important to further push the limits in crypto, secure processors, etc, etc.

But the reason why here and today pretty every system is rather vulnerable to attacks is the lousy minimum and average security and not a lack of high security availability or quality.

So, sorry to sound boring, but nsa doesn’t need to have the most brilliant cryptographers or capabilities lightyears ahead of ourselves – neither is that observed. What actually is observed again and again that attacks, no matter whether by nsa or by a(nother) crime gang succeed because even sensitive systems are shockingly often lousily secure.

itgrrl • May 25, 2014 1:38 AM

@Nick P (re: optimism/pessimism):

I tend to agree with your assessment. To adulterate Clarke, “Any sufficiently advanced technology renders magic unnecessary.”

In the context of security vulnerabilities in software, the bar for ‘sufficiently advanced’ is directly proportional to the quality of the security of the software in use. Hence, currently low enough that no magic is needed. 🙁

Nick P • May 25, 2014 1:55 AM

@ itgrrl

Exactly. The only things that Ive seen that come close to magic are active emanation attacks and the use of ultrasound to bypass air gaps. Both were effective enough that most potential targets are (a) still unaware of these or (b) still vulnerable. Maybe should add the DNS attack via bit flips from cosmic rays. Most security ignores that, while that researcher weaponized it. Clever.

Most of the rest, as you say, just comes from systems setting the bar very low.

Clive Robinson • May 25, 2014 5:41 AM

@Nick P,

Most of the rest, as you say, just comes from systems setting the bar very low.

Which gives rise to the “How to climb a Mountain” issue I’ve been going on about since the last century when banks could not get online banking secure “no way no how”.

Humans don’t climb mountains naturaly, anything aproaching a steep surface is problematical for most which is why ditches, banks, fences and later walls made good defences.

However with a bit of ingenuity or luck some humans found ditches could be “bridged” with fallen trees etc, or with sufficient practice and training jumped. Which initialy resulted in bigger ditches, which in turn gave rise to better bridging equipment and tactics. Each time a small increment in defence gave rise to a coresponding improvment in offence.

The same is true of mountain climbing, few ordinary humans can climb mountains but with first climbing hills and steep surfaces their strength, ability, and knowledge improves, eventualy vertical rock faces and overhangs are climbable freehand. However those who make mountain climbing their proffesion used science and ingenuity to make equipment to help make climbing a mountain easier to the point that all mountains are climbable.

The same applies to defenders and attackers, generaly each small improvement a defender makes generaly forces attackers to come up with improved attacks based on new methods and tools. However occasionaly some defenders raised the bar significantly by building fortresses on cliffs and mountains with only one easily defended aproach an army could make. Which in turn gave rise to other forms of attack such as siege, or lone attackers such as spies/assasins etc, but untill much later with the advent of long range artillery and aircraft the fortresses could be succesfully defended.

The important take away being that small improvments in defence only encorage attackers to improve. So in effect the defenders are training their attackers, and thus get attacked again in short order. However large improvments in defence make the learning/technology curve so large that the attackers go else where.

And this was my gripe with banks and their online systems, at best they would make tiny improvments in security, thus trained up their attackers to build new tools and come up with new methods untill they could climb any bank mountain they chose. The banks only started making significant changes due to bad publicity where the changes made could be used as a marketing tool, and where not they changed their terms of business to externalise the risk/blaim onto their customers, unless legislation stopped them doing so.

Thus in ICTsec small increments in security are nothing more than very short delaying tactics. Even medium sized improvments are delaying tactics unless all defenders adopt the improvments together (which seldom or never happens). Because the attackers continue to fund new methods and tools from those who did not make the improvments. Thus we have the situation where even Two Factor Security systems that should have provided a sufficient raising of the bar have failed, because the attackers have had the funds from other sources to give them time to work out new and significantly different methods.

Thus from the security perspective when you “raise the bar” it’s prudent to do it in large measure, not in next to non existant increments, otherwise your attackers will climb all over you in short order.

Nick P • May 25, 2014 12:00 PM

@ Clive Robinson

It’s a decent metaphor but computers aren’t like the real world. They, by design, have a specific number of potential interactions between components and types of information flow. The systems so far have been designed where it’s easy to manipulate in attackers’ favor. Some were designed where it was extremely hard. The difference is enough that it’s almost conceivable to secure a computer, esp special purpose, it inconceivable to secure an arbitrary location in physical world.

Note: Oak Island Trap is the closest thing I’ve seen to perfect physical security in practice. Maybe do the same for a nuclear powered, water-tight, pressure-resistant, etc computer? 😉

“The important take away being that small improvments in defence only encorage attackers to improve. So in effect the defenders are training their attackers, and thus get attacked again in short order. However large improvments in defence make the learning/technology curve so large that the attackers go else where.”

The best part of your post. Well-said. I can see myself repeating it in a conference room at some point. 😉

Clive Robinson • May 26, 2014 5:39 AM

@Nick P,

Another viewpoint –from some one involved in startups– on for profit attackers from the end of 2005, which shows the low or non existant “lifting of the bar” by defenders untill it is to late to do anything,

http://mengwong.livejournal.com/3479.html

Perhaps unsprisingly getting on for a decade later little or nothing has changed and the strategy outlined for attackers still holds…

Nick P • May 26, 2014 5:20 PM

@ Clive Robinson

“The biggest banks, brands, and ISPs in the world are struggling today because you picked the right time to strike: you started spamming in earnest when email was entrenched, when it was too late to fix.”

My emphasis added. Best point in the post. So, look at new trends, wait until market picks one, wait until a bunch is built on it, and then hit it massively. That’s the recipe for black hat innovation right there. Of course, they often store the customer data in easily accessed databases making an attack on that aspect productive, too.

anonymous • May 29, 2014 12:54 PM

Bruce, you can find in the internet, that Snowden describe his qualification. He said, he is really just an administrator, not a officer of NSA.

He said, that somewhen early, he spent some time to learning spies knowledge. He got kind of skills as a CIA agent or FBI, I don’t know.

Anyway, he is just an Admin.

Let’s get down to the human’s platitude. Once, an admin who works in the Police Office got kind of data about werewolves in epaulets. It is real, see movie called “Leon”: http://www.imdb.com/title/tt0110413/

He bring this data to the internet with caution about his care and now he is in Russia.

What we are know about this Police Office? Are we know that they are not able to do anything more powerful than Snowden represent?

Methinks, everything what he brings is just a skimming, first strata of real NSA’s job. Just a promotion.

You are know what Bro is? You are pro in configuration of Snort? You are hacker with Suricata? You are great penetration tester with Metasploit? Look at us, we have much better and more robust tools with real hardware platform in the background. With a government donations. With a real idea in the our job.

We are protect United States of America. Welcome! Come in!

He, Snowden, – is just an admin of part of Secret Service. I’m not a specialist. I’m thinking NSA not greater than CIA. But methinks NSA is a part of CIA.

Everything what he bring – just a injection into public from NSA(or CIA) side.

Experiment.

Not a real(full) ability of National Security Agency. Just a first strata. Fat.

Steve Kinney • June 2, 2014 11:54 PM

“…the most surprising thing about the Snowden NSA documents. It’s this: the NSA is not made of magic. Its tools are no different from what we have in our world, it’s just better-funded.”

I’m afraid that’s consistent with my initial take on The Snowden Affair, as limited hangout operation by the NSA. Every scandal raised is an old scandal, already firmly settled in the intel community’s favor. Every big revelation is something we already knew about, or already confidently guessed. My initial observations:

http://globalresearch.ca/nsa-deception-operation-questions-surround-leaked-prism-documents-authenticity/

… and nothing I have seen since has caused me to revise my estimate. Rather the opposite: For instance, our Mr. Snowden has gone on record with stories about his service record that seem to prove he is, at best, an unstable and unreliable source. But why would our intel services stage this spectacle?

Perhaps to seize control of the intelligence service leaks market and flood it, drowning out uncontrolled and potentially harmful information, saturating press resources and fatiguing the public’s attention.

The Snowden Affair forcefully directed the press and public to pay attention to the excesses of an agency that collects against domestic targets only as a component of spying on foreign targets: Never mind the FBI’s CALEA wiretaps on all communications carriers inside the U.S., DHS fusion centers that erase the lines between military and civilian intelligence, DoJ political warfare operations such as its use of State and local police forces nationwide to forcefully suppress the Occupy Movement, etc. Ordinary Americans don’t know anything about any of this, and Congress could not care less.

With the passage of the toothless USAFreedom Act, a supposed intelligence reform bill in response to The Snowden Affair, the NSA and the rest of the U.S. intel community, military and civilian (to the extent that this distinction still exists) can declare Mission Accomplished: A troublesome grass roots anti-surveillance lobby was co-opted, made to dance, and handed a completely bogus victory. Foreign intelligence services and domestic spook-watchers learned nothing of substance about U.S. programs and methods, except maybe the right names of programs already known or assumed to be in place.

If any new trouble with a real leaker develops, Glenn Greenwald stands ready, on a $250 million retainer, to present anything his handlers send him as a “Snowden document” – and grab whatever meagre attention the press is still willing to pay to such matters away from the trouble spot. At minimum, they bought themselves some extra time to tighten up the Insider Threat program. If my guess is right, that fledgling program scored a stunning victory in spotting and profiling Ed Snowden. Seen through this lens, the loose ends all tie themselves up and the whole Snowden Saga at least makes sense.

Imjustglad • April 30, 2018 1:43 AM

Mistake here, mistake there, soon it is everywhere.

“Did you enable that policy, or did I, and does it overide this rule or that, and is it supposed to be exposed to the web, aren’t we using the same hardware that we inserted a backdoor into the firmware, and if we have no oversight who’s checking anyone understands any of this?”

jf • May 1, 2018 10:52 AM

It doesn’t seem odd to you or anyone that the SSL is talking about cracking thousands of SSL keys per day but the highly activist utilized OTR is this magically implausible thing to bypass…which is mostly used by pidgin, where a person can invest 5 minutes and start finding buffer overflows in the message receive code path?

It doesn’t seem odd that cryptonyms, whose entire function is to hide the project details and thus has a requirement that it cannot describe any aspect of the project includes a lot of cryptonyms that describe precisely what it is supposed to hide?

It doesn’t seem odd that an agency that has invested significant sums into “multi-layer security” protocols seemingly never has any? Somehow the Russians cannot penetrate this data effectively, but a security guard ended up with expansive access?

They profile people, identify people likely to leak, high-risk individuals, then put them into positions where counter-intelligence data is mixed with real data and then bombard them with fear and scare until they break, steal and run.

If you’re sitting in a room full of military intelligence personnel and they all seem dumb, you’re the mark.

jf • May 1, 2018 10:55 AM

ADDENDUM:

That said, there actually were significant leaks inside of that dump, mostly stuff that was already known but now confirmed. The world missed them because it was surrounded and covered with decorations because that’s how the IC rolls. Echelon is avowed, in the IG reports full data captures of “internet data” is avowed (and internet data basically is phone data), and so forth.

Basically, the entire world is tapped, full data capture on everything and that’s all avowed in there, but we somehow missed that all to focus on things of lesser significance, id est metadata.

Leave a comment

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.