Schneier on Security

Clive Robinson • June 11, 2011 9:03 PM

@ Brian,

“Certainly an interesting idea…but does he offer any proof to support his theory? Well, no. All he really offers is conjecture that this MUST be the state of things because…”

There is the old maxim about “absence of proof…”

That being said we know the NSA has two conflicting personalities because that is what their charter basicaly asks for,

1, Use talent to protect American Comms.
2, Use talent to breach all other Comms.

And this was less than a decade after WWII just as the Cold War was getting interesting but before the concept of Data Comms and Computers had realy got underway. With the public view of “communications” being basicaly voice and vision at a distance.

For the military however Command and Control was mainly about Data Comms not voice, because voice was too difficult to secure and thus reserved for battle field communications where security was a lower priority compared to timelyness of delivery.

Also known to the UK and US military leadership and kept quite was the fact that crypto systems such as the German Enigma for morse traffic and higher security telex systems (Fish etc) where all breakable. It was not till the 1970’s book by Fred Winterbottom that the rest of the world became aware of “Ultra”.

It was a real shock to most nonwestern countries who had purchased the old German Enigma’s and US Haglin machines as war surplus to use for their Diplomatic traffic. Even though the original machines where obsoleate by that time these countries where still buying augmented systems using the same design technology from Crypto AG in Switzerland.

As for the NATO military command and control comms that used a variation of the NSA ECM machine still using rotors and code wheels even though US and UK mainline command and control and diplomatic traffic had gone electronic. Initialy this was mainly in the form of stream ciphers, however developments in aircraft Identification of Friend or Foe (IFF) systems had started serious development in Block ciphers.

It was strongly suggested by David Khan that the NSA had stolen work by independent crypto developers and had handed it on in a ‘poisoned chalice’ form to others and had kept “back channels” open on development in the only independent crypto equipment supplier (subsiquently brought to light in a more serious form in 1992 when Hans Buehler a Crytpo AG senior sales reprsentative was arrested on espionage charges by the Iranian’s and interogated for nine months).

What is now known is that a number of the Boris Haglin’s “coin counting” mechanism field cipher systems where “improved” by the NSA and that they had various key strengths ranging from weak to moderatly strong (for the time). Importantly that is was not obvious in any way from examining the mechanical design which keys were which, and only certain mathmatical analysis would show which were strong or not.

It has been suggested that this “quirk of design” was one way the NSA could resolve the differences between their two charter aims. That is the NSA responsable with issuing key material for use could ensure that only the stronger key settings were used by the US military. Whilst systems that had been captured and were now being used by others would be unaware of the key strength differences and would end up using both strong and weak keys to the NSA’s benifit.

In the mid 1970’s single chip CPU’s had arived and a major push was to put Crypto onto them. It had however became clear to most financial organisations ten or more years prior to this that strong crypto was a necesity. Thus the US Government in 1973 put out requests for designs for crypto systems via the National Bureau of Standards (NBS was NIST’s forerunner) under the guidance of the NSA. None of the candidates were of any use some being little more than modified versions of hand ciphers that had been broken prior to WWII.

In 1974 NBS issued a second request, the winner of the selection process was from an IBM design team and was based on Horst Fiestel’s Lucifer cipher. However the design was felt to be not satisfactory and was subject to significant changes before eventualy becoming the DEA for the FIPS-46 DES.

It is now known that the NSA had a very strong influance on changing the original IBM submission (see information from the likes of Don Coppersmith). Some of it good (improving the S-Boxes) some of it bad (reducing the key length), and there have been various comments that the NSA wanted the key length to be only 48bits and it was a compromise between that and IBM’s original 64 bits that resulted in the 56bits of the standard.

Right from the start DES was attacked due to the seen and unseen hands of the NSA. However it is reaonably clear that as the only publicly available algorthm that had been “blessed” by the NSA it was going to receive a lot of interest. On the academic side it opened up areas of research that had not received much in the way of attention and kick started a whole new area. It became the standard by which others were judged. However important as DES was it was FEAL that brought about it’s down fall.

What was clear from a practical perspective was that the DEA was effectivly an “anti-software” design. That is the design used functions that ment implementing it in software would be considerably more difficult and execution time expensive than it needed to be (things like the permutations are effortless in hardware but high effort in software). Some of these functions (initial and final permutations) actualy served no purpose in the strength of the cipher.

The Fast data Enciperment ALgorithm (FEAL) was presented as a more software friendly alternative to DES in 1987 by Akihiro Shimizu and Shoji Miyaguchi. Both working for Nippon Telegraph and Ttelephone the algorithm was also a Fiestel round design. It received considerable attention but it had weaknesses, right from the very start at the conference it was presented it was attacked. The attack was improved and formed the beginings of what later became Differential Cryptanalysis which was the first attack to bring DES under serious question. It became clear that DES was such that it’s strength under DC was effectivly the same as it’s Brut Force strength giving rise to the idea (later confirmed by Don Coppersmith) that the NSA were aware of DC at the time of DES’s design.

However subsiquent versions of FEAL though strengthend against DC opened up other avenues of attack that gave rise to Linear Crypanalysis and this was the academic nail in DES’s coffin.

However the NSA had spotted that DES had escaped their control and the academic community was catching them up quickly. Whilst fighting a reguard action to keep DES alive they started up a series of developments one of which delivered in 1980 the SkipJack algorithm that was to become the basis of the Key Escrow program through the classified Clipper Chip.

SkipJack was bassed on research work carried out prior and during WWII in combinatorics and abstract algebra. The design was classified and this created a significant backlash and the NSA yet again wrongfooted went on another reguard action and belatedly invited a small group of academics to review both the design and the process behind it in 1993.

Although it was given a clean bill of health by the team the whole Key Escrow system received such a backlash it was cancelled, however SkipJack lived on in other US Government systems. In 1998 the SkipJack algorithm was suddenly declasified and made public due to a significant problem the NSA had brought upon themselves, whereby the algorithm that was only ever intended to be in CapStone hardware on Fortezza Cards due to production difficulties had to be put in software.

Skipjack has some odd properties which Bruce described in the July 1998 Crypto-Gram ( http://www.schneier.com/crypto-gram-9807.html#skip ).

Of primary interest was it was the first electronic age NSA “inhouse design” ever to become public, although it was secure the design was very very brittle, in that even very small changes would drasticaly reduce it’s security.

This brittleness harked back to the key strength tricks used on the mechanical Haglin cipher machines, in that anyone on discovering the design without fully mathematically understanding it would be tempted to use it in a slightly modified way and would thus lose significant security.

It had in the mean time become abudently clear that DES was now a “dead duck” and NBS’s successor NIST started a new encryption competition to replace DES with the “Advanced Encryption System” or AES. On the face of it although NIST where in consultation with the NSA the competition was held openly.

However the AES competition may have been riged by it’s rules (commented on but ignored at the time). Put simply the candidates had to submit candidate designs for software and hardware and all the emphasise was placed on speed and efficiency.

The result was software and hardware designed to meet the competition criteria not for security. The resulting designs were public and ended up “as is” in by far the majority of products and software libraries and still are in many today.

The problem as the NSA would have been only too well aware is that Efficiency and Security are often on opposit ends of a see-saw. In general unless you take specific precautions (many of which are clasified) an efficient design will open up a significant number of side channels through which information on the plaintext or worse key will hemorrhage.

And before the ink was dry on the draft of the new FIPS AES standard a practical demonstration of AES key recovery across a local area network was put up on the net.

Bob Cringely may not have given any evidence to support his NSA view, and direct evidence may not be available in some peoples view. However the information I have given above certainly suggests that there is a probability that they have at times in the past acted in that way either directly or indirectly weakened security.

Now the question is do you have to prove it before taking precautions or just assume it’s sufficiently probable you take the precautions any way?

Prudence suggest it’s best to assume the worst when it comes to security.

That being said at the end of the day the NSA still has to play the dual role game, and as I’ve indicated there is plenty of information out there showing where they may well have put their hands out in one way or another to ensure that the game goes on.

But what of the future? There are a number of areas the NSA have their lives made easier.

Firstly “Standardised Plaintext” in headers of files and comms protocols. As seen with WEP exploiting a design weakness is significantly strengthened when you know where and what plaintext to look for.

Secondly “side channels” algorithms might well be secure when considering only their input to output mapping (the so called “data at rest”), but what about the dynamic performance of an implementation?

Some algorithms are inherantly more secure than others in this respect, whilst others need specific caution when implementing in general, but importantly some algorithms are very bad when being implemented on certain common hardware architectural features such as caches etc.

Thirdly algorithms don’t exist in isolation in the real world the exist not just in hardware and software implementations but also in standards. Not just for modes of usage but in all maner of standards across the entire software and hardware stacks.

Although an algorithm can be secure the manner in which it is used within a protocol can make it considerably less secure, protocols come into existance in two ways, as standards and being compatable with existing implementations.

A knowledgable person can make a secure standard brittle in design. And by providing the first implementation that although compliant with the standard on paper is actually broken effectivly forces others to then break their design by making them compatable.

Another trick is legacy compatability, a standard contains a hiden or unknown security flaw, if and when it’s discovered the standard is reissued as Rev2 with the protocol flaw fixed. However practical implementations for legacy reasons maintain compatability with the broken Rev1 of the standard. In most cases the implementations can via a man in the middle attack be forced transparently back into the insecure Rev1 one working mode.

I would expect the NSA as part of it’s working brief to be influencing the design of protocols and standards to achive these desirable (for them) outcomes.

Even if they are not, these things happen any way because developers know to little about security and don’t usually have the luxury of time or other resources to ensure their products are secure.

As an individual you are at liberty to decide the level of caution you take, however as an employee this is generaly dictated by managment who do not want to sacrifice “short term shareholder profit” for “long term security”. Simply this is often because their own career is usually judged by the metric of “shareholder gain” and they don’t seriously expect to be in any given job for more than a year.