Friday Squid Blogging: Magnapinna Squid Photo

Neat photo. Video, too.

As usual, you can also use this squid post to talk about the security stories in the news that I haven't covered.

Posted on November 22, 2013 at 4:53 PM • 85 Comments

Comments

NobodySpecialNovember 22, 2013 6:05 PM

Crypto generally relies on the difficulty of factoring large primes

There is a famous theory that proposes there are an infinite number of pairs primes separated by only 2 (ie 11/13, 17/19 ).
This result shows that there are an infinite number of pairs separated by no more than 70million (other workers have pushed this down to 600)
It doesn't suggest anything to help you factor a prime, or even identify if it is one of a pair, or even if it is prime.

SpaceTruckNovember 22, 2013 6:18 PM

If we can't trust the CPUs we buy these days, then maybe we can print them using the tech that allows transistors to be printed with ink jet printers.

Failing that, maybe we can do some kind of simplified homemade semiconductor fabrication. High end chips are far far beyond home fabrication, but it can be surprising what do it yourselfers can accomplish these days. At least we wouldn't have to grow silicon at home, since I can't imagine any way for them to backdoor a blank silicon wafer. Such homemade chips would be slow, but might be acceptable if they incorporated a cryptographic accelerator. It might be possible to purchase pre-manufactured masks and other verifiable parts to ease the process. The town geek might be a more trustworthy source to buy a CPU from than any commercial factory.

Andre DeMarreNovember 22, 2013 6:24 PM

The Wired article cited by Whiskey Tango refers to both advances, first that of Yitang Zhang to reduce the prime gap to 70 million, then James Maynard's gap of 600.

This appears to be an advancement on the twin primes conjecture, but not being a mathematician, I haven't a clue if any of this work has consequences for factoring large semiprimes.

AnuraNovember 22, 2013 6:28 PM

@SpaceTruck

As is the case with everything else, the answer is nanoassemblers. Once we have that technology, we can have fully open source hardware capable of constructing top of the line components from raw materials.

AnuraNovember 22, 2013 6:49 PM

In cryptography-related patent troll news, Newegg is set to start dueling with the troll that has been suing everyone for using SSL. Whitfield Diffie is set to testify as an expert witness.

http://arstechnica.com/tech-policy/2013/11/newegg-hurtles-toward-texas-showdown-with-famed-patent-troll/

The patent itself doesn't seem to even cover anything in SSL/TLS as far as I can tell, but companies have been caving in because their licensing fees were cheaper than going to court. Personally, I think there needs to be an ammendment to exclude patent trolls from the protections guaranteed under the eighth ammendment.

Scott FergusonNovember 22, 2013 8:31 PM

MIT researchers have produced a new paper that uncovers security flaws in C and C++ software, generated by compiler optimizations that discard ambiguous code or code, which produce undefined behavior.

Some of that code includes security-relevant checks and the paper includes examples of null pointer checks and pointer overflow checks that the GCC compiler optimizes away, leaving the resulting object code exploitable.

More complex ambiguous code includes things like bit shift operations that operate one way on x86 and operate another way on different architectures such as ARM.

The MIT team produced a new static source code checker named STACK that identifies such code, which they term "undefined behavior" and "unstable code."

The researchers identified 32 bugs inside the Linux kernel, five in the Python programming language and nine in the Posgres DBMS. More worryingly, the team ran Stack against the Debian Linux archive, of which 8575 out of 17432 packages contained C/C++ code. For a whopping 3471 packages, STACK detected at least one instance of unstable code.

The research paper can be found at: http://pdos.csail.mit.edu/~xi/papers/stack-sosp13.pdf

(emphasis mine)

GOLD RSHNovember 22, 2013 10:10 PM

@Scott Ferguson

There is a man at the NSA who runs the division tasked with subverting future cryptographic innovation.

He manages programs to engage with standards organizations for the purpose of weakening encryption.

He thinks long range, in terms of years.

He imagines the millions of programmers working this weekend to escape from NSA domination.

He sees in principle that many of these projects use C, the main language used to build the internet.

He realizes that he can strangle many future encryption products in the cradle by spreading fear, uncertainty, and doubt related to the use of C.

Hence the recent hyping of this "research paper".

Scott FergusonNovember 23, 2013 12:28 AM

@GOLD RSH


There is a man at the NSA who runs the division tasked with subverting future cryptographic innovation.


He manages programs to engage with standards organizations for the purpose of weakening encryption.


He thinks long range, in terms of years.


He imagines the millions of programmers working this weekend to escape from NSA domination.


He sees in principle that many of these projects use C, the main language used to build the internet.


He realizes that he can strangle many future encryption products in the cradle by spreading fear, uncertainty, and doubt related to the use of C.


Hence the recent hyping of this "research paper".

I read the "research paper". I'm familiar with some of the problems found by STACK, and "research papers" in general.
I have no reason to doubt the findings - nor do I doubt that they appear in a "research paper". Those flaws patently and demonstrably exist, and are potentially exploitable.

I'm also familiar with FUD.
Your post smells like "troofer" FUD.

Please expand on where they over-hype potential problems so I don't dismiss your "insights" into the NSA as paranoid delusions stemming from cognitive dissonance.

Facts good.
"Intuition" and "belief" not so much.

Bruce ClementNovember 23, 2013 1:35 AM

@NobodySpecial "Crypto generally relies on the difficulty of factoring large primes"

Sorry to nitpick, but factoring any large prime x is simple: The factors being x and 1.

I think you mean "the difficulty of factoring the product of two large primes"

Bruce ClementNovember 23, 2013 1:58 AM

LG Smart TVs spy on users. http://doctorbeet.blogspot.co.uk/2013/11/lg-smart-tvs-logging-usb-filenames-and.html

According to this report by "Doctor Beet", even when set to not collect personal viewing info, these TVs still report channel changes, timing, and filenames of videos on USB drives to web servers.

There's some suggestion that the information is not actually collected as these web addresses return 404 (not found) responses but these could still be logged on the server and, naturally, as they are sent unencrypted could be monitored at any point on their way to the server.

MartinNovember 23, 2013 4:06 AM

@Bruce Clement "Sorry to nitpick, but factoring any large prime x is simple: The factors being x and 1."

Sorry to nitpick, but factoring any large prime x is impossible: The definition of factoring only applies to composite numbers, so a prime number can by definition not be factored. :)

Clive RobinsonNovember 23, 2013 4:38 AM

OFF Topic :

@ Bryan, and others thinking HomeBrew CPU,

The basic logic design of the ALU and register circuitry is relativly easy, as is often the "main memory" control. Whilst performance can be improved slightly by such techniques as pipelining to reduce gate delays and up clock frequencies, it's not the gate delays in general that are the biggest killer in CPU performance.

This fate generaly falls to the instruction set design and importantly how you map it to the underlying logic. In my dead tree cave I have a whole bunch of design notes and papers going back to the 1950's generaly it's the stuff written in the 70's and 80's when "bit slice" and "super computers" had their haydays.

In general the process for designing the mapping is know as "Microcode Design" and is often done in "Register Transfer Language" (not to be confused with later meanings of RTL). You will see refrence to "vertical" and "horizontal" instructions and also "hard wired" or "non mapped" instructions.

A good introductory article in Byte appeared in Jan 87 written by Prof Phil Koopman and you can download it from his web site at CMU,

http://www.ece.cmu.edu/~koopman/misc/byte87a.pdf

When you've read it you will find quite a few other interesting papers etc there including ones on stack based architectures which I'd sugest people have a look at if they want reduced gate / SSI chip count and thus less wiring.

TrivialityNovember 23, 2013 4:41 AM

If a large prime x can be trivially factored into x a 1...

Then the product y of two large primes a and b can be trivially factored into y and 1 :P

kashmarekNovember 23, 2013 11:03 AM

@ OverQuantum:

Nice. This identifies exactly the location of the targets they are looking for.

-OR-

Progoganda, F.U.D. ?

NobodySpecialNovember 23, 2013 1:14 PM

@Bruce Clement - thank you. That explains why my new prime factoring software isn't working as well as I had hoped ;-)

BryanNovember 23, 2013 1:28 PM

My current issues with CPU design mostly relate to recalling all I learned from the late '70s to the mid '90s. Keeping up on CPU and chip design, and programming language design were serious hobbies for me. I was a member of SIGARCH for most of that period. I didn't just limit my reading to SIGARCH. I do love a good research library, especially if it has a good engineering section. I'm the type that would rather learn something new than watch a sitcom.

On making one's own silicon. It may be possible to buy some old FAB hardware. I'm thinking early '80s vintage stuff. I could see making at home the rough level of tech used for 60's vintage FABs.

Another option for making one's own is printing it. I know transistors can be printed as that is done for some LCD displays. Recently there were reports on a researcher at MIT that printed OLEDs on a flexible substrate. Speed won't be there, but then it should be possible to put masses of them in parallel. I'm wondering how soon before they have figured out good materials for transistors that can be printed by current 3D printers. May take a multi head printer, but I see no reason it couldn't be done now.

OT: One of my current active projects is to get my 1 micron resolution XY stage repurposed as the XY stage for a PCB mill. It may be possible to add a 3D print head to it at a later date.

OT: Another active project is converting a toaster oven into a surface mount reflow oven. I have the fan and heating elements controlled. Next step is reassembling the case and writing the control program.

AspieNovember 23, 2013 2:53 PM

@NobodySpecial

FWIW, twin primes, to which you are alluding and prime constellations (multiple adjacent primes) are a particular interest of this guy.
Interesting site.

RobertTNovember 23, 2013 3:18 PM

RE ALU design: I'm not wanting to rain on everyone's parade BUT ALU's simply are not designed this way anymore.

OK think about a high speed CPU. If the clock rate is fixed then the period of the clock is a little greater then the worst case delay to complete an operation.

For a traditional CPU, in simplest terms this "worst case delay" is set by a combination of signal propagation delays AND Bus Drive strength delays.

Lets look at both,
signal propagation delays: function of Gate capacitance, routing capacitance AND drive strength (current capacity). OK so the solution for fast speed digital is LOW gate capacitance, low routing capacitance and high drive strength. OK most of these things are Process rather then design but you can do things in parallel by design to make up for process limitations. (Look ahead carry's, Wallace Tree Multipliers ....) The cost for this design decision is extra power (big problem these days...HUGE problem for any future product)

Bus Drive Delays: Each register that we add to a Bus adds routing length (extra capacitance) and extra Gates / source/ drain areas. So a Harvard style CPU with a Data Bus and say 32 registers has probably 34 or more end points (ALU, RAM plus 32 regs). Usually the delay to drive this is on of the dominate system delays. So if you want maximum speed you need to reduce the number of registers such as ALU connects only to RAM, or Registar bus is separate from RAM bus.

I'm not an ALU designer (the last one I worked on was in the late 1980's) at the time 100Mhz CPU was considered incredible. So Clive's ideas sound good BUT they'd result in a sub 100Mhz CPU (10 nsec clock) because they have not gone down the path of optimizing pipelines so that RAM and Reg transfers can be overlapped with next instruction execution, real important stuff if you want speed.


Bottom line is there are only maybe 100 people in the world that'd I'd trust to design a high speed CPU core from the ground up AND I'm definitely not one of them. In the end you'll have to trust one of them so what difference does it make if they work for you directly or for some big company. They are the ONLY people qualified to make the architectural decisions that'll result in a multi Ghz CPU, if you want the product you MUST trust them.


Nick PNovember 23, 2013 4:10 PM

@ RobertT

What about the existing 1GH+ CPU's whose specs are open? Wouldn't that be more a review than a design problem?

@ Bryan

You mention the importance of SIGARCH but not anything specific. Readers here might be interested in seeing whatever your basing your thought process on. What are a few of the very best papers or whatnot from SIGARCH that would benefit people in hardware redesign efforts?

RobertTNovember 23, 2013 7:44 PM

@Nick P

To be honest for something like this I always go with the person not the spec.

If I were developing it in Europe I'd be looking for help from guys that worked at companies like Acron or Element14 (now Broadcom based at the Bristol UK site)a lot of these guys have focused on DSP type applications like soft DSL cores (basically ADSL/VDSL soft-stack running on a very fast RISC core). Ex Siemens DSL guys also developed a very fast comms CPU core they are now a private company (cant think of the name)

ARM guys also know what they're doing, in the US I'd be looking at the MIP's guys or some of the old HP PA RISC team, it'd definiteky be worth talking with some of Qualcomms CPU guys or Nvidia's GPU gurus. When last I talked with any of them they were looking for something new and exciting. Not sure High security would interest them, but who knows, a lot depends on the market trend.

From a hardware on-chip security perspective there are very few people that really know what they're talking about and I mean a handful of people, so it'd be essential to get these guys involved. Infineon is one of the sources, but frankly I find a lot of their defensive structures anti-probe structures to be a complete joke.

The few synthesisable Ghz cores that I've looked at get the speed by ramping up the power. For most mobile apps these days its the speed power product that matters, not just raw speed. As a matter of fact power equals HEAT which is very hard to get rid off on a very small die. This means forced cooling, expensive packages and a product ill suited to mobile (which is where the market is focused)


off-topic-2November 23, 2013 7:58 PM

I have read a couple of articles in the last week about a group sponsoring an audit of the Truecrypt source code. I support this effort. There is a crowdfunded site on Indiegogo that raised almost twice its target of money and a website istruecryptauditedyet.com that reports progress toward this goal. I have used Truecrypt for a while and I would like to know that my information is safe.

Clive RobinsonNovember 23, 2013 8:52 PM

@ Robert T,

    ... because they have not gone down the path of optimizing pipelines so that RAM and Reg transfers can be overlapped with next instruction execution, real important stuff if you want speed

Like you it's been a while since I designed ALU/CPU and it was back in the 80's, when the 29xxx family was still used to prototype what would become silicon in the likes of Motorola chips (the 68K prototype went on three fullsize Eurocard boards).

Back then where I was "earning a crust" piplining was seen as limited to the process of breaking the circuit down into blocks and inserting the latches, and the business of RAM/Reg optomisation was seen as part of the instruction hard-wiring / microcoding process not the pipelining process.

Realisticaly though as I've noted before the use of TTL SSI chips that were around in the late 60's and early 70's is not the way to go (TTL of that time had gate delays up around 70nS). Homebrew design clocks would be 1-5MHz range using such chips. Even using the likes of PALs won't get you up to much above 10MHz.

Having built multiphase ultra wide bus systems I have a reasonable knowledge of just how much extra logic and wiring are required to do it. And realisticaly it's not going to be within the relms of 99.99% of Homebrew designs no matter what technology they use. Thus the average design clock speen is going to be dictated by just how fast they can get a full adder of the required bus width to work (for those reading along the widest adder is usually found in the memory addressing circuitry, but there are tricks that can mittigate it's impact).

As for people cutting their own silicon, it's not worth it at all sorts of levels, the resources required are way to large unless you can piggy-back onto somebody elses run.

As @ Nick P and myself have intermitantly discussed in the past Open Cores for the likes of FPGA's are probably the best route these days for people to go. But... it comes back to the question of "do you trust the FPGA?". As a number of people have indicated "NO" then they are stuck with limited choices of SSI / MSI TTL chips and the like (and if they are then looking at a dual stack based design might help).

It's why I favour low cost parts off of the shelf from different architectures and manufactures and the use of voting protocols and similar techniques to try and mitigate trust issues, as I've repeatedly said over a period of time.

SomeoneNovember 23, 2013 9:14 PM

http://www.nytimes.com/2013/11/23/us/politics/nsa-report-outlined-goals-for-more-power.html

I can't handle this for much longer. Every day is a waking nightmare. I have viewed the Internet as an extension of my own thought processes until the NSA butted in and turned my life into a nightmare. Every thought I have I have to remind myself that it is not broadcast on the Internet so I can't be retaliated against in the future for them. There's so much I want to do, but I fear actually rocking the boat with ideas I have will get me scrutinized for the rest of my life. I can't handle the thought of every single action I take and every single website I go to being recorded somewhere, just waiting for someone to dig it up and potentially use against me.

Fuck you, NSA. Your actions and goals have transformed my life into a living nightmare.

SomeoneNovember 23, 2013 9:21 PM

And yes, I'm a U.S. citizen. One of the people you profess to protect, you aggressively voyeuristic pieces of shit.

Nick PNovember 23, 2013 11:26 PM

@ RobertT

I appreciate the tips. It's funny you mentioned PA-RISC: William S. Worley, Jr., its main designer, is also the main guy behind Secure64's SourceT OS I've praised here. The praise was because they designed it ground up to be safer and take advantage of Itanium's safety/security features. I think he had a hand in Itanium's design too. Come to think of it, having done PA-RISC and a security-centered OS, he might be the perfect guy for a secure hardware/firmware/OS project.

Of course, the project would somewhat compete with his company's efforts so I'm not sure he'd be onboard. Unless perhaps they could port their stack to whatever was developed and make money licensing it. Still think it's unlikely to get him beyond an advisor role. Oh well, it's not like I have a bankroll to do an ASIC on anyway.

More Serious Point

RobertT, what are the odds that a maker of one of the now defunct RISC chips would straight up license the chip design and/or anything needed to manufacture it? I'm thinking Alpha, PA-RISC, etc. I mean, just to keep the options open as a new project already would have ARM, MIPS and (tks to new initiative) POWER to choose from far as mature ISA's go. Just tweak it, then build it. I'm just wondering if there could be some big savings in using one that has no financial future on its own over one of the popular ISA's. Software support isn't huge deal as mature BSD's/Linux and compilers exist for all of them.

(Not to mention the PA-RISC and Alpha's had some desirable properties.)

RobertTNovember 24, 2013 2:42 AM

@Nick p
Re Licensing RISC cores. Frankly I'd suggest you just rip them off, make an unlicensed equivalent core. The reason I say this is
1) If the project is unsuccessful IP enforcement is too expensive to waste $$$ suing some small fry producer catering to a tiny market.

2) If you succeed they'll all want to understand this new secure device market, so you hold all the cards when it comes to negotiating the license terms. That's how it works if the license holder is another chip company, HOWEVER if the license holder is a Patent troll then they tend to just want lots of money, so check out you holds the patents carefully AND avoid doing the development work in the USA (avoids triple damages rules) and limits patent enforcement to ITC exclusion which still lets you ship to the rest of the world.

One other place to look for talent is the guys doing high speed routers and associated chips, Cavium comes to mind there is a lot of good people there.


One thing to keep in mind is that the IDEAL solution depends on the technology and market focus. For instance Alpha chips NEVER designed or intended for low power applications so it's probably a very bad starting point if you want to chase secure mobile device market. Also the source of on-chip dominant delays has shifted from Gate loading and gate delay in say the 90's to routing and other parasitic loading delays today. So Architecturally minimizing routing can deliver huge speed increases, these days routing loading impacts things like the clock tree layout, so some designs will try to get Clocks and data propagating together through a chip. It's a long way from the early days of single clock source CPU's.


RE Bank rolling secure processor: I definitely wouldn't hold my breath waiting but I think a lot of the recent NSA revaluations have got 2'nd and 3rd tier gov'ts and many big companies thinking about data security, so this might be exactly the sort of market dislocation that will enable a new player to break into the chip market. I've just got no idea how you really sell added security.


493g4oi9jNovember 24, 2013 3:45 AM

Funny thing. I've always used Outlook for temp. email accounts using Tor and then delete them after use (at least some sort of security, in my mind). Now they've broken the account delete process by sending you to a msn web page that doesnt do anything. Basically u cant suspend your accounts any more.

Very strange. I wonder where I can find a good webmail that preferably dont use javascript and that is easy to suspend (make unreachable from the net).

Clive RobinsonNovember 24, 2013 4:36 AM

@ Nick P, et al,

Right now the chip business is moving a little more rapidly and fluidly than it has in the past so interesting times are here again.

One such item of news from earlier this month is Intel plans to open it's foundry doors to making product for other potentialy rival organisations.

The details are sketchy (but when are they not) but the news is that Intel's new president Renee James told analysts and investors at Intel's investors day,

    “We are evolving and expanding our business model,”

And further the new CEO Brian Krzanich appears to have eliminated the precondition of his predecessor Paul Otellini of not "enabbling competitors".

It may be due to the fact that the PC market is not growing and thus the "flat" expectations for the next financial year.

http://www.benzinga.com/analyst-ratings/analyst-color/13/11/4103168/mixed-response-from-analysts-following-intels-investor-d

And Intel looking to get some kind of return on the 10nm investments.

However how much competitor enabeling there is going to be and on what terms is unclear. James stated that Intel will evaluate prospective foundry clients "even in areas where there may be some competition with businesses that we're in" but on a “deal by deal basis, not on an architecture by architecture basis.”

Thus it might involve more "cross licensing" technology swaps for a number of reasons such as getting something from the currently expanding "smart mobile" market. Or even potentialy heading off the mess that is the legal war currently raging in the mobile market that is so destructive even the UPSO is feeling the heat.

However with other recent Ed Snowden revelations of the NSA "Four Year Plan" by the NYT with it's emphassis on getting it's tenticles into the commercial technology sector for backdoors and the like, I'm sure there are going to be many taking a very jaundiced view, worse than even that over the Intel RNG...

Which would actually make the idea of a secure chip much more "investor friendly" which might see quite a few punts at "angels" in the near future ;-)

WinterNovember 24, 2013 4:57 AM

@Nick P ao
Producing secure CPUs is a worthy goal, but will take some time. Also, what use is a secure CPU when the screen will broadcast screen shots? Just to take a wild idea.

I wonder what can be done already by simply containing the communication abilities of the computer hardware.

Most attack schemes I see discussed assume the hardware will communicate with the attacker either over wireless or over a wired network. So, if you can control what goes out or gets in, you could already block many attacks now.

I really wonder how far this could go?

Faraday cages for all appliances (computer, screen, keyboard etc), shielded cables (use radiation to roast hidden electronics), decoupling power from grid, insert a router/proxy with very aggressive packet filtering and Tor only external connections. External (packet) logging? Maybe even several such filters and compare performance?

The idea is that if side channels can be contained, all attacks will have to go through every link in the chain between my work and the open net. An attack can be blocked at every link in that chain. Which makes the odds a little better. But I have no idea how much better.

DTWGNovember 24, 2013 7:33 AM

We are watching you. We are tracking you. We save everything. Don't think that you are free, you are not. You are meaningless, stay like that, or else.

BryanNovember 24, 2013 8:13 AM

@Nick P

You mention the importance of SIGARCH but not anything specific. Readers here might be interested in seeing whatever your basing your thought process on. What are a few of the very best papers or whatnot from SIGARCH that would benefit people in hardware redesign efforts?

SIGARCH is the Association of Computing Machinists Special Interest Group on Arcitecture.
SIGARCH serves a unique community of computer professionals working on the forefront of computer design in both industry and academia. It is ACM’s primary forum for interchange of ideas about tomorrow’s hardware and its interactions with compilers and operating systems.

Most of what I'm thinking about right now is snippets drifting out of my memory that fit the needs I'm seeing at the moment. It has been over a decade since I actively did any reading up on computer system design, and it looks like I need to do some updating of what is current state of the art. I really don't have any specific articles I can recommend right now. Also my memory is such that I could tell you the concepts I learned from various papers, but not which papers they were in.

@Someone and everybody else

Time to write to all your congress/Parliament/whatever critters and tell them what you think of the NSA spying, and that it must stop. Stay polite, and it works better if it is your own words, not the words of some group.

@Nick P

I appreciate the tips. It's funny you mentioned PA-RISC: William S. Worley, Jr., its main designer, is also the main guy behind Secure64's SourceT OS I've praised here. The praise was because they designed it ground up to be safer and take advantage of Itanium's safety/security features. I think he had a hand in Itanium's design too. Come to think of it, having done PA-RISC and a security-centered OS, he might be the perfect guy for a secure hardware/firmware/OS project.
Now, how to get it funded?

@RobertT

I've just got no idea how you really sell added security.
Insurance against loss of company secrets.

@Winter

Producing secure CPUs is a worthy goal, but will take some time. Also, what use is a secure CPU when the screen will broadcast screen shots? Just to take a wild idea.
Eye glasses display. Anyways, you can't plug every potential leak. If they are watching your screen with a camera, then they are dedicating huge amounts of dollars tracking you.

On BGP route injection attacks. Here's a possible scenario. Infect a computer. Change the route to a popular site that is used to being seen used at the target site to go past one of your sniffers. Start the data dump of the sensitive information. Once transfered, change the routes back.

JacobNovember 24, 2013 12:35 PM

From the NYT article:

"document makes it clear that countering “ubiquitous, strong, commercial network encryption” is a top priority. The agency plans to fight back against the rise of encryption through relationships with companies that develop encryption tools and through espionage operations. In other countries, the document said, the N.S.A. must also “counter indigenous cryptographic programs by targeting their industrial bases with all available Sigint and Humint”

Scary.

WinterNovember 24, 2013 1:20 PM

@Jacob
So Bruce and others were right that we should support the use of encryption everywhere?

Or is that what they want us to believe ;-)

BryanNovember 24, 2013 2:15 PM

@Winter

@Jacob
So Bruce and others were right that we should support the use of encryption everywhere?

Encryption is only part of the answer. From the NSA disclosures, it looks like they have throughly compromised the standard computer model in use today.

Nick PNovember 24, 2013 3:39 PM

@ Winter

Once again it's only a small part because:

1. Many attacks hit unsafe software constructions (widespread problem)

2. Many attacks use poor configuration or overprivileged processes as a vector.

3. The users are always happy to install software on their machines thats built in US.

4. If these dont work, lets recall that NSA compromised both SSL and IPsec with protocol level issues that took years to come out.

WinterNovember 24, 2013 4:26 PM

@Nick P
Attacking "interesting" computers is (much) more work than eavesdropping. Moreover, you have to know whom to target. Which requires eavesdropping.

Spearfishing is good to catch a few fish. You need a dragnet to haule in the whole school of fish.

Nick PNovember 24, 2013 5:57 PM

@ Winter

It's certainly a nice hurdle to put up. Thing is, how many people are really worth hacking to them at any given point in time? It's probably closer to thousands than millions. We do know they can grab info by the billions, built weaknesses into crypto standards to automate attacks, regularly turn routers/CAs against users, and have hacked at least 50,000 machines. Put it togetger, they have more than enough resources to demolish security of any small organization trying to hide with crypto on mainstream machines or networking gear.

The only digital method of beating such an adversary involves high assurance secutity processes. For systems or software, there's less than 200 such people worldwide according to an older estimate. RobrtT indicates there are similarly few guys trusted for high end hardware architecture and very few fabs to produce their chips. The NSA could target all of them at the same time to ensure they dont become a threat (or subvert threatening efforts). They can also target companies people tfust for security.

So, this is a larger problem than crypto can solve. Making Internet leak less info overall is a good thing. Im for a major overhaul of it myself. It's just not gonna keep NSA in check for situations people here woukd *really* worry about. Crowds creating such situations are small enough and have poor enough OPSEC that NSA could probably manage that despite extra crypto.

Except for mike with his crypto over serial data diode to obsolete non-Intel PC and whatever Clive uses in his "cave." If need be, Im sure they could get an entire team on that problem as quickly as they can sign a Gamma contract. ;)

Dirk PraetNovember 24, 2013 8:26 PM

@ Nick P

We do know they can grab info by the billions, built weaknesses into crypto standards to automate attacks, regularly turn routers/CAs against users, and have hacked at least 50,000 machines

50k Belgacoms is a seriously disturbing thought. What I really would like to know however is if Dutch company Fox-IT whose consultants alledgedly found and removed the NSA/GCHQ malware at Belgacom has sent a copy of it to VirusTotal or has shared it with the major antivirus/malware companies. Or what leverage has been used against them, and by whom, not to do so. One would think that this case that has been in the news all over the world would prominently feature on their website, but quite surprisingly it is not mentioned anywhere, which really makes me think that they're under some kind of pressure to keep the details of their work at Belgacom very low-profile.

FluffytheObeseCatNovember 24, 2013 8:34 PM

"Encryption is only part of the answer. From the NSA disclosures, it looks like they have throughly compromised the standard computer model in use today."

Bryan, that is kind of the worry, isn't it? A titch paranoid perhaps. Regardless, it has to be a conceptual starting point for anyone who is serious about counteracting their dominance. It's the baseline assumption for action.

Those of you who know what you are talking about...... tell me I'm wrong. I've seen many febrile kooks posting in Bruce's comment section. Tell me I'm among them. Please.

GarfieldNovember 24, 2013 10:45 PM

These may have been posted somewhere here by others, in which case my apologies.

Taking the “Meh” out of Metadata

http://www.slate.com/articles/news_and_politics/jurisprudence/2013/11/nsa_and_metadata_how_the_government_can_spy_on_your_health_political_beliefs.html?wpisrc=burger_bar

How the government can discover your health problems, political beliefs, and religious practices using just your metadata.




NSA infected 50,000 computer networks with malicious software

http://www.nrc.nl/nieuws/2013/11/23/nsa-infected-50000-computer-networks-with-malicious-software/

The American intelligence service - NSA - infected more than 50,000 computer networks worldwide with malicious software designed to steal sensitive information.

A management presentation dating from 2012 explains how the NSA collects information worldwide. In addition, the presentation shows that the intelligence service uses ‘Computer Network Exploitation’ (CNE) in more than 50,000 locations. CNE is the secret infiltration of computer systems achieved by installing malware, malicious software.

One example of this type of hacking was discovered in September 2013 at the Belgium telecom provider Belgacom. For a number of years the British intelligence service - GCHQ – has been installing this malicious software in the Belgacom network in order to tap their customers’ telephone and data traffic. The Belgacom network was infiltrated by GCHQ through a process of luring employees to a false Linkedin page.

NSA special department employs more than a thousand hackers

CzakóNovember 25, 2013 12:21 AM

The agency also intends to improve its access to encrypted communications used by individuals, businesses and foreign governments, the strategy document said. The N.S.A. has already had some success in defeating encryption, The New York Times has reported, but the document makes it clear that countering “ubiquitous, strong, commercial network encryption” is a top priority. The agency plans to fight back against the rise of encryption through relationships with companies that develop encryption tools and through espionage operations. In other countries, the document said, the N.S.A. must also “counter indigenous cryptographic programs by targeting their industrial bases with all available Sigint and Humint” — human intelligence, meaning spies.

http://www-nc.nytimes.com/2013/11/23/us/politics/nsa-report-outlined-goals-for-more-power.html?=hp&_r=6&pagewanted=all&

WinterNovember 25, 2013 1:17 AM

"One would think that this case that has been in the news all over the world would prominently feature on their website, but quite surprisingly it is not mentioned anywhere, which really makes me think that they're under some kind of pressure to keep the details of their work at Belgacom very low-profile. "

From the official releases I understand that Fox-it is bound by what Belgacom wants to be released. They are not free to discuss what they find at clients.

Furthermore, I suspect they are put under pressure by the Dutch government, the Belgian government, the EU, and Nato to be very careful with what they say and release.

Remember that the Belgacom network that was hacked was also used by EU and Nato "headquarters" in Brussels. Just blurting out that the British were behind this, and how they did it, might be very unwise for Fox-it.

It is in the newspapers that the people in the EU parliament are a "little" upset about this. The others keep quiet. But I expect that Nato partners and the EU commission too are not amused.

The 'beasts' share the same scent - how to piss off an alien/human hybridNovember 25, 2013 4:11 AM

the hybrids carrying filthy spawn (like in the days of Noah) are easy to SNIFF out, literally, they all smell the same when you're in the proper state of mind.

some of them have eyes which appear to be bugging out of their face.

even if you can't detect the scent of the hybrids, or 'beasts', inhale deeply whenever the hybrids are close, don't express any emotion, just keep inhaling deeply and make your facial expression be that of deep contemplation.

when you do this, they know that you know what their true reality is - it's like the movie THEY LIVE where Nada sees the truth through the glasses and confronts them.

don't confront, just inhale deeply. maybe shake your head and laugh, mumble about stupid aliens but nothing deep.

JabbaNovember 25, 2013 5:37 AM

@ Winter

> Faraday cages for all appliances (computer, screen, keyboard etc), shielded cables

There is supposedly a keyboard about that uses key presses to interrupt light beams conveyed by fibre to near the keys so reducing electrical signalling. And the keyboard scan could be encrypted.

Clive RobinsonNovember 25, 2013 6:14 AM

@ Winter,

    Attacking "interesting" computers is (much) more work than eavesdropping. Moreover you have to know whom to target. Which requires eavesdropping

It depends on your definition of "eavesdropping" is it on routing metadata or content?

Whilst content can fairly easily be encrypted it usually still leakes information by packet size, number of packets and frequency/speed/nature of packet transfer. This is because the encryption is usually just "bolted on" as an after thought to an existing and well charecterised service. Such analysis is relativly simple in most cases and such information whilst far from reliable still makes quite a good indicator of "interesting".

Further on it's own a host cannot hide routing or port information due to the necessary "point to point" nature of all large multi point networks. Such information is again can make a good indicator of "interesting". However a properly designed "mix network" can hide such information.

The problem is few mix networks exist and the few that do are usually not propperly designed to withstand traffic analysis and thus information can be correllated at entry / exit points and also at network choke points, so though harder "interesting" traffic identifing information leaks out.

But as has been seen, the user tools such as web browsers can be "gamed" to provide information either directly or indirectly as can web servers. A major part of this problem can be laid at the feet of "rich content" presentation requiring an individual user experiance thus programable content...

Further the user tools such as web browsers whilst being "applications" are nolonger single client single server in nature and use, and thus bypass many of the security mechanisms provided by the OS, in effect taking modern PC's back to Win95 level security...

Sorting out each and every problem is a Ageian Stables style problem and requires an almost similar solution, which is very unlikely to happen.

And worse for the most part the NSA is playing the game of the passive preditor, like a spider that sits and waits for it's extended web of sensors to provide "interesting" indications as and when they occure, then jumping into action to inject poison into the pray. But unlike nature due to the mass hoovering and storage of data they can move backwards in time throught their archive. So if new identification techniques are discovered today they can in effect be run the day before yesterday or last year... Thus if you make one "human error" at any point they will be able to find it and use it...

But at the same time the NSA have a busy team of people aquiring the equivalent of "zero days" not just on PC & Server's but any infrastructure they can find an opening into. But unlike most of us who can only look for "zero days" they have people they control sitting inside the companies making "zero days", and people hobbling protocols and standards in ways we mostly have not dared to guess at, and when a few do, they are generaly either ignored or not treated well. And thus few or no precautions are taken.

So yes "attacking" interesting targets might appear harder than finding them as a task for you or I, but not realy for the likes of the NSA, and it's a task they have well in hand.

WinterNovember 25, 2013 6:51 AM

@Clive
"So yes "attacking" interesting targets might appear harder than finding them as a task for you or I, but not realy for the likes of the NSA, and it's a task they have well in hand."

My speculation was based on the professed fear inside the NSA for blanket encryption of data.

So, that fear for encryption of data is then limited to those attacks that cannot rely on meta-data but really need the content. That would be large pipes hosing data like those between Google data centers. Or maybe also for contact lists on-the-fly?

It also seems to be a lot easier when they can directly get the text of, say Airbus bid for a tender or non-USA bids for a Brazilian air port.

Dirk PraetNovember 25, 2013 7:15 AM

@ Winter

From the official releases I understand that Fox-it is bound by what Belgacom wants to be released. They are not free to discuss what they find at clients.

Well, it stands to reason that Fox-IT and similar outfits are under NDA to disclose only what the customer allows them to, but I very much doubt that Belgacom is the prime actor preventing them from doing so. If indeed they have discovered and removed some piece of NSA malware that may be sitting on thousands of other networks, then Fox-IT is acting AGAINST the public interest not to share their findings with others to the point that they are complicit with the perpetrators who infected those networks in the first place. This is exactly why we need international laws and regulations for this type of incidents.

To me, this is exactly the same thing as some medical team discovering a dangerous new virus strain in the wild and then being prevented by the CDC and the USG from telling the world about it.

Clive RobinsonNovember 25, 2013 8:02 AM

@ Nick P,

    Except for mike with his crypto over serial data diode to obsolete non-Intel PC and whatever Clive uses in his "cave."

Ouch, that makes me sound like a trogladite :-)

My dead tree "cave" is my library and it's stacked from floor to ceiling with old fashioned printed books and papers etc which only need a working eyeball as a "reader". However calling it a library would have a few people here pulling "Col Mustard, with the lead pipe" comments.

I do my "stuff" in the lab or workshop depending on what I'm doing (metal work produces "swarf" which does not work well with electronics).

As I've said befor I use old PC's and development boards for various microcontrolers to do some of my stuff and like Mike I do have an old Sun Sparc in what was once called "lunchbox" format that went under the monitor, but it's sitting up in the loft unused with Sun OS 5.7 under Solaris 7, ontop of another unused Sparc telco grade system with Solaris 8. But I'm keeping an eye on the illiminos project for the Raspberry Pi it sounds interesting ;-)

Essentialy I've developed microcontroler based diodes / gates / pumps / sluices that have serial interfaces. Whilst some of these are traditional 7bit ASCII DCE interfaces some do 8bit SLIP or PPP with dial up modem support. Whilst I have built some with 10 meg ethernet they realy cann't keep up with data at those rates except for very simple tasks. I also use PC104 systems to make network appliance devices some of which run on extended RTOS, some on striped down Linux, and one of these days when I have the time BSD.

Essentialy they are for security by "choke point" between segregated untrusted units. That is you assume neither side can be trusted and you mandate the acceptable traffic policy through the gate.

One thing I'm thinking about is how to "whitelist" user network requests such that when you click on a link traffic is limited by the gate to that IP address or site only and not to any other site. At first sight it's a simple problem, the reality is it's not.

kashmarekNovember 25, 2013 9:27 AM

For those of you that followed the U.S. laments about Chinese communications equipment and alleged spying efforts, our government admits guilt (not that they wanted to) in doing EXACTLY the same thing in the far eastern arena:

Singapore & South Korea help NSA tap undersea cables

http://yro.slashdot.org/story/13/11/25/0546240/http://yro.slashdot.org/story/13/11/25/0546240/singapore-south-korea-help-nsa-tap-undersea-cables

Which begs the question: will documentation of such activities be prohibited such as to avoid this type of disclosure?

We now have known rogue agencies performing know rogue activities (not all is know yet). One can expect to have stealth agencies (multiples to disperse the chance of discovery) performing stealth (illegal) activities leaving no trail for discovery, except perhaps by each other. The disaster of this is when one of those agencies falls under control of a foreign government or organization that wants to change our existing national foundation to something inappropriate for the country. This will not be the rise of the machines (or SkyNet so to speak; a distraction), but the rise of the behind the scenes mechanism for totalitariun control.

Nick PNovember 25, 2013 9:40 AM

@ Clive Robinson

"One thing I'm thinking about is how to "whitelist" user network requests such that when you click on a link traffic is limited by the gate to that IP address or site only and not to any other site. At first sight it's a simple problem, the reality is it's not."

That's actually better done at the endpoint, esp browser. The browser security architectures I've posted can all be setup to enforce that by prevention or isolation. Copy them. If you're wanting an extra layer of defence in networking system of the host OS, you can always offload TCP/IP onto a device with high assurance stack or operation. Then, your app on the host can use the vanilla TCP/IP interface knowing (a) the incoming packets are safe and (b) some filtering is happening.

vas pupNovember 25, 2013 9:46 AM

www.history.com/shows/your-bleeped-up-brain
The main lier is our own brain - amazing tests proving that (including pure deception part) as well as person is always the weakest link in security. Enjoy!

Nick PNovember 25, 2013 9:53 AM

@ kashmarek

I wonder if this means we should add Japan to the list of countries with which to obtain NSA resistant products or services. My concern is that this might not be indicative of any real resistance to NSA or snooping: it might have been rejected due to high resources and risk.

To people following all the leaks and other espionage revelations: what is there that will tell us about trustworthiness of Japan against NSA?

My reason for sourcing Japanese equipment in the past was that reports of industrial/economic espionage showed Japan to be highly active against the US. I figured US didn't have backdoors in equipment produced and used by such an opponent. I did, however, assume that Japanese intelligence would know anything done on the device. Always a safe assumption to think of host country as a risk.

RobertTNovember 25, 2013 4:52 PM

@Clive Robinson
they have people they control sitting inside the companies making "zero days", and people hobbling protocols and standards in ways we mostly have not dared to guess at, and when a few do, they are generaly either ignored or not treated well.

It's amazing how effective a quiet word whispered (well not really whispered) in right ear can be. You have the unfortunate problem that those that either know whats happening, or have the smarts to figure it out, are all known players.

Most somehow making a living from the system existing as it is, so really "improving" security is not even in their own best interests. Those with higher moral / ethical standards usually have families so their independent wills are easily bent.

Frankly I've no idea how we move forward from this point, but I suspect the push will actually come from the worlds emerging countries. They'll develop methods and systems that simply bypass the US data collection engines, we are seeing the beginnings of this realization with the Indonesia/Australia and China/US spying storms.

AnuraNovember 25, 2013 6:08 PM

@RobertT

My fear is that even if another country steps up, that the US will still be able to subvert it.

We need the public of the US and other superpowers to elect officials that will fight for privacy, fight for placing our ability to secure our systems above our ability to defeat the systems of others. Unfortunately, the public seems to be less and less in control of government these days.

We are being hammered with propaganda, distracted by scandles. Even our politics are a distraction from the issues, engineered to keep power in the hands of a small group of people. The media is always turning every issue into Democrat vs Republican, presenting "both sides" (usually a false dichotomy). The media loves it; it's best for ratings if you keep everything simple and confrontational, the wealthy love it because it's easier to control the voters if they are uninformed, and the Democrat and Republican parties love it because it makes sure that they have no challengers (which, thanks to plurality voting, is incredibly easy to do). I hate it, because it is killing intelligent debate.

Ever read the comments of a site like CNN? It's horrid, and quite scary (especially so for the followers of Fox News, who seem to hate socialism and communism with a passion but don't actually know what they are). There seems to be very little dissent between members of any team (I get called a communist for supporting an increase in minimum wage, and a teabagger for not supporting an assault rifle ban). Until the public recognizes the propaganda and starts debating the points made instead of the side taken, nothing will ever change. What's worse is that we risk these developing countries following our model.

BryanNovember 25, 2013 7:07 PM

Ever read the comments of a site like CNN? It's horrid, and quite scary (especially so for the followers of Fox News, who seem to hate socialism and communism with a passion but don't actually know what they are). There seems to be very little dissent between members of any team (I get called a communist for supporting an increase in minimum wage, and a teabagger for not supporting an assault rifle ban). Until the public recognizes the propaganda and starts debating the points made instead of the side taken, nothing will ever change. What's worse is that we risk these developing countries following our model.
Many people can't handle shades of gray. Their world is all us vs. them. So, if you aren't an us, you are obviously a "them" and get labeled as such. The other thing is many also don't spend the time to get informed. They only listen to one voice and ditto their favorite pundit's stances.

My biggest worry right now is the state of the press. The plurality of views is not making it to the people. Hence, no debate on the issues.

Anybody have a few tens of millions?

BryanNovember 25, 2013 8:16 PM

Security Theater...
http://www.popehat.com/2013/11/25/dispatches-from-the-junk-science-front/

Ken White's article starts out:

In 2008 I pointed out that the TSA's pseudo-scientific "behavior detection" program seemed almost indistinguishable from random chance. Five years and millions of gropes-by-government-agents later, the General Accounting Office agrees:

Articles linked by Ken.
http://www.usatoday.com/story/travel/flights/2013/11/13/gao-tsa-behavior-detection-spot/3518775/

http://www.cnn.com/2013/11/14/travel/tsa-pistole-behavior-detection/

Nick PNovember 25, 2013 9:44 PM

@ Dirk Praet

It's a nice idea but it has the same risks of an article saying, say, "an American embedded system designer... hardware... protect users from unwanted snooping." If you're first thought would be "his country would subvert it if it was important," then you'd be correct. Australia already filters the Internet and is closely connected with US intelligence. I can't remember if they participated in Echelon, but I do know NSA trusts the Tenix products.

At least the software might be usable as it can be peer reviewed. A somewhat paranoid person could have the software peer reviewed and ported to hardware of their choice. So, the project might still be useful.

FigureitoutNovember 25, 2013 11:37 PM

Frankly I've no idea how we move forward from this point
RobertT
--How about you talk w/ your contacts and ask if they can give public contact info or ask yourself? I don't know who to look into the companies you stated and how to check their security; and I'm going to have to do some traveling to check out some Fabs in the US (which are still highly questionable for this project). Stop living your life afraid of little tweety birds; or imagine life when all production of chips is military controlled.

We still need an agreed upon build obviously, we need to make this effort mostly resistant (the best will slip thru) to agent subversion (so redundant checking by core members of the team and then again by all, no secrets) and ensuring a vetted team member competent in handling virgin chips keeps them clean until one-time-programmed. This is a massive task I realize (no need to patronize), I'm researching and I think I'm going to try some college-research trips to some labs next year.

It's been posted before by Nick P: http://en.wikipedia.org/wiki/List_of_semiconductor_fabrication_plants

http://wiki.fablab.is/wiki/Starting_a_FabLab

http://wiki.fablab.is/wiki/Portal:Labs

http://p2pfoundation.net/Fab_Labs

http://www.minifablab.nl/
--This one is a possible path for trusted purposes. Mini-fabs (I happen to know the town Utrecht,NL); European fabs may not even be trusted though.

So much potential for this project, and which way it will go. Go for money-making and highly functional computer w/ more security risks or a hardened minimalist "boring" computer that obeys its master at all times.

First steps in my view are multiple secure labs scattered all over and start accumulating some money. Also, what computers and other IC's are already being used at the fabs that may allow remote subversion?--Creation of the tools to do the job may even be another starting point which makes this project even harder...

Mike the goatNovember 26, 2013 1:54 AM

Figureitout: I wonder how many old SPARC and 80386 chips there are lying around in inventory rooms around the world. I know that one local aerospace autopilot vendor relies on old stock of 80386 processors.

FigureitoutNovember 26, 2013 7:45 PM

Mike the goat
--Yeah you don't want your autopilot behaving like that drone that crashed into a navy ship. I'll take 'em but I wonder how they have hidden memory sectors and backdoor commands/sequences. I don't want to think that anymore b/c it's a major distraction to real work/progress so a secure chip project needs to continue.

Dirk PraetNovember 26, 2013 8:44 PM

@ dbCooper

Re. The internet mystery that has the world baffled

The wait is on for January 4th 2014 when many Cicada 3301 enthusiasts all over the world expect/hope for a new series of puzzles. And a well-paid job at the NSA/GCHQ/FSB/PLA, I assume.

JamesNovember 27, 2013 1:59 PM

Some folks from Nokia went and started making their own phones under the name Jolla and they have now come out with their first phone (www.jolla.com).

It would seem like a good niche if they put forth effort to make their phones secure against eavesdropping.

DMCCNovember 29, 2013 10:25 AM

Mr Schneier, i searched your site for "Bipedal Walking Octopus" and came empty handed! You do realize that the objects of your obsession are evolving, right?:

http://wn.com/bipedal_walking_octopus

Care to comment on what happens when they evolve to *your* level? Will humans go the way of the dodo, due to a competitive disadvantage, you know... arm-wise? xD

Nick PNovember 29, 2013 2:23 PM

To all working on secure comms

Just found something cool I missed a while back. I've often said that govt makes two standards: a subverted one for us and a good version for them. The NSA's Type 1 systems are probably pretty good. They go by names such as FIREFLY, HAIPE, SCIP, custom ciphers, etc. SCIP, for instance, is their protocol for secure voice over a variety of mediums. They keep the details of most of this secret minus some leaks I've posted here on occasion. So...

SCIP Specification Documents
https://www.iad.gov/SecurePhone/documents.cfm?j6lj7MLH01PpRDBuMa66UU7GyePn0P6PlX0BM7n0j5A=

Released in 2011, it gives you plenty of details on SCIP. They also have an analysis of applying it over IP. I wonder if this is accurate, usable information or tainted. So, I'm posting the link here for other's review, particularly those with crypto or wireless experience.

FigureitoutNovember 29, 2013 2:59 PM

Nick P
I've often said that govt makes two standards: a subverted one for us and a good version for them.
--Do you know anything about PS/2 connectors? I've started messing w/ some and even though all I've seen is the extra 2 "unused" pins on a 6-pin DIN connector could be used w/ a Y-splitter or something to use one port for 2 devices (mouse & keyboard).

I wonder b/c there's a 5-pin DIN connector but what is the purpose of that since you need clock and data lines, no? Also PS/2 ports aren't made for lots of unplugging due to pins starting to bend, so maybe just leaving room for 2 extra lines in keyboards/mice PS/2 connectors and their cables.

FigureitoutNovember 29, 2013 3:05 PM

Nick P
--BTW, you're link is showing up as untrusted on my browser. Do you have another one?

Nick PNovember 29, 2013 5:05 PM

@ figureitout

Been too long since I worked on PS2 devices for me to remember any details. I do recall the ps2 keyboards leak data over power lines and public papers exist about that. I used to advice a generator, battery or power line filter for that.

Far as untrusted site, that usually happens with govt sites that use DOD's CAs. The CAs themselves arent default in browsers and govt webmasters often change sites in ways that break them. Just use same precautions you do with a non SSL site to limit risk. Cuz remember that all SSL does is say someone possessed a cert: the whole Internet should still be considered untrustworthy due to protocol and host risks.

Leave a comment

Allowed HTML: <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre>

Photo of Bruce Schneier by Per Ervland.

Schneier on Security is a personal website. Opinions expressed are not necessarily those of Co3 Systems, Inc..