Schneier on Security

Nick P • April 21, 2014 10:31 PM

@ DB

“The problem with the closed source model is that, suppose I don’t trust any of those sources and I want to review it myself, then the NDA might become too high of a bar, not only for me, but for a lot of security professionals… ”

I understand the feeling. I’ve been feeling it a lot more recently.

” I am a bit untrusting of your claim that open source has so much less review than closed source. Sure some of it does, even much of it… but all of it? even all the popular stuff? by default? hmm…”

I wasn’t saying OSS had less review. I was saying that it’s had fewer high assurance designs, evaluations, etc. Almost all OSS code is the run of the mill variety you see outside open source. “crappy code” as you pointed out. Then, there’s some made by good developers with a quality focus. OpenBSD and Bernstein’s work stand out there. There’s some with good security architecture, such as Dresden’s TUDOS and Shapiro’s EROS. I can think of very, very few that used an EAL6-7 style development process. Whereas, for closed source, I instantly gave you three that were also put in production use.

“market forces are always causing companies to take shortcuts and rush stuff out before it’s ready, it’s basically a business requirement to stay afloat in almost all software companies as far as I can tell.”

That’s totally true. I’ve said it here myself. Lipner said it best. He was worth listening to as he’s done it both ways: shipping as much as possible with Microsoft’s SDL & doing an A1-class VMM with Karger. This is a problem that can only go away if the model is nonprofit, subsidized, regulated, or subscription (revenue per company instead of per sale). I don’t see the problem going away for majority of market anytime soon.

” I CAN fix the open source one… eventually, at least.”

Rarely works out in practice for a complex application. It is a possibility, though, with open source so an advantage of it. Remember, though, that source can be shared with commercial users under NDA if needed. Company can also get service revenue by doing fixes, extensions, etc on clients behalf charging reasonable rates. The latter option aids secrecy of source, gets client what they want, and makes developer more money. That said, some model with open source wins on this one.

“You speak of “high assurance” as if it’s a specific kind of technique or methodology that nobody would ever do normally because it’s the most un-fun thing in the universe… ”

If you doubt that, you’ve never done it. 😉 Ok, certain kinds of people are attracted to it & get a perverse sense of enjoyment. (eg moi) There are also newer tools and methods that aren’t so bad. The problem is that they’re somewhat unproven. If we use “tried and true,” it’s an extremely detail-oriented process that requires the vetting of every aspect of design, development, config management, documentation, use cases, testing, covert flows, deployment, recovery, etc. (see EAL6-7 and my framework, then compare to OSS project of your choosing) According to Lipner, making one significant feature change on VAX Security Kernel took “several quarters.” That it took that long shows you how rigorous and time-consuming their development process was.

So, quite simply, would you rather work at a shop that did code reviews + testing or a shop that did the above process? If you’re like most developers (that care), then you will compromise by choosing option 1. As a bonus, you have more hair on your head, better health, and longer lifespan. 🙂

“and actually do save you a lot of time and hair pulling in the end… Surely less hair pulling is more fun? :)”

I mention hair then see this haha.

“There isn’t a way to do things in a much more “safe” way that isn’t so terrible? I mean, real security, not just piling on paperwork and certifications and spending money.”

It isn’t just about spending tons of money & doing paperwork. That’s actually the parts of Common Criteria I’d rather ditch. The benefit is in the process that higher EAL’s force on you. They absolutely force you to minimize bad stuff while maximizing good stuff. I mean, last thing you want is to put in that much time and money, then be told the product doesn’t make the cut. That said, I do prefer private evaluations that focus on the good stuff while trimming the fat of the process.

And remember I’m talking about high assurance: medium assurance that stops plenty riff raff, but not high end attackers, is often possible at reasonable costs. More tools and precedents for that all the time. 🙂

Nick P • April 22, 2014 10:26 PM

@ DB

“we have to somehow develop “higher” assurance techniques that are cheaper, easier, more fun, and can be more widely deployed. It’s either that or give up and give in. Heil Obama.”

Yes. There’s ongoing work in this area. Making it work enough that average developer can jump in and produce results is a necessity.

“Since so many projects (both open and closed source, actually) don’t even HAVE a test suite… at all… (what is that, EAL0?)”

EAL(-4) if they use signed integers, EAL0 if not.

re Tests, Refactoring, Code analysis, etc.

Yes, it certainly can make a big difference. One tip I can give here is that an objective review process of design, code, tests, etc. produces the largest security ROI. The other things you mention are all good. The focus on test-driven development is a modern thing that benfits quality, but not as much as a strong review process. So, I’d say a good choice of language, safe coding style, reviews of any artificts created, black box tests, fuzz tests, regression tests, and static/dynamic analysis. This combo doesn’t take so much effort. And like you said of many of these things, it can make a “dramatic” difference in quality. The old Cleanroom and Fagan Inspection processes had empirical evidence backing that claim, too, so the facts are on our side. 🙂

“I still know people who write code and still haven’t touched a version control system!! I’m like, “what the heck, dude… do you not care?””

I’ll add that it’s an explicit requirement even in Orange Book era, it benefits development in so many ways, and if properly set up it can help address problem of malicious developers. The latter is an issue in High Robustness projects & NSA’s actions have proven it’s especially relevant. Previously, I recommended Aegis and/or OpenCM as good designs for security-centered projects.

“And that’s the thing: when I write code for pay, I CANNOT write high quality code. I am forced by management to write shoddy code. Always. ”

That’s sad and quite believable. In your case, what does that specifically mean? Do you have to use bad constructions? Do they force you to throw something together without testing? Do they ask you to leave the bugs in as they don’t care or can fix them for money later? I’m curious as to what happens in your area.

” But when I write for free in my spare time on an open source project, it’s totally different. I’m free to create quality works of art with my code.”

Truly. It’s amazing what I’ve seen people come up with when they have all the right incentives and time to make it work. It’s why the volunteer and open models have so much potential. Yet, they haven’t delivered on it by far in INFOSEC. My hypothesis has always been a combo of a lack of knowledge/wisdom transfer to next generation, inertia in the industry, and that finding flaws is more stressful than fun for majority. The problem must be solved or a continuous stream of problems will continue to occur. That simple.