Schneier on Security

Nick P • January 29, 2013 12:51 PM

Complexity is an enemy to security. We have all discussed this plenty on the blog. Bruce’s post was a start. Years on, we’ve discussed how to manage complexity of system development and put more predictability into software methodology. I’ve often cited research and actual production use of such technology. So, what to say at this point in response to comments?

First off, congrats to Vles for beating me to DJB’s excellent paper. 😉 The easiest rules of thumb to learn from it are “less code” in general, “less trusted code,” and isolation of problematic code. The latter two are old concepts going way back to pre-Orange Book days. You might say DJB was the first to apply them to an open source project with long-term usefulness.

What of system complexity, though? How do we deal with the problems? Another link someone posted was of the opinion that all systems are in a degraded mode just waiting for a catastrophe and root cause analysis isn’t the right way to go. I think that might be true for many systems. I disagree with it as a maxim. I’m not the only one either. Here’s why: humans have made plenty of complex hardware and software products that did their job precisely without major failure for years on end.

So, why do some systems achieve these good results and others don’t? It’s more than methodology. There’s plenty of domain knowledge required for many of these systems. Others might push components or technology in unknown directions. Some are mostly static, while some change plenty. So, what does it take to make a complex system that doesn’t fail?

A Japanese writer on IEEE/ACM gave us a clue. He gave a chronological account of how the Japanese train system got from trouble to highly assured operation. One common trend was understanding the operating environment, the failure modes of components (e.g. trains, passengers), and probable system wide failures. The more we understood about problems in the domain, the more easily we could build systems that survive them. LOCK project noted this as “theories of security come from theories of insecurity.”

This all sounds like common sense but you will see all too often risky projects go forward without a thorough hazard or security analysis. The safety-critical and risk management communities have many methodical ways of doing such analyses. SQUARE is a recent one for the system/software community. Designing a survivable system means anticipating problems, general and specific, that might lead to catastrophe. Then dealing with them somehow.

I’ve repeatedly discussed how to handle requirements, design, implementation, testing, and integration. I refer people to previous discussions on this blog for those.

User interactions can make or break the security of a system. Two common ways of dealing with this: the system restricts their choices by technical means; the organization restricts choices via policy and/or provides recommended usage guidelines. Real systems are usually a combination of these two. So, which should we lean toward?

Well, a decade of IT indicates the pessimists were right: users will destroy themselves if given the option to. This might be the result of ignorance, apathy or malice. Restricting access to critical control code, recoverability, sanity checks on user input, and safeguards built into automated processes are all good ideas. It also pays to assume automated control might glitch and do something ridiculous. Countermeasures might include putting a human in the loop to apply common sense to control decisions or using a simpler system to do this. Such a countermeasure has saved at least one bank from an access control system gone wild.

Some readers might say, “Well, that’s nice for the problems code can solve, but what of the others?” That’s where good practices and policy come in. Let’s talk practices first.

Good practices will ensure the system operates smoothly. Good practices [ideally] won’t introduce errant behavior into the system. How to create these? I think these aspects of the requirements are best created by having a good group brainstorm, analyse and double-check each other. The group should be a mix of users, domain experts, system designers, and maybe a security expert. Including users increases their feeling of ownership of the system and helps them understand why they do the best practices. Justifications for these should be written down so the knowledge can be transferred later to new hires. This has an added benefit later on in that business heuristics aren’t opaquely included in the legacy system itself, making it hard to replace.

Now for policy. I’m focusing now on the issue Bruce mentioned: complicated policy. It’s obvious that complex policy is hard to follow and possibly ambiguous. Old Orange Book guidelines required formal, precise security policy for high assurance systems and proof they were embedded in requirements. The takeaway that it might help to apply system engineering principles to policy itself: make it unambiguous; check it for inconsistency when developed and changed; decompose it into smaller pieces maybe module-by-module; include regular review steps that incorporate feedback from users and system admins.

I’ll probably write something on the “nature” angle later on. Hope you enjoy this for now.