Schneier on Security

Skeptical • March 29, 2016 1:29 PM

Almost all computer security experts have two arguments against this idea: first, adding this back door makes the system vulnerable to all attackers and doesn’t just provide surreptitious access for the “good guys,” and second, creating this sort of access greatly increases the underlying system’s complexity, exponentially increasing the possibility of getting the security wrong and introducing new vulnerabilities.

Well, a couple of things. First, I wonder whether the complexity issue is too heavy a hammer to use here (I’ll explain what I mean), and second, I wonder whether you fail to consider the full effect of a particular policy on security by failing to consider how that policy structures the incentives and behaviors of key actors.

So, first, part of the problem I have with the complexity argument – on its own – is that it proves too much. Sometimes we find ways to improve systems that also involves simplifying them – certainly this can be an important type of improvement to make to a system if complexity in itself is a problem – but frequently the improvements involve greater complexity.

Obviously complexity is a pretty damn ubiquitous problem, especially in the context of information technology.

And so we devise ways of managing that complexity – and in doing so, even though we make unforeseen mistakes along the way, we build up complex systems that nevertheless function safely and as intended.

You say that it’s hard. Sure, it’s hard. Lots of things we do to improve systems are hard. They’re not easy to figure out, and even when we think we have a solution, it may not be easy to design tests that might fully satisfy us.

But there’s an important difference between sounding a cautionary note (hey, we need to be really careful and deliberate in how we do this, we need to expect certain failures, we need to do lots of testing before implementing, etc.) or criticizing a specific proposal (e.g. this won’t work because in cold temperatures the O-rings will not expand to fill the changing volume of space they’re meant to fill), and simply taking a general stance against adding any complexity, regardless of how susceptible to testing, regardless of how limited the damage might be if certain failures occurred, regardless of the alternatives.

Hell, encryption itself adds enormous complexity to a system of communication – and I suppose if we didn’t investigate how we could design and conceptualize encryption in a manner to make it susceptible to understanding and testing, as to how we could limit the probability of mangled or lost signals to within acceptable ranges, etc., using it might be a bad idea in certain circumstances.

Indeed, on the morning of December 7, 1941, a Navy aircraft sank a submarine at 0700, approximately one mile distant from Pearl Harbor. The pilot communicated this information in code, which meant a delay (considerable, in that case) between the coded communication being received, and the information contained therein being decoded and distributed to parties who needed to know such information. And somewhat less dramatically, that tradeoff between the benefits and costs of using encryption and the type of encryption to be used is one that is obviously still a very active factor today.

Now, obviously strong encryption is a vital and a good thing in a great many circumstances. I don’t doubt that it’s quite hard to get right, and quite hard to implement. And we actually insist that some of our most precious information is strongly encrypted, despite the increased complexity and the risks that it poses. And just as encryption itself may make a system of communication more complex, so too will any design that seeks to both secure systems against certain types of access attempts while allowing access in specified circumstances that involve parties other than a particular user be more complex as well.

There are different ways of going about ameliorating the undesired costs of strong encryption without sacrificing too much of the desired benefits. One might be to render the processes involved in a lawful access mechanism diverse, obscure, and the consistent focus of testing.

In many ways, I would far prefer that the US Government and various companies work together than work against each other. I’d rather Apple cooperate with requests for assistance, and that the US Government aid Apple in securing the device against unlawful access. Had Apple provided the assistance requested, both parties would have incentive to protect the mechanism devised, and to devise means of ascertaining when others may have discovered them as well.

That type of system facilitates information sharing and cooperation for a common purpose.

Otherwise – and you want to talk about introducing complexity – you have the US Government in the position of balancing the equities in deciding whether to disclose vulnerabilities to companies while simultaneously seeking to discover such vulnerabilities in order to fulfill its duties as a government.

And if we’re going to talk about the question of security overall, then that broader, systemic effect on the incentives and behavior of the various actors involved must be considered. If we incent the most powerful and knowledgeable actors NOT to work cooperatively on security, and to seek ways to crack security (rather than seeking to add a designed-in means of authorized access), then we’ve reduced the security of our devices, all else being equal.

So from an engineering perspective, I wonder whether the objection is just too broad; and from a strategic perspective, I wonder whether you’ve thought through how the moves and incentives of other players in reaction to stances like Apple’s will affect security, regardless of the challenges that the complexity of lawful access – whether limited to narrow conditions or not – brings to the security design of a device or system.