Schneier on Security

At least it was found now; no corporation has an incentive to hide that the bug exists and how long; and bugs in open source security software happen by accident, not purpose.

From Bruce's previous article: "Second, you need to be sure that security problems are fixed promptly when found. People will find security flaws in open source security code. This is a good thing. There's no reason to believe that open source code is, at the time of its writing, more secure than proprietary code. The point of making it open source is so that many, many people look at the code for security flaws and find them. Quickly. These then have to be fixed. So a two year-old piece of open source code is likely to have far fewer security flaws than proprietary code, simply because so many of them have been found and fixed over that time. Security flaws will also be discovered in proprietary code, but at a much slower rate."

I think that's basically what you (Anonymous) were trying to say, and what I was going to say as well.

To the above poster: you sound like an open source evangelist-lunatic. That is all.

One thing that Bruce didn't mention is that the bug is already fixed in GnuPG version 1.4.2.2.

I think another advantage of Open Source is that when bugs like this are found, and publicised, it may make programmers on other projects that use similar methods think again about security of their system.

Another example of this was the XMLRPC issues that arose some time back - because this is a series of functions that are used by lots of systems, even those who didn't use the exact code needed to think about how they had implemented it.

If it had just been a closed source issue that had been found, then it would not have been fixed in so many places where it, or similar systems, had been used.

I can understand that open source projects with very few developers would have a similar number of bugs and and similar length of time to patch bugs a closed project would. But won't the increased number of developers (several hundred?) in very large projects mean that fewer bugs will make it into the build (because more people will see the patches) and when bugs crop up they will be patched more quickly?

Further, don't open source licenses provide a lower barrier to entry so that updates can come through package systems like synaptic? Closed systems (such as Windows) can't bundled application updates with system updates because of commercial licensing.

So doesn't this mean:

1. Fewer bugs
2. Faster patch times
3. Significant greater penetration of patches amongst the user base

?

I can see that these systems are still going to get bugs - Ubuntu 5.10's admin password bug was a particular stinker - but won't the general situation be better?

I may be wrong. I would welcome any comments to convince me otherwise :-)

I wonder whether anybody has already bothered to check old signed documents to see if they still verify completely with the new version...

Why should more developers = fewer bugs? It's been shown time and time again in a wide variety of circumstances that throwing more people at a problem is not a solution. Spotting and fixing bugs is about experience and knowledge; a large number of inexperienced developers looking at a problem may all miss it. As Bruce says in the article from 1999 that he refers to, there needs to be review by experts.

As a developer myself, I think it's far more likely that more bugs occur as more people work on the same code. Consider that a large proportion of work is not about bug fixing, it's about adding new features. Someone makes a change to add a new feature, and creates a bug by accident. The likelihood of this bug being spotted depends on peer review by experienced developers. That is what finds bugs before they make it into a released build.

Whether open or closed source has experienced development code review is up to the individual process. Generalisations about the two differing ideologies therefore cannot be made.

With that in mind, to the very first poster in this list: the allegation that all closed-source development introduces bugs on purpose is frankly insane. No company could possibly get away with such practices without some disgruntled employee outing them.

Addendum to my previous comment:

Code review is of course not the only way to find bugs prior to release. Thorough and complete testing is the other way of finding them.

In this regard, open source can experience more problems than closed source, in that closed source can guarantee a level of testing since they pay qualified testers. Open source relies on people actively using betas; the quantity and quality of testing therefore varies.

You should point out that GPG encryption is not compromised - just the email signing portion of the software. The many companies that rely on GPG to encrypt sensitive files are not affected by this exploit.

What is interesting for the Open Source world is this represents a critial bug in core security related software went undetected for a long time. In theory this means that a lot of people looked at the code and nobody saw it.

And soon after the inital code, additional defects were found.

This points to a statement that the OpenBSD team has made a number of times: people have to know what they are looking for. Just letting people look at the source code does not provide any reason to hope that someone who knows what they are doing will stop and take the time to do it right.

Open Source projects might want to think about finding people who are willing to take the time to learn proper methods and become core parts of the team. Some already do this, more need to.

Well, of course some old signed documents will no longer verify: those that relied upon the concatenation feature that's just been removed.

The point is not that open source is better or worse than closed source. There are very few pieces of software we can have more faith in than GPG; if GPG has such vulnerabilities, then our existing ways of building security software just don't work and we need better ones.

I wonder how such a basic bug - one fundamental to the whole purpose of signing a message in the first place - made it into a release in the first place. Don't they test it before releasing it?

It should be pointed out that this isn't so much a flaw in the signing system itself, but rather misleading presentation to the user.

You can't sign the whole email message, headers and all, because it changes in transit and the signature would always be wrong. So you have to deliniate, somehow, which part of the message is signed, and which part is not.

If the signature is inlined into the message (instead of using a multipart attachment system), then the presentation system can lead the user to believe that more of the content is signed than really is. Of course, someone can add unsigned content outside that boundary without affecting the signature, and therein lies your problem.

An ill-advised feature of all previous versions of this product made it easier to confuse the user into thinking that more data is signed than really is. This feature wasn't ever caught as a "bug" because the code was, in fact, working as intended. The "feature" in question has now been identified as an unhelpful source of confusion, and removed.

Your gpg digital signatures are still cryptographically secure, and if you have any worries about a previously-seen signature, just download the new gpg version and re-verify it.

Ah, that explains a lot. Thanks for the explanation, Tyler.

To an excelent explanation made by Tyler Larson I would like to add a little comment, just narrowing attention to the fact that: from the "Background" part it's absolutely clear, that danger to the end user arises not from buggy coding, but from buggy policy.

Well, yes and no. It is a presentation issue, and the problem does lie in the way people use the software. If you get a signed email, run it through GPG to check the signature and it says OK, what it really means is "some part of this email has a vaild signautre in it," which probably isn't what you meant to ask. The right thing to do is to give gpg the email and ask it to output the signed portion, but this operation is buggy in the broken versions of gpg. If an attacker prepended some bogus data to the email, gpg would sometimes output the bogus data before the signed data. Some versions are reportedly even worse in that they would output only the bogus data.

I was about to make a similar point, Paeniteo, I don't think one should assume that "It appears this bug has existed for years without anybody finding it." should be assumed true.

Max

"Open source does not necessarily mean 'fewer bugs.'"

Or, to Tyler's point, open source does not mean "fewer features".

Open Source "is what it is". There is nothing about putting source code up on a web site that will remove bugs. What open source code does is provide a tool that makes it easier to see how things are written and work with what has already been done.

In this case we see that the problem was not a bug in the software, but a problem with the design and interface.

Question: Was this bug discovered through analysis of the source, or by playing around with the execution?

Most people seem to assume that analysis of the source is the best way to find security flaws; however, I haven't found reports that distinguish between whether security flaws were found by direct source code analysis, or by simply running the software with fractured inputs.

As a developer, it seems simpler to me to test, say, for buffer overflows by simply listing all the places that buffer lengths are specified in user-supplied data, and tweak each of them until something "interesting" happens, rather than poring over the code to try and do the same - after all, we have a tool for poring over code to see what it does, and it's called a processor.

Moral of the story. Security is hard, with or without money

Another problem that Open Source contributes to the security field is that the moment a piece of source code becomes available, it gets copied and used in other projects. As a result, a bug can propagate far without the ability for the fix to propagate nearly as far.

[When's the last time you went back to the sources of all your copied code samples, to see if the samples had changed?]

Maybe there needs to be a dynamic compiled source code library, like a DLL, that allows new versions of source to be distributed and fix all code compiled from that source base. :-)

@Paul Crowley
"The point is not that open source is better or worse than closed source. There are very few pieces of software we can have more faith in than GPG; if GPG has such vulnerabilities, then our existing ways of building security software just don't work and we need better ones."

Interesting point. I'm not sure I would agree that "there are very few pieces of software we can have more faith in than GPG." But that does not really change your argument. As far as we know, all software has "bugs."

Nevertheless, I remain upbeat. There is a large community of academics, professionals, and hobbyists interested in security. They are actively looking for weaknesses and designing new technologies.

This flaw was found and fixed before, as far as we know, it has been exploited. I take this as evidence that the current system is working.

@Alun Jones:
"Another problem that Open Source contributes to the security field is that the moment a piece of source code becomes available, it gets copied and used in other projects."

True, but the same problem occurs with commercial software. It may not be distributed in source form, but it is copied nonetheless.

"When's the last time you went back to the sources of all your copied code samples, to see if the samples had changed?"

Any deployed system needs a process for implementing security updates, regardless of where the code comes from.

@Jeff Hotchkiss

"Why should more developers = fewer bugs? It's been shown time and time again in a wide variety of circumstances that throwing more people at a problem is not a solution."

Are you referring the @Mythical Man-Month"? That showed that you cannot increase productivity simply throwing more people at a problem. I was not referring to productivity to code quality. If a patch is posted to an open source list where more people can read it and try it out, won't the bugs be more likely to be found before being committed to the code stream?

davidg:
"[The Mythical Man-Month] showed that you cannot increase productivity simply [by] throwing more people at a problem."

Err, no, it showed that "Adding manpower to a late software project makes it later." -- Brooks' Law. In other words, adding people actually *decreases* productivity, due to communication overhead.

"If a patch is posted to an open source list where more people can read it and try it out, won't the bugs be more likely to be found before being committed to the code stream?"

Certainly, the more testing and review, the better. But it is impossible to make a blanket statement about open source vs. closed source, with regards to quality.

the critical advantage of open source isn't quality or security, but economy.

@CZ:

"some people have forgotten the 'Holy Quadfold of Software Development': description and design and static and dynamic testing."

Is "Holy Quadfold of Software Development" a phrase that I (or Google) should be familiar with?

"There has to be a formal description of the software in question but a formal proof is rarely needed and sometimes not even possible."

Isn't this somewhat misleading? The evidence indicates that, for all but the most trivial programs, a formal proof is not possible.

"Dynamic testing needs time and time only (as shown by Bishop/Pullen. A nice oversight is here: http://www.adelard.co.uk/resources/papers/pdf/... These tests are practically not distributable, at least not very well."

Again, that doesn't sound like standard terminology from where I sit. Bishop's paper also does not mention "dynamic testing." References?

It should also be pointed out that the paper concerns itself "primarily" with embedded, real-time systems. That is not surprising, since it is easier to apply formal methods to smaller, simpler systems.

"[Dynamic] tests are practically not distributable, at least not very well. That means, that closed-source is worse than open-source, because you do not have the source, so a static testing of it is practically impossible."

You've really lost me there. First of all, there is no evidence that "distributability" of tests is the limiting factor for software quality. Second, you are only looking at one narrow aspect of testing (formal methods), and using that to claim that "closed-source is worse." Third, your logic is specious: X (dynamic tests are not distributable) means Y (open source is best) because Z (static testing is practically impossible). Fourth, your statement about static testing is simply not true:

"Try [static analysis] with machinecode. It's possible in theory, but ..."

It is possible, and the technology exists. For example, Logiscan (formerly BugScan):

http://www.logiclibrary.com/resources/...

Static analysis is almost always done on the intermediate representation. You can get there from the source (by compiling) or from the machine code. In some cases -- Java, for instance -- it might be easier to start with the "machine" code.

How many years ?

> Is "Holy Quadfold of Software Development" a phrase that I (or Google) should be familiar with?

You don't like it? Feel free to come up with a better one.

> The evidence indicates that, for all but the most trivial programs, a formal proof is not possible.

Which evidence?

> Again, that doesn't sound like standard terminology from where I sit.

Where do you sit? I don't care for it normaly, but it's relevant for the terminology. It's even different in the UK and the USA and I don't even speak english natively.

> Bishop's paper also does not mention "dynamic testing." References?

I can't find out who came up with it first, but I found it at a lot of places--with some slight variations--to distinguish the tests of the code/description/design et al, the static sources, from the runtime tests, the dynamic interplay of input and algorithm.

> It should also be pointed out that the paper concerns itself "primarily" with embedded, real-time systems.

That doesn't matter, it's just the place where formal methods fits best, not the only one.

> That is not surprising, since it is easier to apply formal methods to smaller, simpler systems.

Yes, that's a good argument for smaller and simpler systems.

> You've really lost me there. First of all, there is no evidence that "distributability" of tests is the limiting factor for software quality.

I never said that and apologize for my bad english.

I measure the "many-eyes-hypothesis" as some form of "distributability" or, if you prefer: "parallelizability", a quite well known area of IT, but I think "distributability" is better because it refers to the statistical terminus technicus too.
One pair of eyes needs some time to look thru the code, many eyes need less time. These eyes are of different quality of course, distributed over some curve. If you have enough eyes, you'll approach a normal distribution. If you do not care which pair of eyes take a look at which part of the program the quality of the different parts of the code approach a normal distribution. That means that parts of the code are of better quality than others--some are ingenious, some are pure crap and you do not know which is which. So you have to regulate that mess quite strict, but it works very well if done right. Yes, that means that the tag Open-Source on its own will not lead stringently to heaven, it needs a lot of extra work, especially of the non-technical kind to success. It seems as if I concur with our generous host here.

But that all means too that static testing is distributable.

Now on to the dynamical testing, the working on the living program: the vivisection. You know the functions and bounds of the input a priori. Searching for bugs means to test the program for all of the functions with input within the bounds and outside of the bounds too. You can do the first if the input is not only finit but also very small, but you can't do the latter. Even if you limit yourself to the 256 octets and one megabibyte (byte = 8 bit here) input you have to test all (2^8)^1024 = 1.09074813e2540 combinations. Right? No, it's much more, it's the power set of these combination because you'll have to test all subsets too and that ruins distributability in theory. The limits are practical limits as one can easily show that nobody can spot out the function of a black box without limits.

> Second, you are only looking at one narrow aspect of testing (formal methods), and using that to claim that "closed-source is worse."

If I start with the hypothesis, that the best software is made with all of the four points mentioned above and show that Closed-Source does fullfill less points than Open-Source I can righteously say that Closed-Source is worse than Open-Source. And the word 'worst' was choosen very carefully, because Open-Source in its present form does not fullfill all four points too. Or did you see a lot of Open-Source projects delivered complete with formal description and the complete design? ;-)

> Third, your logic is specious: X (dynamic tests are not distributable) means Y (open source is best) because Z (static testing is practically impossible).

I don't think I said that, but I'll shove all the blame on my former english teacher and the availability of (La)TeX in "Movable Type 3.2" ;-)
BTW: logic is either wrong, true or indetermined but never "specious".

> It is possible, and the technology exists. For example, Logiscan (formerly BugScan):

It is not possible (vid. Goedel et al).
And it does not work with deliberatly obfuscated (it goes against the grain for me to call that "encryption" as some suppliers of obfuscators like to call it) machinecode and it's impossible with encrypted machinecode (needs hardware encryption&keys, tamper proof memory and all that stuff of course).

> In some cases -- Java, for instance -- it might be easier to start with the "machine" code.

OK, that's sadly true: it is sometimes easier to read the bytecode directly than some highly messed up code.

CZ

@CZ:
"[Static analysis of machine code] is not possible (vid. Goedel et al)."

Huh? Godel's Incompleteness Theorem? The Godel programming language?

Frankly, I don't see how either one of those would support your point.

Moreover, I provided you with a specific counterexample. You merely reiterated your claim -- directly contradicting me -- without addressing it.

I don't agree with a lot of the other things you said, either. But, perhaps this isn't the best forum to discuss software development methodology. (Lambda the Ultimate would be more appropriate: http://lambda-the-ultimate.org/) Nor is at an appropriate forum for English language pedantry.

'''Moral: Open source does not necessarily mean "fewer bugs." '''

'fewer' than what? A closed-source competitor that you're not mentioning?

The moral is that oss does not always mean *zero* bugs, and I guess that might be interesting to a couple of people. By some (silly) measure, though, all oss may still have fewer bugs than all closed-source software. The GPG example doesn't prove anything when it comes to numbers like that.

With closed source software you get those bugs too,and they also takes years to fix (look at the Oracle debacle or Microsoft), or you simple didn't notice it..since them keep in in secret.. ;-)