Schneier on Security

Davi Ottenheimer • February 25, 2005 12:20 PM

Bruce, panels are great but perhaps you will find more traction in open venues such as the Secure Software forum where the (large) audience can participate and help set the tone for the group the panel is meant to represent.

Thanks for the link. A very thoughtful essay. I think this quote boils the problem down nicely:
"Most designs for engines and safety features were based on the assumption that owners and operators would behave rationally, conscientiously, and capably. But operators and maintainers were poorly trained, and economic incentives existed to override the safety devices in order to get more work done. Owners and operators had little understanding of the workings of the engine and the limits of its operation."

Perhaps ss a fitting example of Microsoft's mockery of the seriousness of this point, the Microsoft Certified Systems Engineer program has always been widely regarded to be a marketing/revenue system, which has absolutely nothing to do with real training or understanding the workings of distributed systems and their security. Twelve year olds were being awarded MCSEs in the late 1990s...

Also, note that the article claims it took over 120 years, thousands of deaths, and millions in damages before regulation started to appear at the turn of the century. Watt's astute warnings in the 1870s did not create the market for safer boilers -- extensive death and destruction did.

Why are we predisposed to wait for a major disaster before we start regulating? "Predictable Surprises" by Bazerman and Watkins, 2004, have a pretty good answer. They claim the following general characteristics of predictable surprises:

- Leaders know a problem exists -- and that the problem will not solve itself

- People recognize that the problem is worsening over time

- Fixing the problem will certainly cost money, while the reward is an avoided cost that is uncertain (but likely to be large)

- The up-front costs will be significant, but the benefits will be delayed

- And last, but not least, a small but vocal minority benefits from inaction and is motivated to lobby for its private gain

So based on the lesson of Steam Engines, and the theory of predictible disasters, I figure we could have as many as ten to twenty years (with many more ChoicePoint fiascos to come) before the general public will force regulation from Int'l leadership. That is unless elected officials and public figures such as yourself start actively pushing for real regulation right now. Thank you California.

Is Counterpane ready to make bold statements like Watt did to help advance the industry? Without active pressure from experts for *safer* boilers, steam engines might have suffered a much greater public backlash that would have stopped progress altogether...we already see reports wondering about a serious slowdown in people willing to use the Internet for commerce.

Bruce, it is not clear what you see as your role in all this but I hope you accelerate your work with legislators on regulation and campaign widely for federal adoption. And the next time you run into public figures like Howard Schmidt, please remind him that he is actually "pro-regulation" when he says that we need fair and balanced laws.

Davi Ottenheimer • February 25, 2005 6:15 PM

@Mark
Interesting point. I just heard Jeff Hawkins talk about this issue. He just published a book that claims to explain why technology fails to achieve real intelligence. Today we can confidently say "computers don't make mistakes, people do", but Jeff seemed to think that computers might someday be able to correllate data more accurately and thereby make predictions...

Davi Ottenheimer • February 27, 2005 9:34 PM

@Chung

Imagine that a large consumer group finds that they are at great risk due to false promises and flawed design (resulting from sabotage or otherwise). They will demand manufacturer liability, even if they signed a waiver. If the risk is high enough, consumers will suspect fraudulent activity and demand redress. Do you want to run out and buy a car described as "unsafe at any speed"?

Just a minor nit, boilers were exploding, and not the steam engines. Steam engines benefited from demand for innovation, while the lowly boiler was ignored. Liability was put on boiler manufacturers after their foolish and inadequate designs had killed thousands of people and caused millions in property damage. But whether it was caused by sabotage or shoddy engineering is irrelevant to the people getting blown off the boat. The danger is simply a function of an overall risk equation (Risk = Asset x Vulnerability x Threat).

A consumer looks for ways to minimize risk, but all alone they rarely have the level of technical expertise or influence necessary. It also might be noted that the maturity of the law and likelyhood of litigation was probably lower at the turn of the century. In any case, if we find that a million people have their consumer records stolen due to sabotage of a database and only one thousand consumers of one million stolen identities actually experience ID theft, then the direct damage to the consumers still totals three million dollars. That is not to mention damage to merchants and providers due to fraudulent charges and other criminal activity. Yet the average individual consumer loss is still only USD$3000.

Sadly, it seems that damages probably have to be in the trillion dollar range before it registers as a real enough disaster or "Predicitible Surprise" that the market is willing to correct itself. What is three million dollars in 1900 property damage in today's terms?

Back to the angle of your statement, if a consumer is going into an area where sabotage (threat) is high, then they most definitely will look for assurances of safety or a trustworthy merchant.

In other words, would you buy padlock to protect a valuable asset and say to yourself "who cares if this lock is *vulnerable* to being opened without my permission"? Or do you say "who cares if every other person here is a professional thief (threat)"? If you care about your assets, I seriously doubt you will keep all liability to yourself. You will surely seek products, calculate your risks and hope that manufacturers accept a reasonalbe amount of limited liability (e.g. a warranty or guarantee).

Funny, I doubt you would call yourself a "zealous consumer advocate" just because you want to buy products you can trust.

Davi Ottenheimer • February 27, 2005 10:13 PM