June 2012 Archives

Friday Squid Blogging: Another Giant Squid Found

A dead 13-foot-long giant squid has been found off the coast of New South Wales.

As usual, you can also use this squid post to talk about the security stories in the news that I haven't covered.

Posted on June 29, 2012 at 4:14 PM42 Comments

On Securing Potentially Dangerous Virology Research

Abstract: The problem of securing biological research data is a difficult and complicated one. Our ability to secure data on computers is not robust enough to ensure the security of existing data sets. Lessons from cryptography illustrate that neither secrecy measures, such as deleting technical details, nor national solutions, such as export controls, will work.

---------

Science and Nature have each published papers on the H5N1 virus in humans after considerable debate about whether the research results in those papers could help terrorists create a bioweapon. This notion of "dual use" research is an important one for the community, and one that will sooner or later become critical. Perhaps these two papers are not dangerous in the wrong hands, but eventually there will be research results that are.

My background is in cryptography and computer security. I cannot comment on the potential value or harm from any particular piece of biological research, but I can discuss what works and what does not to keep research data secure. The cryptography and computer security communities have been wrestling for decades now with dual-use research: for example, whether to publish new Windows (Microsoft Corporation) vulnerabilities that can be immediately used to attack computers but whose publication helps us make the operating system more secure in the long run. From this experience, I offer five points to the virology community.

First, security based on secrecy is inherently fragile. The more secrets a system has, the less secure it is. A door lock that has a secret but unchangeable locking mechanism is less secure than a commercially purchased door lock with an easily changeable key. In cryptography, this is known as Kerckhoffs' principle: Put all your secrecy into the key and none into the cryptographic algorithm. The key is unique and easily changeable; the algorithm is system-wide and much more likely to become public. In fact, algorithms are deliberately published so that they get analyzed broadly. The lesson for dual-use virology research is that it is risky to base your security on keeping research secret. Militaries spend an enormous amount of money trying to maintain secret research laboratories, and even they do not always get security right. Once secret data become public, there is no way to go back.

Second, omitting technical details from published research is a poor security measure. We tried this in computer security with regard to vulnerabilities, announcing general information but not publishing specifics. The problem is that once the general information is announced, it is much easier for another researcher to replicate the results and generate the details. This is probably even more true in virology research than in computer security research, where the very existence of a result can provide much of the road map to that result.

Third, technical difficulty as a security measure has only short-term value. Technology only gets better; it never gets worse. To believe that some research cannot be replicated by amateurs because it requires equipment only available to state-of-the-art research institutions is short-sighted at best. What is impossible today will be a Ph.D. thesis in 20 years, and what was a Ph.D. thesis 20 years ago is a high-school science fair project today.

Fourth, securing research data in computer networks is risky at best. If you read newspapers, you know the current state of the art in computer security: Everything gets hacked. Cyber criminals steal money from banks. Cyber spies steal data from military computers. Although people talk about H5N1 research in terms of securing the research papers, that is largely a red herring; even if no papers existed, the research data would still be on a network-connected computer somewhere.

Not all computers are hacked and not all data gets stolen, but the risks are there. There are two basic types of threats in cyberspace. There are the opportunists: for example, criminals who want to break into a retail merchant's system and steal a thousand credit card numbers. Against these attackers, relative security is what matters. Because the criminals do not care whom they attack, you are safe if you are more secure than other networks. The other type of threat is a targeted attack. These are attackers who, for whatever reason, want to attack a particular network. The buzzword in Internet security for this is "advanced persistent threat." It is almost impossible to secure a network against a sufficiently skilled and tenacious adversary. All we can do is make the attacker's job harder.

This does not mean that all virology data will be stolen via computer networks, but it does mean that, once the existence of that data becomes public knowledge, you should assume that the bad guys will be able to get their hands on it.

Lastly, national measures that prohibit publication will not work in an international community, especially in the Internet age. If either Science or Nature had refused to publish the H5N1 papers, they would have been published somewhere else. Even if some countries stop funding -- or ban -- this sort of research, it will still happen in another country.

The U.S. cryptography community saw this in the 1970s and early 1980s. At that time, the National Security Agency (NSA) controlled cryptography research, which included denying funding for research, classifying results after the fact, and using export-control laws to limit what ended up in products. This was the pre-Internet world, and it worked for a while. In the 1980s they gave up on classifying research, because an international community arose. The limited ability for U.S. researchers to get funding for block-cipher cryptanalysis merely moved that research to Europe and Asia. The NSA continued to limit the spread of cryptography via export-control laws; the U.S.-centric nature of the computer industry meant that this was effective. In the 1990s they gave up on controlling software because the international online community became mainstream; this period was called "the Crypto Wars." Export-control laws did prevent Microsoft from embedding cryptography into Windows for over a decade, but it did nothing to prevent products made in other countries from filling the market gaps.

Today, there are no restrictions on cryptography, and many U.S. government standards are the result of public international competitions. Right now the National Institute of Standards and Technology is working on a new Secure Hash Algorithm standard. When it is announced next year, it will be the product of a public call for algorithms that resulted in 64 submissions from over a dozen countries and then years of international analysis. The practical effects of unrestricted research are seen in the computer security you use today: on your computer, as you browse the Internet and engage in commerce, and on your cell phone and other smart devices. Sure, the bad guys make use of this research, too, but the beneficial uses far outweigh the malicious ones.

The computer security community has also had to wrestle with these dual-use issues. In the early days of public computing, researchers who discovered vulnerabilities would quietly tell the product vendors so as to not also alert hackers. But all too often, the vendors would ignore the researchers. Because the vulnerability was not public, there was no urgency to fix it. Fixes might go into the next product release. Researchers, tired of this, started publishing the existence of vulnerabilities but not the details. Vendors, in response, tried to muzzle the researchers. They threatened them with lawsuits and belittled them in the press, calling the vulnerabilities only theoretical and not practical. The response from the researchers was predictable: They started publishing full details, and sometimes even code, demonstrating the vulnerabilities they found. This was called "full disclosure" and is the primary reason vendors now patch vulnerabilities quickly. Faced with published vulnerabilities that they could not pretend did not exist and that the hackers could use, they started building internal procedures to quickly issue patches. If you use Microsoft Windows, you know about "patch Tuesday"; the once-a-month automatic download and installation of security patches.

Once vendors started taking security patches seriously, the research community (university researchers, security consultants, and informal hackers) moved to something called "responsible disclosure." Now it is common for researchers to alert vendors before publication, giving them a month or two head start to release a security patch. But without the threat of full disclosure, responsible disclosure would not work, and vendors would go back to ignoring security vulnerabilities.

Could a similar process work for viruses? That is, could the makers work in concert with people who develop vaccines so that vaccines become available at the same time as the original results are released? Certainly this is not easy in practice, but perhaps it is a goal to work toward.

Limiting research, either through government classification or legal threats from venders, has a chilling effect. Why would professors or graduate students choose cryptography or computer security if they were going to be prevented from publishing their results? Once these sorts of research slow down, the increasing ignorance hurts us all.

On the other hand, the current vibrant fields of cryptography and computer security are a direct result of our willingness to publish methods of attack. Making and breaking systems are one and the same; you cannot learn one without the other. (Some universities even offer classes in computer virus writing.) Cryptography is better, and computers and networks are more secure, because our communities openly publish details on how to attack systems.

Virology is not computer science. A biological virus is not the same as a computer virus. A vulnerability that affects every individual copy of Windows is not as bad as a vulnerability that affects every individual person. Still, the lessons from computer security are valuable to anyone considering policies intended to encourage life-saving research in virology while at the same time prevent that research from being used to cause harm. This debate will not go away; it will only get more urgent.

This essay was originally published in Science.

EDITED TO ADD (7/14): Related article: "What Biology Can Learn from Infosec."

Posted on June 29, 2012 at 6:35 AM28 Comments

Nuclear Fears

Interesting review -- by David Roepik -- of The Rise of Nuclear Fear, by Spencer Weart:

Along with contributing to the birth of the environmental movement, Weart shows how fear of radiation began to undermine society's faith in science and modern technology. He writes "Polls showed that the number of Americans who felt 'a great deal' of confidence in science declined from more than half in 1966 to about a third in 1973. A main reason for misgivings about science, according to a poll that had studied the matter in detail was 'Unspoken fear of atomic war.'"

Even more, Weart suggests that nuclear fears have contributed to increasing mistrust not just in modern technology and the people and companies and institutions who control and regulate those technologies, but even in the societal structures that support them. He cites a widely read anti-nuclear book in the late 70s that warned that "the nuclear industry is driving us into a robotic slave society, an empire of death more evil even than Hitler's." He notes how strongly these underlying anti-establishment cultural worldviews informed a 1976 article opposing nuclear power by energy expert Amory Lovins, who wrote "reactors necessarily required high centralized power systems, which by their very nature were inflexible, hard to understand, unresponsive to ordinary people, inequitable (my emphasis), and vulnerable to disruption." Weart observes that "people with a more egalitarian ideology who thought that wealth and power should be widely distributed, were more anxious about environmental risks in general and nuclear power above all than people who believed in a more hierarchical social order." "By the mid-1970's," Weart writes, "many nuclear opponents were saying that their battle was not just against the reactor industry but against all modern hierarchies and their technologies."

Posted on June 28, 2012 at 8:50 AM14 Comments

Top Secret America on the Post-9/11 Cycle of Fear and Funding

I'm reading Top Secret America: The Rise of the New American Security State, by Dana Priest and William M. Arkin. Both work for The Washington Post. The book talks about the rise of the security-industrial complex in post 9/11 America. This short quote is from Chapter 3:

Such dread was a large part of the post-9/11 decade. A culture of fear had created a culture of spending to control it, which, in turn, had led to a belief that the government had to be able to stop every single plot before it took place, regardless of whether it involved one network of twenty terrorists or one single deranged person. This expectation propelled more spending and even more zero-defect expectations. There were tens of thousands of unsolved murders in the United States by 2010, but few newspapers ever blared this across their front pages or even tried to investigate how their police departments had to failed to solve them all over the years. But when it came to terrorism, newspaper and other media outlets amplified each mistake, which amplified the threat, which amplified the fear, which prompted more spending, and on and on and on.

It's a really good book so far. I recommend it.

EDITED TO ADD (7/13): The project's website has a lot of interesting information as well.

Posted on June 27, 2012 at 6:35 AM18 Comments

Russian Nuclear Launch Code Backup Procedure

If the safe doesn't open, use a sledgehammer:

The sledgehammer's existence first came to light in 1980, when a group of inspecting officers from the General Staff visiting Strategic Missile Forces headquarters asked General Georgy Novikov what he would do if he received a missile launch order but the safe containing the launch codes failed to open.

Novikov said he would “knock off the safe’s lock with the sledgehammer” he kept nearby, the spokesman said.

At the time the inspectors severely criticized the general's response, but the General Staff’s top official said Novikov would be acting correctly.

EDITED TO ADD (7/14): British nukes used to be protected by bike locks.

Posted on June 27, 2012 at 6:30 AM40 Comments

E-Mail Accounts More Valuable than Bank Accounts

This informal survey produced the following result: "45% of the users found their email accounts more valuable than their bank accounts."

The author believes this is evidence of some sophisticated security reasoning on the part of users:

From a security standpoint, I can’t agree more with these people. Email accounts are used most commonly to reset other websites’ account passwords, so if it gets compromised, the others will fall like dominos.

I disagree. I think something a lot simpler is going on. People believe that if their bank account is hacked, the bank will help them clean up the mess and they'll get their money back. And in most cases, they will. They know that if their e-mail is hacked, all the damage will be theirs to deal with. I think this is public opinion reflecting reality.

Posted on June 26, 2012 at 1:57 PM35 Comments

Resilience

There was a conference on resilience (highlights here, and complete videos here) earlier this year. Here's an interview with professor Sander van der Leeuw on the topic. Although he never mentions security, it's all about security.

Any system, whether it’s the financial system, the environmental system, or something else, is always subject to all kinds of pressures. If it can withstand those pressures without really changing its behavior, then it’s robust. When a system can’t withstand them anymore but can deal with them by integrating some changes so the pressures fall off and it can keep going, then it’s resilient. If it comes to the point where the only choices are to make fundamental structural changes or to cease existence, then it becomes vulnerable.

And:

I’ve worked a lot on the end of the Roman Empire. Let’s go back to sometime before the end. The Roman Empire expands all around the Mediterranean and becomes very, very big. It can do that because wherever it goes, it finds and then takes away existing treasure that has been accumulated over the centuries before. That treasure pays for the army, it pays for the administration, it pays for everything. But there’s a certain moment, beginning in the third century, when there is no more treasure to be had. The empire has already taken in all of the civilized world. At that point, to maintain its administration and military and feed its poor, it must depend basically on the annual yield of agriculture, or the actual product of solar energy. At the same time, the empire becomes less attractive because it has less to offer, because it has less extra energy. So now it has to deal with all kinds of unrest, and ultimately, the energy that it has available for its administration is no longer sufficient to maintain the empire. So between the third century and the fifth century, the empire has to make changes. That is the period when it adapts its behavior to all kinds of pressures. That is the resilience period. At the end of that period, when it is no longer able to maintain that, it quickly becomes vulnerable and falls apart.

And here's sort of a counter-argument, that resilience in national security is overrated:

But it can go wrong. Rebuilding a community that sits in a flood zone shows plenty of resilience but less wisdom. American Idol contestants who have no singing ability but compete year after year are resilient -- and delusional. Winston Churchill once joked that success is the ability to go from failure to failure without losing your enthusiasm. But there is a fine line between perseverance and stupidity. Sometimes it is better to give up and pursue a different course than continuing down the same failing path in the face of adversity.

The potential problems are particularly acute in foreign affairs, where effective resilience requires a tireless effort to adapt to changes in the threat environment. In the world of national security, bad things don’t just happen. Thinking, scheming people cause them. Allies and adversaries are constantly devising new ways to serve their own interests and gain advantage. Each player’s move generates countermoves, unintended consequences, and unforeseen ripple effects. Forging an alliance with one insurgent group alienates another. Hardening some terrorist targets leaves others more vulnerable. Supporting today’s freedom fighters could be arming tomorrow’s enemies. Effective resilience in this realm is not just bouncing back and trying again. It is bouncing back, closing the weaknesses that got you there in the first place, and trying things differently the next time. Adaptation is key. A country’s resilience hinges on being able to adapt to continuously changing threats in the world.

Honestly, this essay doesn't make much sense to me. Yes, resilience can be done badly. Yes, relying solely on reslience can be sub-optimal. But that doesn't make resilience bad, or even overrated.

EDITED TO ADD (7/14): Paper on resilience and control systems.

Posted on June 25, 2012 at 11:17 AM20 Comments

Friday Squid Blogging: Giant Mutant Squid at the Queen's Jubilee

I think this is a parody, but you can never be sure.

Millions of Britons turned out for the Queen’s four-day celebrations, undaunted by the 500-foot mutant squid that was destroying London.

Huge crowds of well-wishers lined the banks of the Thames on Sunday to watch a spectacular flotilla, continuing to cheer and wave even as tentacles thicker than tree trunks emerged from the water, seizing boats and smashing them against each other.

The Queen and Prince Philip waved and smiled, undaunted as a vast gelatinous shape hauled itself from the belly of the river, tossing tenctaclefuls of screaming bystanders into its beaked maw.

As usual, you can also use this squid post to talk about the security stories in the news that I haven't covered.

Posted on June 22, 2012 at 4:03 PM34 Comments

Colbert Report on the Orangutan Cyberthreat

Very funny video exposé of the cyberthreat posed by giving iPads to orangutans. Best part is near the end, when Richard Clarke suddenly realizes that he's being interviewed about orangutans -- and not the Chinese.

Posted on June 22, 2012 at 2:01 PM15 Comments

Economic Analysis of Bank Robberies

Yes, it's clever:

The basic problem is the average haul from a bank job: for the three-year period, it was only £20,330.50 (~$31,613). And it gets worse, as the average robbery involved 1.6 thieves. So the authors conclude, "The return on an average bank robbery is, frankly, rubbish. It is not unimaginable wealth. It is a very modest £12,706.60 per person per raid."

"Given that the average UK wage for those in full-time employment is around £26,000, it will give him a modest life-style for no more than 6 months," the authors note. If a robber keeps hitting banks at a rate sufficient to maintain that modest lifestyle, by a year and a half into their career, odds are better than not they'll have been caught. "As a profitable occupation, bank robbery leaves a lot to be desired."

Worse still, the success of a robbery was a bit like winning the lottery, as the standard deviation on the £20,330.50 was £53,510.20. That means some robbers did far better than average, but it also means that fully a third of robberies failed entirely.

(If, at this point, you're thinking that the UK is just a poor location for the bank robbery industry, think again, as the authors use FBI figures to determine that the average heist in the States only nets $4,330.00.)

There are ways to increase your chance of getting a larger haul. "Every extra member of the gang raises the expected value of the robbery proceeds by £9,033.20, on average and other things being equal," the authors note. Brandishing some sort of firearm adds another £10 300.50, "again on average and other things being equal."

We all kind of knew this -- that's why most of us aren't bank robbers. The interesting question, at least to me, is why anyone is a bank robber. Why do people do things that, by any rational economic analysis, are irrational?

The answer is that people are terrible at figuring this sort of stuff out. They're terrible at estimating the probability that any of their endeavors will succeed, and they're terrible at estimating what their reward will be if they do succeed. There is a lot of research supporting this, but the most recent -- and entertaining -- thing on the topic I've seen recently is this TED talk by Daniel Gilbert.

Note bonus discussion terrorism at the very end.

EDITED TO ADD (7/14): Bank robbery and the Dunning-Kruger effect.

Posted on June 22, 2012 at 7:20 AM47 Comments

Far-Fetched Scams Separate the Gullible from Everyone Else

Interesting conclusion by Cormac Herley, in this paper: "Why Do Nigerian Scammers Say They are From Nigeria?"

Abstract: False positives cause many promising detection technologies to be unworkable in practice. Attackers, we show, face this problem too. In deciding who to attack true positives are targets successfully attacked, while false positives are those that are attacked but yield nothing. This allows us to view the attacker’s problem as a binary classification. The most profitable strategy requires accurately distinguishing viable from non-viable users, and balancing the relative costs of true and false positives. We show that as victim density decreases the fraction of viable users than can be profitably attacked drops dramatically. For example, a 10x reduction in density can produce a 1000x reduction in the number of victims found. At very low victim densities the attacker faces a seemingly intractable Catch-22: unless he can distinguish viable from non-viable users with great accuracy the attacker cannot find enough victims to be profitable. However, only by finding large numbers of victims can he learn how to accurately distinguish the two.

Finally, this approach suggests an answer to the question in the title. Far-fetched tales of West African riches strike most as comical. Our analysis suggests that is an advantage to the attacker, not a disadvantage. Since his attack has a low density of victims the Nigerian scammer has an over-riding need to reduce false positives. By sending an email that repels all but the most gullible the scammer gets the most promising marks to self-select, and tilts the true to false positive ratio in his favor.

Posted on June 21, 2012 at 1:03 PM28 Comments

Apple Patents Data-Poisoning

It's not a new idea, but Apple Computer has received a patent on "Techniques to pollute electronic profiling":

Abstract: Techniques to pollute electronic profiling are provided. A cloned identity is created for a principal. Areas of interest are assigned to the cloned identity, where a number of the areas of interest are divergent from true interests of the principal. One or more actions are automatically processed in response to the assigned areas of interest. The actions appear to network eavesdroppers to be associated with the principal and not with the cloned identity.

Claim 1:

A device-implemented method, comprising: cloning, by a device, an identity for a principal to form a cloned identity; configuring, by the device, areas of interest to be associated with the cloned identity, the areas of interest are divergent from true areas of interest for a true identity for the principal; and automatically processing actions associated with the areas of interest for the cloned identity over a network to pollute information gathered by eavesdroppers performing dataveillance on the principal and refraining from processing the actions when the principal is detected as being logged onto the network and also refraining from processing the actions when the principal is unlikely to be logged onto the network.

EDITED TO ADD (7/12): Similar technology and concept has already been developed by Breadcrumbs Solutions, and will be out as a free beta software in a few months.

Posted on June 21, 2012 at 5:51 AM24 Comments

Rand Paul Takes on the TSA

Rand Paul has introduced legislation to rein in the TSA. There are two bills:

One bill would require that the mostly federalized program be turned over to private screeners and allow airports ­ with Department of Homeland Security approval ­ to select companies to handle the work.

This seems to be a result of a fundamental misunderstanding of the economic incentives involved here, combined with magical thinking that a market solution solves all. In airport screening, the passenger isn't the customer. (Technically he is, but only indirectly.) The airline isn't even the customer. The customer is the U.S. government, which is in the grip of an irrational fear of terrorism.

It doesn't matter if an airport screener receives a paycheck signed by the Department of the Treasury or Private Airport Screening Services, Inc. As long as a terrorized government -- one that needs to be seen by voters as "tough on terror" and wants to stop every terrorist attack, regardless of the cost, and is willing to sacrifice all for the illusion of security -- gets to set the security standards, we're going to get TSA-style security.

We can put the airlines, either directly or via airport fees, in charge of security, but that has problems in the other direction. Airlines don't really care about terrorism; it's rare, the costs to the airline are relatively small (remember that the government bailed the industry out after 9/11), and the rest of the costs are externalities and are borne by other people. So if airlines are in charge, we're likely to get less security than makes sense.

It makes sense for a government to be in charge of airport security -- either directly or by setting standards for contractors to follow, I don't care -- but we'll only get sensible security when the government starts behaving sensibly.

The second bill would permit travelers to opt out of pat-downs and be rescreened, allow them to call a lawyer when detained, increase the role of dogs in explosive detection, let passengers "appropriately object to mistreatment," allow children 12 years old and younger to avoid "unnecessary pat-downs" and require the distribution of the new rights at airports.

That legislation also would let airports decide to privatize if wanted and expand TSA’s PreCheck program for trusted travelers.

This is a mixed bag. Airports can already privatize security -- SFO has done so already -- and TSA's PreCheck is being expanded. Opting out of pat downs and being rescreened only makes sense if the pat down request was the result of an anomaly in the screening process; my guess is that rescreening will just produce the same anomaly and still require a pat down. The right to call a lawyer when detained is a good one, although in reality we passengers just want to make our flights; that's why we let ourselves be subjected to this sort of treatment at airports. And the phrase "unnecessary pat-downs" all comes down to what is considered necessary. If a 12-year-old goes through a full-body scanner and a gun-shaped image shows up on the screen, is the subsequent pat down necessary? What if it's a long and thin image? What if he goes through a metal detector and it beeps? And who gets to decide what's necessary? If it's the TSA, nothing will change.

And dogs: a great idea, but a logistical nightmare. Dogs require space to eat, sleep, run, poop, and so on. They just don't fit into your typical airport setup.

The problem isn't government-run airport security, full-body scanners, the screening of children and the elderly, or even a paucity of dogs. The problem is that we were so terrorized that we demanded our government keep us safe at all costs. The problem is that our government was so terrorized after 9/11 that it gave an enormous amount of power to our security organizations. The problem is that the security-industrial complex has gotten large and powerful -- and good at advancing its agenda -- and that we've scared our public officials into being so scared that they don't notice when security goes too far.

I too want to rein in the TSA, but the only way to do that is to change the TSA's mission. And the only way to do that is to change the government that gives the TSA its mission. We need to refuse to be terrorized, and we need to elect non-terrorized legislators.

But that's a long way off. In the near term, I'd like to see legislation that forces the TSA, the DHS, and anyone working in counterterrorism, to justify their systems, procedures, and expenditures with cost-benefit analyses.

This is me on that issue:

An even more meaningful response to any of these issues would be to perform a cost-benefit analysis. These sorts of analyses are standard, even with regard to rare risks, but the TSA (and, in fact, the whole Department of Homeland Security) has never conducted them on any of its programmes or technologies. It's incredible but true: he TSA does not analyse whether the security measures it deploys are worth deploying. In 2010, the National Academies of Science wrote a pretty damning report on this topic.

Filling in where the TSA and the DHS have left a void, academics have performed some cost-benefit analyses on specific airline-security measures. The results are pretty much what you would expect: the security benefits of most post-9/11 security changes do not justify the costs.

More on security cost-benefit analyses here and here. It's not going to magically dismantle the security-industrial complex, eliminate the culture of fear, or imbue our elected officials with common sense -- but it's a start.

EDITED TO ADD (7/13): A rebuttal to my essay. It's too insulting to respond directly to, but there are points worth debating.

Posted on June 20, 2012 at 1:19 PM50 Comments

Switzerland National Defense

Interesting blog post about this book about Switzerland's national defense.

To make a long story short, McPhee describes two things: how Switzerland requires military service from every able-bodied male Swiss citizen -- a model later emulated and expanded by Israel -- and how the Swiss military has, in effect, wired the entire country to blow in the event of foreign invasion. To keep enemy armies out, bridges will be dynamited and, whenever possible, deliberately collapsed onto other roads and bridges below; hills have been weaponized to be activated as valley-sweeping artificial landslides; mountain tunnels will be sealed from within to act as nuclear-proof air raid shelters; and much more.

[...]

To interrupt the utility of bridges, tunnels, highways, railroads, Switzerland has established three thousand points of demolition. That is the number officially printed. It has been suggested to me that to approximate a true figure a reader ought to multiply by two. Where a highway bridge crosses a railroad, a segment of the bridge is programmed to drop on the railroad. Primacord fuses are built into the bridge. Hidden artillery is in place on either side, set to prevent the enemy from clearing or repairing the damage.

Further:

Near the German border of Switzerland, every railroad and highway tunnel has been prepared to pinch shut explosively. Nearby mountains have been made so porous that whole divisions can fit inside them. There are weapons and soldiers under barns. There are cannons inside pretty houses. Where Swiss highways happen to run on narrow ground between the edges of lakes and to the bottoms of cliffs, man-made rockslides are ready to slide.

[...]

McPhee points to small moments of "fake stonework, concealing the artillery behind it," that dot Switzerland's Alpine geology, little doors that will pop open to reveal internal cannons and blast the country's roads to smithereens. Later, passing under a mountain bridge, McPhee notices "small steel doors in one pier" hinting that the bridge "was ready to blow. It had been superceded, however, by an even higher bridge, which leaped through the sky above -- a part of the new road to Simplon. In an extreme emergency, the midspan of the new bridge would no doubt drop on the old one."

The book is on my Kindle.

Posted on June 20, 2012 at 7:27 AM56 Comments

Attack Against Point-of-Sale Terminal

Clever attack:

When you pay a restaurant bill at your table using a point-of-sale machine, are you sure it's legit? In the past three months, Toronto and Peel police have discovered many that aren't.

In what is the latest financial fraud, crooks are using distraction techniques to replace merchants' machines with their own, police say. At the end of the day, they create another distraction to pull the switch again.

Using information inputted by customers, including PIN data, the criminals are reproducing credit cards at an alarming rate.

Presumably these hacked point-of-sale terminals look and function normally, and additionally save a copy of the credit card information.

Note that this attack works despite any customer-focused security, like chip-and-pin systems.

Posted on June 19, 2012 at 1:02 PM40 Comments

The Failure of Anti-Virus Companies to Catch Military Malware

Mikko Hypponen of F-Secure attempts to explain why anti-virus companies didn't catch Stuxnet, DuQu, and Flame:

When we went digging through our archive for related samples of malware, we were surprised to find that we already had samples of Flame, dating back to 2010 and 2011, that we were unaware we possessed. They had come through automated reporting mechanisms, but had never been flagged by the system as something we should examine closely. Researchers at other antivirus firms have found evidence that they received samples of the malware even earlier than this, indicating that the malware was older than 2010.

What this means is that all of us had missed detecting this malware for two years, or more. That’s a spectacular failure for our company, and for the antivirus industry in general.

It wasn’t the first time this has happened, either. Stuxnet went undetected for more than a year after it was unleashed in the wild, and was only discovered after an antivirus firm in Belarus was called in to look at machines in Iran that were having problems. When researchers dug back through their archives for anything similar to Stuxnet, they found that a zero-day exploit that was used in Stuxnet had been used before with another piece of malware, but had never been noticed at the time. A related malware called DuQu also went undetected by antivirus firms for over a year.

Stuxnet, Duqu and Flame are not normal, everyday malware, of course. All three of them were most likely developed by a Western intelligence agency as part of covert operations that weren’t meant to be discovered.

His conclusion is simply that the attackers -- in this case, military intelligence agencies -- are simply better than commercial-grade anti-virus programs.

The truth is, consumer-grade antivirus products can’t protect against targeted malware created by well-resourced nation-states with bulging budgets. They can protect you against run-of-the-mill malware: banking trojans, keystroke loggers and e-mail worms. But targeted attacks like these go to great lengths to avoid antivirus products on purpose. And the zero-day exploits used in these attacks are unknown to antivirus companies by definition. As far as we can tell, before releasing their malicious codes to attack victims, the attackers tested them against all of the relevant antivirus products on the market to make sure that the malware wouldn’t be detected. They have unlimited time to perfect their attacks. It’s not a fair war between the attackers and the defenders when the attackers have access to our weapons.

We really should have been able to do better. But we didn’t. We were out of our league, in our own game.

I don't buy this. It isn't just the military that tests its malware against commercial defense products; criminals do it, too. Virus and worm writers do it. Spam writers do it. This is the never-ending arms race between attacker and defender, and it's been going on for decades. Probably the people who wrote Flame had a larger budget than a large-scale criminal organization, but their evasive techniques weren't magically better. Note that F-Secure and others had samples of Flame; they just didn't do anything about them.

I think the difference has more to do with the ways in which these military malware programs spread. That is, slowly and stealthily. It was never a priority to understand -- and then write signatures to detect -- the Flame samples because they were never considered a problem. Maybe they were classified as a one-off. Or as an anomaly. I don't know, but it seems clear that conventional non-military malware writers who want to evade detection should adopt the propagation techniques of Flame, Stuxnet, and DuQu.

EDITED TO ADD (6/23): F-Secure responded. Unfortunately, it's not a very substantive response. It's a pity; I think there's an interesting discussion to be had about why the anti-virus companies all missed Flame for so long.

Posted on June 19, 2012 at 7:11 AM59 Comments

Britain's Prince Philip on Security

On banning guns:

"If a cricketer, for instance, suddenly decided to go into a school and batter a lot of people to death with a cricket bat,which he could do very easily, I mean, are you going to ban cricket bats?" In a Radio 4 interview shortly after the Dunblane shootings in 1996. He said to the interviewer off-air afterwards: "That will really set the cat among the pigeons, won't it?"

Posted on June 18, 2012 at 12:38 PM125 Comments

Honor System Farm Stands

Many roadside farm stands in the U.S. are unstaffed. They work on the honor system: take what you want, and pay what you owe.

And today at his farm stand, Cochran says, just as at the donut shop years ago, most customers leave more money than they owe.

That doesn't surprise social psychologist Michael Cunningham of the University of Louisville who has used "trust games" to investigate what spurs good and bad behavior for the last 25 years. For many people, Cunningham says, trust seems to be at least as strong a motivator as guilt. He thinks he knows why.

"When you sell me something I want and trust me to pay you even when you're not looking, you've made my life good in two ways," Cunningham tells The Salt. "I get something delicious, and I also get a good feeling about myself. Both of those things make me feel good about the world­ that I'm in a good place. And I also see you as a contributor to that good ­ as somebody I want to reward. It's a win win."

I like systems that leverage personal moral codes for security. But I'll bet that the pay boxes are bolted to the tables. It's one thing for someone to take produce without paying. It's quite another for him to take the entire day's receipts.

Posted on June 18, 2012 at 6:40 AM45 Comments

Friday Squid Blogging: Woman's Mouth Inseminated by Cooked Squid

This story is so freaky I'm not even sure I want to post it. But if I don't, you'll all send me the links.

As usual, you can also use this squid post to talk about the security stories in the news that I haven't covered.

Posted on June 15, 2012 at 4:02 PM31 Comments

FireDogLake Book Salon for Liars and Outliers

On Sunday, I will be participating in a public discussion about my new book on the FireDogLake website. James Fallows will be the moderator, and I will be answering questions from all comers -- you do have to register an ID, though -- from 5:00–7:00 PM EDT.

Stop by and join the discussion.

Posted on June 15, 2012 at 2:55 PM1 Comments

Rare Rational Comment on al Qaeda's Capabilities

From "CNN national security analyst" Peter Bergen:

Few Americans harbor irrational fears about being killed by a lightning bolt. Abu Yahya al-Libi's death on Monday should remind them that fear of al Qaeda in its present state is even more irrational.

Will anyone listen?

Posted on June 15, 2012 at 6:51 AM29 Comments

Cheating in Online Classes

Interesting article:

In the case of that student, the professor in the course had tried to prevent cheating by using a testing system that pulled questions at random from a bank of possibilities. The online tests could be taken anywhere and were open-book, but students had only a short window each week in which to take them, which was not long enough for most people to look up the answers on the fly. As the students proceeded, they were told whether each answer was right or wrong.

Mr. Smith figured out that the actual number of possible questions in the test bank was pretty small. If he and his friends got together to take the test jointly, they could paste the questions they saw into the shared Google Doc, along with the right or wrong answers. The schemers would go through the test quickly, one at a time, logging their work as they went. The first student often did poorly, since he had never seen the material before, though he would search an online version of the textbook on Google Books for relevant keywords to make informed guesses. The next student did significantly better, thanks to the cheat sheet, and subsequent test-takers upped their scores even further. They took turns going first. Students in the course were allowed to take each test twice, with the two results averaged into a final score.

"So the grades are bouncing back and forth, but we're all guaranteed an A in the end," Mr. Smith told me. "We're playing the system, and we're playing the system pretty well."

Posted on June 14, 2012 at 12:27 PM39 Comments

Cyberwar Treaties

We're in the early years of a cyberwar arms race. It's expensive, it's destabilizing, and it threatens the very fabric of the Internet we use every day. Cyberwar treaties, as imperfect as they might be, are the only way to contain the threat.

If you read the press and listen to government leaders, we're already in the middle of a cyberwar. By any normal definition of the word "war," this is ridiculous. But the definition of cyberwar has been expanded to include government-sponsored espionage, potential terrorist attacks in cyberspace, large-scale criminal fraud, and even hacker kids attacking government networks and critical infrastructure. This definition is being pushed both by the military and by government contractors, who are gaining power and making money on cyberwar fear.

The danger is that military problems beg for military solutions. We're starting to see a power grab in cyberspace by the world's militaries: large-scale monitoring of networks, military control of Internet standards, even military takeover of cyberspace. Last year's debate over an "Internet kill switch" is an example of this; it's the sort of measure that might be deployed in wartime but makes no sense in peacetime. At the same time, countries are engaging in offensive actions in cyberspace, with tools like Stuxnet and Flame.

Arms races stem from ignorance and fear: ignorance of the other side's capabilities, and fear that their capabilities are greater than yours. Once cyberweapons exist, there will be an impetus to use them. Both Stuxnet and Flame damaged networks other than their intended targets. Any military-inserted back doors in Internet systems make us more vulnerable to criminals and hackers. And it is only a matter of time before something big happens, perhaps by the rash actions of a low-level military officer, perhaps by a non-state actor, perhaps by accident. And if the target nation retaliates, we could find ourselves in a real cyberwar.

The cyberwar arms race is destabilizing.

International cooperation and treaties are the only way to reverse this. Banning cyberweapons entirely is a good goal, but almost certainly unachievable. More likely are treaties that stipulate a no-first-use policy, outlaw unaimed or broadly targeted weapons, and mandate weapons that self-destruct at the end of hostilities. Treaties that restrict tactics and limit stockpiles could be a next step. We could prohibit cyberattacks against civilian infrastructure; international banking, for example, could be declared off-limits.

Yes, enforcement will be difficult. Remember how easy it was to hide a chemical weapons facility? Hiding a cyberweapons facility will be even easier. But we've learned a lot from our Cold War experience in negotiating nuclear, chemical, and biological treaties. The very act of negotiating limits the arms race and paves the way to peace. And even if they're breached, the world is safer because the treaties exist.

There's a common belief within the U.S. military that cyberweapons treaties are not in our best interest: that we currently have a military advantage in cyberspace that we should not squander. That's not true. We might have an offensive advantage­although that's debatable­but we certainly don't have a defensive advantage. More importantly, as a heavily networked country, we are inherently vulnerable in cyberspace.

Cyberspace threats are real. Military threats might get the publicity, but the criminal threats are both more dangerous and more damaging. Militarizing cyberspace will do more harm than good. The value of a free and open Internet is enormous.

Stop cyberwar fear mongering. Ratchet down cyberspace saber rattling. Start negotiations on limiting the militarization of cyberspace and increasing international police cooperation. This won't magically make us safe, but it will make us safer.

This essay first appeared on the U.S. News and World Report website, as part of a series of essays on the question: "Should there be an international treaty on cyberwarfare?"

Posted on June 14, 2012 at 6:40 AM39 Comments

Teaching the Security Mindset

In 2008, I wrote about the security mindset and how difficult it is to teach. Two professors teaching a cyberwarfare class gave an exam where they expected their students to cheat:

Our variation of the Kobayashi Maru utilized a deliberately unfair exam -- write the first 100 digits of pi (3.14159...) from memory and took place in the pilot offering of a governmental cyber warfare course. The topic of the test itself was somewhat arbitrary; we only sought a scenario that would be too challenging to meet through traditional studying. By design, students were given little advance warning for the exam. Insurrection immediately followed. Why were we giving them such an unfair exam? What conceivable purpose would it serve? Now that we had their attention, we informed the class that we had no expectation that they would actually memorize the digits of pi, we expected them to cheat. How they chose to cheat was entirely up to the student. Collaborative cheating was also encouraged, but importantly, students would fail the exam if caught.

Excerpt:

Students took diverse approaches to cheating, and of the 20 students in the course, none were caught. One student used his Mandarin Chinese skills to hide the answers. Another built a small PowerPoint presentation consisting of three slides (all black slide, digits of pi slide, all black slide). The idea being that the student could flip to the answer when the proctor wasn’t looking and easily flip forwards or backward to a blank screen to hide the answer. Several students chose to hide answers on a slip of paper under the keyboards on their desks. One student hand wrote the answers on a blank sheet of paper (in advance) and simply turned it in, exploiting the fact that we didn’t pass out a formal exam sheet. Another just memorized the first ten digits of pi and randomly filled in the rest, assuming the instructors would be too lazy to check every digit. His assumption was correct.

Read the whole paper. This is the conclusion:

Teach yourself and your students to cheat. We’ve always been taught to color inside the lines, stick to the rules, and never, ever, cheat. In seeking cyber security, we must drop that mindset. It is difficult to defeat a creative and determined adversary who must find only a single flaw among myriad defensive measures to be successful. We must not tie our hands, and our intellects, at the same time. If we truly wish to create the best possible information security professionals, being able to think like an adversary is an essential skill. Cheating exercises provide long term remembrance, teach students how to effectively evaluate a system, and motivate them to think imaginatively. Cheating will challenge students’ assumptions about security and the trust models they envision. Some will find the process uncomfortable. That is OK and by design. For it is only by learning the thought processes of our adversaries that we can hope to unleash the creative thinking needed to build the best secure systems, become effective at red teaming and penetration testing, defend against attacks, and conduct ethical hacking activities.

Here's a Boing Boing post, including a video of a presentation about the exercise.

Posted on June 13, 2012 at 12:08 PM59 Comments

High-Quality Fake IDs from China

USA Today article:

Most troubling to authorities is the sophistication of the forgeries: Digital holograms are replicated, PVC plastic identical to that found in credit cards is used, and ink appearing only under ultraviolet light is stamped onto the cards.

Each of those manufacturing methods helps the IDs defeat security measures aimed at identifying forged documents.

The overseas forgers are bold enough to sell their wares on websites, USA TODAY research finds. Anyone with an Internet connection and $75 to $200 can order their personalized ID card online from such companies as ID Chief. Buyers pick the state, address, name and send in a scanned photo and signature to complete their profile.

ID Chief, whose website is based in China, responds personally to each buyer with a money-order request.

[...]

According to Huff of the Virginia agency, it has always been easy for the untrained eye to be fooled by fake IDs. The difference is, Huff said, that the new generation of forged IDs is "good enough to fool the trained eye."

The only real solution here is to move the security model from the document to the database. With online verification, the document matters much less, because it is nothing more than a pointer into a database. Think about credit cards.

Posted on June 13, 2012 at 6:45 AM57 Comments

Israel Demanding Passwords at the Border

There have been a bunch of stories about employers demanding passwords to social networking sites, like Facebook, from prospective employees, and several states have passed laws prohibiting this practice.

This is the first story I've seen of a country doing this at its borders. The country is Israel, and they're asking for passwords to e-mail accounts.

Posted on June 12, 2012 at 5:09 AM69 Comments

Changing Surveillance Techniques for Changed Communications Technologies

New paper by Peter P. Swire -- "From Real-Time Intercepts to Stored Records: Why Encryption Drives the Government to Seek Access to the Cloud":

Abstract: This paper explains how changing technology, especially the rising adoption of encryption, is shifting law enforcement and national security lawful access to far greater emphasis on stored records, notably records stored in the cloud. The major and growing reliance on surveillance access to stored records results from the following changes:

(1) Encryption. Adoption of strong encryption is becoming much more common for data and voice communications, via virtual private networks, encrypted webmail, SSL web sessions, and encrypted Voice over IP voice communications.

(2) Declining effectiveness of traditional wiretaps. Traditional wiretap techniques at the ISP or local telephone network increasingly encounter these encrypted communications, blocking the effectiveness of the traditional techniques.

(3) New importance of the cloud. Government access to communications thus increasingly relies on a new and limited set of methods, notably featuring access to stored records in the cloud.

(4) The "haves" and "have-nots." The first three changes create a new division between the "haves" and "have-nots" when it comes to government access to communications. The "have-nots" become increasingly dependent, for access to communications, on cooperation from the "have" jurisdictions.

Part 1 of the paper describes the changing technology of wiretaps and government access. Part 2 documents the growing adoption of strong encryption in a wide and growing range of settings of interest to government agencies. Part 3 explains how these technological trends create a major shift from real-time intercepts to stored records, especially in the cloud.

Posted on June 11, 2012 at 6:36 AM15 Comments

Friday Squid Blogging: Baby Opalescent Squid

Baby squid larvae are transparent after they hatch, so you can see the chromataphores (color control mechanisms) developing after a few days.

As usual, you can also use this squid post to talk about the security stories in the news that I haven't covered.

Posted on June 8, 2012 at 4:28 PM94 Comments

The Catastrophic Consequences of 9/11

This is an interesting essay -- it claims to be the first in a series -- that looks at the rise of "homeland security" as a catastrophic consequence of the 9/11 terrorist attacks:

In this usage catastrophic is not a pejorative, it is a description of an atypically radical shift in perception and behavior from one condition to another very different condition.

Hypothesis: The velocity of a catastrophic shift is correlated with two factors: 1) preexisting systemic resilience and 2) the intentionality of post-catastrophe response. The more resilience and intentionality depend on control mechanisms, the greater velocity of change. The more resilience and intentionality are predisposed to creative adaptation, the velocity of change is reduced.

More coming.

Posted on June 8, 2012 at 6:43 AM25 Comments

Homeland Security as Security Theater Metaphor

Look at the last sentence in this article on hotel cleanliness:

"I relate this to homeland security. We are not any safer, but many people believe that we are," he said.

It's interesting to see the waste-of-money meme used so cavalierly.

Posted on June 7, 2012 at 6:15 AM33 Comments

Ghostery

Ghostery is a Firefox plug-in that tracks who is tracking your browsing habits in cyberspace. Here's a TED talk by Gary Kovacs, the CEO of Mozilla Corp., on it.

I use AdBlock Plus, and dump my cookies whenever I close Firefox. Should I switch to Ghostery? What do other people do for web privacy?

Posted on June 6, 2012 at 9:36 AM138 Comments

Security and Human Behavior (SHB 2012)

I'm at the Fifth Interdisciplinary Workshop on Security and Human Behavior, SHB 2012. Google is hosting this year, at its offices in lower Manhattan.

SHB is an invitational gathering of psychologists, computer security researchers, behavioral economists, sociologists, law professors, business school professors, political scientists, anthropologists, philosophers, and others -- all of whom are studying the human side of security -- organized by Alessandro Acquisti, Ross Anderson, and me. It's not just an interdisciplinary event; most of the people here are individually interdisciplinary.

This is the best and most intellectually stimulating conference I attend all year. I told that to one of the participants yesterday, and he said something like: "Of course it is. You've specifically invited everyone you want to listen to." Which is basically correct. The workshop is organized into panels of 6-7 people. Each panelist gets ten minutes to talk about what he or she is working on, and then we spend the rest of the hour and a half in discussion.

Here is the list of participants. The list contains links to readings from each of them -- definitely a good place to browse for more information on this topic. Ross Anderson, who has far more discipline than I, is liveblogging this event. Go to the comments of that blog post to see summaries of the individual sessions.

Here are links to my posts on the first, second, third, and fourth SHB workshops. Follow those links to find summaries, papers, and audio recordings of the workshops.

Posted on June 5, 2012 at 1:16 PM6 Comments

Interesting Article on Libyan Internet Intelligence Gathering

This is worth reading, for the insights it provides on how a country goes about monitoring its citizens in the information age: a combination of targeted attacks and wholesale surveillance.

I'll just quote one bit, this list of Western companies that helped:

Amesys, with its Eagle system, was just one of Libya's partners in repression. A South African firm called VASTech had set up a sophisticated monitoring center in Tripoli that snooped on all inbound and outbound international phone calls, gathering and storing 30 million to 40 million minutes of mobile and landline conversations each month. ZTE Corporation, a Chinese firm whose gear powered much of Libya's cell phone infrastructure, is believed to have set up a parallel Internet monitoring system for External Security: Photos from the basement of a makeshift surveillance site, obtained from Human Rights Watch, show components of its ZXMT system, comparable to Eagle. American firms likely bear some blame, as well. On February 15, just prior to the revolution, regime officials reportedly met in Barcelona with officials from Narus, a Boeing subsidiary, to discuss Internet-filtering software. And the Human Rights Watch photos also clearly show a manual for a satellite phone monitoring system sold by a subsidiary of L-3 Communications, a defense conglomerate based in New York.

Posted on June 5, 2012 at 6:07 AM20 Comments

The Unreliability of Eyewitness Testimony

Interesting article:

The reliability of witness testimony is a vastly complex subject, but legal scholars and forensic psychologists say it's possible to extract the truth from contradictory accounts and evolving memories. According to Barbara Tversky, professor emerita of psychology at Stanford University, the bottom line is this: "All other things equal, earlier recountings are more likely to be accurate than later ones. The longer the delay, the more likely that subsequent information will get confused with the target memory."

[...]

Memory is a reconstructive process, says Richard Wise, a forensic psychologist at the University of North Dakota. "When an eyewitness recalls a crime, he or she must reconstruct his or her memory of the crime." This, he says, is an unconscious process. To reconstruct a memory, the eyewitness draws upon several sources of information, only one being his or her actual recollection.

"To fill in gaps in memory, the eyewitness relies upon his or her expectation, attitudes, prejudices, bias, and prior knowledge. Furthermore, information supplied to an eyewitness after a crime (i.e., post-event information) by the police, prosecutor, other eyewitnesses, media, etc., can alter an eyewitness's memory of the crime," Wise said in an email.

That external input is what makes eyewitness testimony so unreliable. Eyewitnesses are generally unaware that their memory has been altered by post-event information, and feel convinced they're recalling only the incident itself. "Once an eyewitness's memory of the crime has been altered by post-event information, it is difficult or impossible to restore the eyewitness's original memory of the crime," Wise told Life's Little Mysteries.

Posted on June 4, 2012 at 6:36 AM33 Comments

Flame

Flame seems to be another military-grade cyber-weapon, this one optimized for espionage. The worm is at least two years old, and is mainly confined to computers in the Middle East. (It does not replicate and spread automatically, which is certainly so that its controllers can target it better and evade detection longer.) And its espionage capabilities are pretty impressive. We'll know more in the coming days and weeks as different groups start analyzing it and publishing their results.

EDITED TO ADD (6/11): Flame's use of spoofed Microsoft security certificates. Flame's use of a yet unknown MD5 chosen-prefix collision attack.

Microsoft has a detailed blog post on the attack. The attackers managed to to get a valid codesigning certificate using a signer which only accepts restricted client certificates.

EDITED TO ADD (6/12): MITM attack in the worm. There's a connection to Stuxnet. A self-destruct command was apparently sent.

Posted on June 4, 2012 at 6:21 AM33 Comments

Friday Squid Blogging: Mimicking Squid Camouflage

Interesting:

Cephalopods - squid, cuttlefish and octopuses - change colour by using tiny muscles in their skins to stretch out small sacs of black colouration.

These sacs are located in the animal's skin cells, and when a cell is ready to change colour, the brain sends a signal to the muscles and they contract.

This makes the sacs expand and creates the optical effect which makes the animal look like it is changing colour.

[...]

To mimic these natural mechanisms, the team used "smart" electro-active polymeric materials, connected to an electric circuit.

When a voltage was applied, the materials contracted; they returned to their original shape when they were short-circuited.

"These artificial muscles can replicate the [natural] muscular action… and can have strong visual effects," said Dr Rossiter.

"These materials, and this approach, is ideal for making smart colour-changing skins or soft devices in which fluid is pumped from one place to another.

As usual, you can also use this squid post to talk about the security stories in the news that I haven't covered.

Posted on June 1, 2012 at 4:40 PM75 Comments

The Vulnerabilities Market and the Future of Security

Recently, there have been several articles about the new market in zero-day exploits: new and unpatched computer vulnerabilities. It's not just software companies, who sometimes pay bounties to researchers who alert them of security vulnerabilities so they can fix them. And it's not only criminal organizations, who pay for vulnerabilities they can exploit. Now there are governments, and companies who sell to governments, who buy vulnerabilities with the intent of keeping them secret so they can exploit them.

This market is larger than most people realize, and it's becoming even larger. Forbes recently published a price list for zero-day exploits, along with the story of a hacker who received $250K from "a U.S. government contractor" (At first I didn't believe the story or the price list, but I have been convinced that they both are true.) Forbes published a profile of a company called Vupen, whose business is selling zero-day exploits. Other companies doing this range from startups like Netragard and Endgame to large defense contractors like Northrop Grumman, General Dynamics, and Raytheon.

This is very different than in 2007, when researcher Charlie Miller wrote about his attempts to sell zero-day exploits; and a 2010 survey implied that there wasn't much money in selling zero days. The market has matured substantially in the past few years.

This new market perturbs the economics of finding security vulnerabilities. And it does so to the detriment of us all.

I've long argued that the process of finding vulnerabilities in software systems increases overall security. This is because the economics of vulnerability hunting favored disclosure. As long as the principal gain from finding a vulnerability was notoriety, publicly disclosing vulnerabilities was the only obvious path. In fact, it took years for our industry to move from a norm of full-disclosure -- announcing the vulnerability publicly and damn the consequences -- to something called "responsible disclosure": giving the software vendor a head start in fixing the vulnerability. Changing economics is what made the change stick: instead of just hacker notoriety, a successful vulnerability finder could land some lucrative consulting gigs, and being a responsible security researcher helped. But regardless of the motivations, a disclosed vulnerability is one that -- at least in most cases -- is patched. And a patched vulnerability makes us all more secure.

This is why the new market for vulnerabilities is so dangerous; it results in vulnerabilities remaining secret and unpatched. That it's even more lucrative than the public vulnerabilities market means that more hackers will choose this path. And unlike the previous reward of notoriety and consulting gigs, it gives software programmers within a company the incentive to deliberately create vulnerabilities in the products they're working on -- and then secretly sell them to some government agency.

No commercial vendors perform the level of code review that would be necessary to detect, and prove mal-intent for, this kind of sabotage.

Even more importantly, the new market for security vulnerabilities results in a variety of government agencies around the world that have a strong interest in those vulnerabilities remaining unpatched. These range from law-enforcement agencies (like the FBI and the German police who are trying to build targeted Internet surveillance tools, to intelligence agencies like the NSA who are trying to build mass Internet surveillance tools, to military organizations who are trying to build cyber-weapons.

All of these agencies have long had to wrestle with the choice of whether to use newly discovered vulnerabilities to protect or to attack. Inside the NSA, this was traditionally known as the "equities issue," and the debate was between the COMSEC (communications security) side of the NSA and the SIGINT (signals intelligence) side. If they found a flaw in a popular cryptographic algorithm, they could either use that knowledge to fix the algorithm and make everyone's communications more secure, or they could exploit the flaw to eavesdrop on others -- while at the same time allowing even the people they wanted to protect to remain vulnerable. This debate raged through the decades inside the NSA. From what I've heard, by 2000, the COMSEC side had largely won, but things flipped completely around after 9/11.

The whole point of disclosing security vulnerabilities is to put pressure on vendors to release more secure software. It's not just that they patch the vulnerabilities that are made public -- the fear of bad press makes them implement more secure software development processes. It's another economic process; the cost of designing software securely in the first place is less than the cost of the bad press after a vulnerability is announced plus the cost of writing and deploying the patch. I'd be the first to admit that this isn't perfect -- there's a lot of very poorly written software still out there -- but it's the best incentive we have.

We've always expected the NSA, and those like them, to keep the vulnerabilities they discover secret. We have been counting on the public community to find and publicize vulnerabilities, forcing vendors to fix them. With the rise of these new pressures to keep zero-day exploits secret, and to sell them for exploitation, there will be even less incentive on software vendors to ensure the security of their products.

As the incentive for hackers to keep their vulnerabilities secret grows, the incentive for vendors to build secure software shrinks. As a recent EFF essay put it, this is "security for the 1%." And it makes the rest of us less safe.

This essay previously appeared on Forbes.com.

Edited to add (6/6): Brazillian Portuguese translation here.

EDITED TO ADD (6/12): This presentation makes similar points as my essay.

Posted on June 1, 2012 at 6:48 AM40 Comments

Photo of Bruce Schneier by Per Ervland.

Schneier on Security is a personal website. Opinions expressed are not necessarily those of Co3 Systems, Inc..