U.S. Power Grid Hacked, Everyone Panic!

Yesterday I talked to at least a dozen reporters about this breathless Wall Street Journal story:

Cyberspies have penetrated the U.S. electrical grid and left behind software programs that could be used to disrupt the system, according to current and former national-security officials.

The spies came from China, Russia and other countries, these officials said, and were believed to be on a mission to navigate the U.S. electrical system and its controls. The intruders haven’t sought to damage the power grid or other key infrastructure, but officials warned they could try during a crisis or war.

“The Chinese have attempted to map our infrastructure, such as the electrical grid,” said a senior intelligence official. “So have the Russians.”

[…]

Authorities investigating the intrusions have found software tools left behind that could be used to destroy infrastructure components, the senior intelligence official said. He added, “If we go to war with them, they will try to turn them on.”

Officials said water, sewage and other infrastructure systems also were at risk.

“Over the past several years, we have seen cyberattacks against critical infrastructures abroad, and many of our own infrastructures are as vulnerable as their foreign counterparts,” Director of National Intelligence Dennis Blair recently told lawmakers. “A number of nations, including Russia and China, can disrupt elements of the U.S. information infrastructure.”

Read the whole story; there aren’t really any facts in it. I don’t know what’s going on; maybe it’s just budget season and someone is jockeying for a bigger slice.

Honestly, I am much more worried about random errors and undirected worms in the computers running our infrastructure than I am about the Chinese military. I am much more worried about criminal hackers than I am about government hackers. I wrote about the risks to our infrastructure here, and about Chinese hacking here.

And I wrote about last year’s reports of international hacking of our SCADA control systems here.

Tags: , , , , , , ,

Posted on April 9, 2009 at 12:02 PM • 55 Comments

Comments

Rich Wilson • April 9, 2009 12:10 PM

Look on the bright side. At least a dozen reporters will have gotten a dose of reality and common sense. Of course, they may just go off and find another source for their Hollywood Movie Plot.

Joe Buck • April 9, 2009 12:16 PM

Whoops. Sorry about my previous comment when the article already has the link to “Threat Level”.

mcb • April 9, 2009 12:20 PM

I was going to suggest that a nationwide power outage would primarily result in some much needed peace and quiet, but then I remembered all the deisel generators that will kick in to keep the wired wired…

Pete • April 9, 2009 12:28 PM

BTDT – 24 has already done “hacking the infrastructure” – a few seasons back with the nuke plants and the toxic gas in the natural gas system, and this season with the “CIP device” macguffin.

No Facts • April 9, 2009 12:30 PM

Again, unnamed people doing unknown things to
unnamed systems in unknown states leaving unnamed “software tools” to do indescribable things to us. Fear, fear this, scary!

I thought Dick Cheney and his sidekick are not in power any more.

Carlo Graziani • April 9, 2009 12:30 PM

“…it’s just budget season and someone is jockeying for a bigger slice.”

Got it in one. Past Yellow Peril news stories sourced inside the securocracy also tended to coincide oddly with the deliberations of congressional appropriators.

Hellfire • April 9, 2009 12:48 PM

It seems incredibly stupid that machines controlling the power production are even on the internet, not because of hackers but for risk of viruses and other security and reliability problems. I guess I had been naive in thinking the core computers that actual could shut down power production would have air-gap protection. After reading this article I tried to look if this was common practice and found this:

http://www.networkworld.com/news/2008/040908-rsa-hack-power-grid.html

A pen-test consultant was able to get into a power company’s computers via browser exploits… I think the article has to be confused/bending the truth, I really doubt the computers directly in control of generators are being used to surf the internet. I am thinking they broke into the company’s network, but into the employees desktops.

Edward Bear • April 9, 2009 12:55 PM

Excuse me. You differentiate between “criminal hackers” and “government hackers.” This strikes me as unwise. They both want to pwn your computer and not to your benefit.

To quote S. M. Stirling: “The police should have the right to search everyone’s hard drive over the network and delete any material they deem suspicious.” How would the quoted behavior would differ from botnet operators?

A nonny bunny • April 9, 2009 1:03 PM

@Edward Bear:

>
Excuse me. You differentiate between “criminal hackers” and “government hackers.” This strikes me as unwise. They both want to pwn your computer and not to your benefit.”
<<

True, but a government hacker acts on the direction of its government and so unlikely to extort you with threats of taking down critical infrastructure. Criminals are more likely to serve their own short term interests.

Edward Bear • April 9, 2009 1:11 PM

@A nonny bunny

>
Criminals are more likely to serve their own short term interests.
<<

And governments don’t “serve their short-term interests”? Even at the expense of long-term disaster?

I take it you haven’t been reading the financial news for the last few months.

David in Chicago • April 9, 2009 1:28 PM

What are you saying, Bruce? I, for one, am outraged that the Chinese can read my electric bill!

A nonny bunny • April 9, 2009 2:04 PM

Well, frack, that didn’t post as I had intended.. Apologies, dear moderator, as I try again.

@Edward Bear

And governments don’t “serve their short-term interests”?
Even at the expense of long-term disaster?

Not in the way the criminals do. Governments have different considerations to deal with; upsetting other governments by destroying their infrastructure is rarely if ever in their interest. Unlike a criminal they can’t make a big score and then take the money and hide. Well, not if they want to keep running their country; and invariably they do.

I take it you haven’t been reading the financial news for
the last few months.

No, I haven’t, and it’s completely irrelevant to this issue.

Davi Ottenheimer • April 9, 2009 2:30 PM

“Governments have different considerations to deal with; upsetting other governments by destroying their infrastructure is rarely if ever in their interest. Unlike a criminal they can’t make a big score and then take the money and hide.”

Eh, why not? That is exactly what South Africa did over a period of many years to its neighboring countries. I would argue the US has been very successful at doing it in Somalia. BTW, what are you defining as government?

The subtext to these government hacker stories is plausible denial. We have to include extra-state actors and not just explicit ones. Note how the US employs civilian “contractors” to do dirty work around the world. Those raids on Mogadishu? Not US soldiers. RDF missions out of Djibouti? A government might not take responsibility. The DDoS on Georgia originating first in the US and then Russian networks was a prime example of this kind of relationship. Moreover, you have to factor massive cooperation and coordination from nationalist/patriotic agents, which makes them more of a threat than squabbling crooks trying to cheat or compete with each other.

The big news is that the energy sector finally seems to be more comfortable with admitting breaches, although in this case the intelligence community broke the news. It used to be strictly taboo on both sides.

mweb • April 9, 2009 2:30 PM

Speaking of “random errors” being a bigger threat than hackers. Any thoughts on the British counter terrorism chief who flashed documents about pending terror arrests to the press. That scares me more than mythical “hackers”

Henning Makholm • April 9, 2009 3:41 PM

Those calling for air gaps, please consider the magnitude of the problem. It’s not just a question of controlling some machines inside a power plant; it’s the distribution network with its (at least) tens of thousands of switching stations that must be controlled and monitored. The great blackouts that make headlines are all about cascading breakdown of the distribution network — when that happens, many fully functional power plants have to shut down or spill their power output into rivers or cooling towers, because the distribution network is down and cannot get the power to consumers.

Now consider that those switching stations are spread all around the country; the cost of providing a separate physical network to connect them all must be enormous. And even then, many switching stations are in remote locations — a determined attacker could easily find a place where he could tamper with the power-network-internal connections in relative safety from discovery.

With a network that large, security architecture demands that the subnet is considered malicious, whether or not it shares routers and wires with the public internet.

Bob • April 9, 2009 3:42 PM

“At least a dozen reporters will have gotten a dose of reality and common sense. Of course, they may just go off and find another source for their Hollywood Movie Plot.”

I would guess many will go for the latter. A lot of journalists don’t let little things like facts or truth get in the way of a good sensational story.

When I was working on a Ph.D. in chemistry, our department had a small spill of a harmless chemical with a potent stench. One of the TV news crews caught one of the faculty entering the building:
“Aren’t you afraid of this deadly chemical?”
“Don’t be silly! It’s perfectly safe, it just smells bad.”
“What do you do here, sir?”
“I’m a chemistry professor.”
Rather than packing up and leaving when told the truth, the TV crew went straight off to interview a gaggle of hysterical secretaries. Of course, the secretaries made it on the broadcast and the prof didn’t.

PackagedBlue • April 9, 2009 3:52 PM

The stock market needs another cycle.

The security market needs more money.

News is the medium to control the system.

New hype fuels more power to the system.

Live free or Die Hard, a good movie for what can and won’t happen.

sooth sayer • April 9, 2009 4:34 PM

Bruce .. you should declare that your blog gets infested with terrorists and you should be given $1T to keep a tab on such people.

sooth sayer • April 9, 2009 4:40 PM

@Davi ..
Why don’t you add that US destroyed most of Europe and Japan by starting WWII to pull itself out of depression and then benefited by selling them for the next 25 years! To an alien who reads Ward Churchill for history and economics – the co-relation will be irrefutable.

Comparing South African Govt. of the past with US in Somalia .. come on .. get a grip.

Clive Robinson • April 9, 2009 6:07 PM

All in all it is an item that is a little “yawn worthy”.

Yes most of the utility networks are very brittle and easily prone to cascade break down.

And yes society is more dependent on the utility networks than at any other point in time.

And the basic cause is the free market choice of low cost utilities where investment in the infrestructure has been deamed less important than the backups and systems required to ensure the security of supply by the utility networks.

But untill very recently power outages where a regular occurance localy as branches of trees droped on overheads or transformers. People learnt to keep candels blankets and botteled water in the cupboard under the stairs etc, and in many parts of the world little has changed.

The simple fact is the utility networks have never got to the point in reliability where “mission” or “life” critical systems cannot justify the cost of a backup system.

And that is the long and the short of it. Yes a utility network outage is a pain but it’s usually not critical. The usual side effects of energy outages are an increased birth rate 40 weeks down the calander.

However the one set of utility networks we realy cannot do without for a prelonged period is those involving the movment of water. For drinking, sanitation, flood and fire prevention.

I would be more worried about people with a good knowledge of phases of the moon and weather paterns opening or closing the wrong sluice gates than crowbaring an energy network.

Davi Ottenheimer • April 9, 2009 6:15 PM

@ sooth sayer

I don’t know what you mean. You are free to believe what you would like about WWII.

I did not compare. I gave examples to illustrate a reality — governments, and their agents, destabilize and destroy infrastructure of other states.

Davi Ottenheimer • April 9, 2009 7:00 PM

“I think that China recognizes if in a very strategic sense you want to ensure you have the ability to exploit another country’s potential weakness or vulnerability but do it in a way that isn’t confrontational or cause an international crisis, then this is a very good way of doing that.”

Eric Rosenbach, executive director for research at Harvard University’s Kennedy School of Government’s Belfer Center

http://www.reuters.com/article/newsOne/idUSN0853911920090408?virtualBrandChannel=10112&pageNumber=2&sp=true

Trichinosis USA • April 9, 2009 7:12 PM

@David in Chicago:

It’s not that the Chinese can read our electric bills that’s outrageous. It’s how hard they’re laughing when they do so.

Aguirre • April 9, 2009 7:15 PM

“The recent espionage won’t reveal more than how the network is connected, and being able to map the infrastructure is not a threat without knowing how the system is operated and controlled” – Professor Gregory Reed, Pitt’s Swanson School of Engineering

via http://infosecurity.us/?p=7781

He also added that some of the information is also publicly available from the U.S. Department of Energy and the Federal Energy Regulatory Commission.

Ben in AZ • April 9, 2009 7:41 PM

Sounds like a case of Journalists acting as responsibly as William Randolph Hearst might when covering a story of a night out on the town for Annie Oakley.

BruceK • April 9, 2009 9:09 PM

We raised this concern back when I was a part of NRIC VI and VII. Would have kept raising this concern if the former FCC Chairman (in his infinite wisdom) hadn’t decided to end this valuable effort.

A nonny bunny • April 10, 2009 3:10 AM

@ Davi Ottenheimer

I did not compare. I gave examples to illustrate a
reality — governments, and their agents, destabilize
and destroy infrastructure of other states.

And why do they do that? They don’t do it to make a profit and then disappear to some tropical island. And it’s rare, as I said. Wasn’t South Africa at war with it neighbouring states at the time? That would be an important consideration to attack infrastructure. But outside of war there is much less often a reason, and it’s much less acceptable to the international community. Even Israel is getting a hard time these days for bulldozing over Palestinian infrastructure.
what benefit would Russia or China get from destabilizing US networks? If they’re found out it will just lead to sanctions and other political consequences. They can’t make a big hit and then run to avoid the consequences. We know who to hold responsible, something which can often not be said in the case of criminals.

Clive Robinson • April 10, 2009 3:14 AM

@ Trichinosis USA,

“It’s not that the Chinese can read our electric bills that’s outrageous. It’s how hard they’re laughing when they do so.”

Especialy so that they manage to laugh through those little white masks they all have to wear to stop themselves choking on the lignite (brown coal) pollution…

The price of electricity can be measured in many ways either today or tomorow, but in the end somebody has to pay one way or the other. But what the heck as long as I’m getting rich to day…

James Lick • April 10, 2009 3:18 AM

The article in my local paper (The China Post) outlined a possible scenario where these terrorists^H^H^H^H^H^H^H^H^H^Hhackers could destroy infrastructure by causing a turbine to spin out of control and explode. Where do they get these crazy ideas? Are turbines so poorly designed that you can get them to explode so easily? Are there no physical safety mechanisms to shutdown a turbine if it were to theoretically spin out of control?

sharvey • April 10, 2009 7:51 AM

On her show last night, Rachel Maddow linked this story to a story of fiber-optic cables being deliberately cut in Silicon Valley.

She’s usually quite rational (despite your opinion of her politics), and this seemed unusually hysterical.

Is this a non-story? Or should we really be afraid of Bad Guys with bolt cutters? Do we need to post guards at all cable vaults?

RSaunders • April 10, 2009 9:42 AM

As someone who has participated in Bruce’s April 1 movie plot contest every year, (See: http://www.schneier.com/blog/archives/2009/04/fourth_annual_m.html ) I an outraged to see the Wall Street Journal cheating in the contest by using an actual newspaper to launch their Movie Plot entry. Not to mention going way over the 150 word limit.

It’s shameless, and I certainly hope you don’t accept their entry.

SimpleSecurity • April 10, 2009 10:21 AM

@Henning Makholm
“Those calling for air gaps, please consider the magnitude of the problem. It’s not just a question of controlling some machines inside a power plant; it’s the distribution network…”

Fine, use the public infrastructure, but then establish VPN tunnels between all endpoints that use that public infrastructure. Basically setting up an extremely low-cost private network on the public infrastructure. Quite simple.

Davi Ottenheimer • April 10, 2009 12:44 PM

“turbine to spin out of control and explode. Where do they get these crazy ideas”

well, probably from a famous experiment conducted for the Department of Homeland Security by the Idaho National Laboratory known as Aurora and detailed extensively in 2007 by the Homeland Security Subcommittee on Emerging Threats, Cybersecurity, and Science and Technology

video of the experiment was leaked to the press. it also became an ES-ISAC (Electricity Sector Information Sharing and Analysis Center) advisory and has been publicly referenced by the federal energy regulatory commission (FERC)

http://elibrary.ferc.gov/idmws/common/OpenNat.asp?fileID=11522578

Davi Ottenheimer • April 10, 2009 12:46 PM

“should we really be afraid of Bad Guys with bolt cutters? Do we need to post guards at all cable vaults”

they opened a manhole cover, dropped down ten feet, stood in four feet of water and used a saw

Davi Ottenheimer • April 10, 2009 1:00 PM

@ A nonny bunny

“We know who to hold responsible”

unfortunately not. the non-state actors are hard to trace. they are more like supra-nationals that span nations but keep a loose affiliation.

“what benefit would Russia or China get from destabilizing US networks”

infiltration first. even if it were just research purposes, it would be significant. they might not know what to use it for now, as destabilization is only one option, but that will not stop them from exploring the capabilities and gathering intelligence

“Wasn’t South Africa at war with it neighbouring states at the time”

sort of. the destabilization of mozambique was based at some point on the SA idea that the world should view black rule as disasterous. SA also wanted to undermine the ability of neighboring states to influence SA issues. thus conflicts of ideology show how sabotage and nefarious interference are a genuine threat. it’s much more subtle than more common territorial or political disputes. i can think of many reasons why Russia and China would like to have the ability to at least map and detail, if not interfere with, american energy

bob • April 10, 2009 2:43 PM

Air Gap?? You mean we need to come up with some sort of nationwide network of electrically conductive material which we could use to link electric power generation facilities?

Oh, no I cant think of anyway to get a signal in and out of those types of places without using the internet! (stipulated that it would still need signal encryption if you used power lines as communications links otherwise you’d be like the phone company back in the days of “blue boxes” assuming no one could possibly generate DTMF.)

Clive Robinson • April 11, 2009 6:25 AM

@ bob,

“Air Gap?? You mean we need to come up with some sort of nationwide network of electrically conductive material which we could use to link electric power generation facilities?”

Sarcasim aside yes a lot of the control signaling does indead go along with the power conductors.

In the UK atleast one energy supplier has the coms system carrying the internet for third parties for comercial reasons (a few gigabits of fiber is not exactly heavy compared to HVAC distrubution lines).

However the power grid is also physicaly fragile and taking a chunk of it out due to weather etc bringing down pylons would also stop those communications. So alternative “down stream” communication methods are required for amongst other reasons safety.

Some is via PMR links others by telco lines. However dedicated point to point communications via these means is very expensive (which is why some rail companies are using Internet across satalite TV systems).

One of the areas that concerns me is the instalation of “local telemetry” nodes using UHF PMR or unlicensed spectrum. These systems are invariably not encrypted and have little or no security. The idea behind them appears to be that they can be used from power company vehicals so saving large amounts of time.

What is not clear about them is what sort of telemetery is being carried and if it could be used for control of the network switching etc.

One of the joys of fiber is it can be put down most convienient conduits using simple pneumatic or hydrolic systems. And being non conductive fiber is of considerably lower risk than conductive cables.

So it could in theory be put down existing gas or water pipes quite safely (you just need to solve the entry-exit and valve issues)…

RF • April 11, 2009 12:27 PM

Eh, I care. If governments can do it today, criminals may be able to in a few years. If there are bigger threats, we should be pay more attention to them, not less to this.

Brons aka Jim Burrows • April 13, 2009 9:10 AM

The coverage of this has been pitifully fact free, but my impression is that the connection to the Russians and the Chinese is that IP addresses that the connections came from are in those two nations.

BUT… so are huge segments of the world’s botnets. Since both nations have very large populations of pirated OS’s that are not up to patch level, they are over-represented amongst compromised machines.

Unless you actually trace the command and control all the way back to the actual individuals who perpetrated the attacks, the locations either of the zombies in the botnets or the servers that they use for first line control are totally irrelevant and tell you more about the geographic patterns of vulnerability than the origins and motivations of the attackers.

Annoying Anonymous • April 14, 2009 8:19 AM

It’s going to be an international twist of Die Hard 4.0. Except it’s going to get really dirty with all the sewers shooting on you when you’re walking down the roads.

Pete • April 14, 2009 9:10 AM

DoT and DoE are way behind the curve on protection of infrastructure. Count the “Zombies Ahead” signs next time you are driving. Just because the article didn’t have any details, doesn’t mean that details don’t exist. Certainly we shouldn’t throw money at DoE and DoT because of the WSJ article. But we better start making sure the non-DoD parts of our government start working quite a bit harder on IA.

So far it’s just malicious kids (Zombie vandals) and probing by hostile actors. One day, it won’t be. Not a matter of “if”, but “when” someone (foriegn state) takes a serious poke at our utility grids.

When it happens, we’ll have no one to blame but ourselves for ignoring the real issues that exist, including Bruce’s random errors and undirected worms.

acp • April 14, 2009 10:21 AM

Aurora video of the experiment was leaked to the press

From what I can read it was not exactly as much an “experiment” as an intentionally produced “propaganda” video:

http://www.chinadaily.com.cn/world/2007-09/27/content_6139437.htm

Still there’s high probability that the security measures in most of the networks could be made better than they are now. But it’s certainly better approaching the problem without panic.

Sarah • April 14, 2009 10:23 AM

If this article interests you and you are currently researching new ideas to respond to the mounting challenges in cyber security you should take a look at the Global Security Challenge website: http://www.globalsecuritychallenge.com.

We have just launched a new award (£9,000 GBP cash grant, mentorship and networking opportunity) for researchers and small companies developing new technologies in cyber security. The judging for this award will focus mainly on the disruptive potential of the technology and less on the idea’s maturity. The closing date is 15 May 2009.

Mike • April 14, 2009 1:24 PM

I work for a utility and we get attacked so often that our security folks are on a first name basis with the local FBI. Most of the trace-backs go back to the usual suspects.

Karthik • April 14, 2009 9:51 PM

You’ve given an overview of the vulnerabilities in the system. If your government did not listen to you, would you become the Die Hard villain to prove a point? Or to get some ‘government funded legal billions’ into your pocket? The latter sounds more realistic.

We should validate our internal systems by distributing the power of information among diverse people. In particular, nobody should have ‘everything/superuser’ privileges. The system should lock down permanently upon initial setup.

We should enable 3rd party auditing: Logs of Internet routers should be signed by at least two different countries with conflicting interests. This would show us if China is lying or the US is lying.

Unless the governments show compelling evidence with the above schemes in place, it won’t make sense to believe these agencies and/or conspiracy theorists.

john • April 14, 2009 11:39 PM

oh its OK. internet 2.0 will keep us all safe. What we need is more government control, and this story proves it! And conficker, omg. Why do we even have such an evil terrorist tool called the internet..thats it, we need some laws and further warrantless wiretapping to protect us from the boogie-man. War on the Boogie-man! My grandma is almost too scared to click on the dancing pigs. LOL how predictable. Look for the legislation.

notme • April 15, 2009 7:19 AM

you’d be better off worrying about the smart-grid cruft and the already demonstrated proof of concept worm that has been written and demonstrated for the vendors who are insisting on moving forward as-is despite the fact that the damn meter on your house is or is about to have networking capabilities a whole lot of ‘but they will never figure that out’ and some laughable crypto implementations (i.e. hardware AES that took up so many resources they actually couldnt use it.)

The recent news articles on the smart grid stuff really played down whats going on there.

Officer Bob - Cyber Cop • April 24, 2009 1:56 PM

I have yet to see any substantiated evidence that anything nefarious has occurred.
My contacts who should know about such things are quiet.
If events such as those described in the press are occurring, I would expect that alerts would be flowing from DHS US Cert and others directly to the Utilities. So far I have neither seen or heard anything.
As you say, this could simply be the result of the new budget season. Nothing like a good scare to beef up the old security budget.

Leave a comment

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.