Schneier on Security
A blog covering security and security technology.
« IEDs in Iraq |
| Latest Terrorist False Alarm: Chili Peppers »
October 3, 2007
Unisys Blamed for DHS Data Breaches
This story has been percolating around for a few days. Basically, Unisys was hired by the U.S. Department of Homeland Security to manage and monitor the department's network security. After data breaches were discovered, DHS blamed Unisys -- and I figured that everyone would be in serious CYA mode and that we'd never know what really happened. But it seems that there was a cover-up at Unisys, and that's a big deal:
As part of the contract, Unisys, based in Blue Bell, Pa., was to install network-intrusion detection devices on the unclassified computer systems for the TSA and DHS headquarters and monitor the networks. But according to evidence gathered by the House Homeland Security Committee, Unisys's failure to properly install and monitor the devices meant that DHS was not aware for at least three months of cyber-intrusions that began in June 2006. Through October of that year, Thompson said, 150 DHS computers -- including one in the Office of Procurement Operations, which handles contract data -- were compromised by hackers, who sent an unknown quantity of information to a Chinese-language Web site that appeared to host hacking tools.
The contractor also allegedly falsely certified that the network had been protected to cover up its lax oversight, according to the committee.
What interests me the most (as someone with a company that does network security management and monitoring) is that there might be some liability here:
"For the hundreds of millions of dollars that have been spent on building this system within Homeland, we should demand accountability by the contractor," [Congressman] Thompson said in an interview. "If, in fact, fraud can be proven, those individuals guilty of it should be prosecuted."
And, as an aside, we see how useless certifications can be:
She said that Unisys has provided DHS "with government-certified and accredited security programs and systems, which were in place throughout 2006 and remain so today."
Posted on October 3, 2007 at 6:50 AM
• 29 Comments
To receive these entries once a month by e-mail, sign up for the Crypto-Gram Newsletter.
Unisys or any technical consulting/integration company (I work for IBM) screwing up a gig and then the engagement team covering it up--is nothing like unusual. Discovering and preventing it is easy, if you want to. What could make this interesting is how far up the Unisys ladder the cover-up decision goes.
1. Unisys should've done an independent technical review, post-engagement. But that's expensive.
2. DHS is distracting from their own incompetence and failure to review and test the system--or at least demand that Unisys both do so and provide proof of having done so (live test in production)--before signing off on completion. Their management of the project carries a certain load of the guilt here.
I summon the vast power of CERTIFICATION!
.... Well, this is embarrassing; that's all I remember from the classes.
I am finally glad to see this getting some attention. I work for a large company in information security and we had Unisys folks hanging open wireless access points off our internal network. Its vendors like Unisys that preach “value add��? to executives but where the rubber meets there is little if any value.
She said that Unisys has provided DHS "with government-certified and accredited security programs and systems, which were in place throughout 2006 and remain so today."
Accredation usually is a process thing. Somebody signed off. Additionally an independent-ish group of people generated paper itemizing the problems they detected during their process to audit the process.
This is too non-specific to know what kind of accredation they are talking about, but this could be as simply as questionares to ask many people at different levels of the organization to establish that they are watching videos on password security, and things like that on regular intervals, and they know who to report incidents, too.
This kind of accredation has nothing to do with competance of course, just repeatability (at best) .... at worst it bores employees to tears, and good useful people leave after they can no longer do their job.
What bugs me is the apparent general acceptance of the proposition that the IDS systems would have detected the intrusions. As a major user of IDS systems I can tell you they miss plenty, and the noise level is tremendous. It remains to be demonstrated that even if those systems had been installed anyone would have noticed the intrusions.
I got a bronze life saving certificate from 1962 - I must already be qualified as a security consulatant at the DHS.
That's not a good defensive argument at all. You can't argue the effectiveness of a tool (or lack thereof) without having actually used the tool in the environment in question.
Perhaps if Unisys had actually done what they were hired to do we might be reading a different article -- like IDS Didn't Detect Intruders at DHS.
D> That's not a good defensive argument at all. You can't argue the effectiveness of a tool (or lack thereof) without having actually used the tool in the environment in question.
First off, have you ever used a NIDS?
Second, they *did* install the devices, just not all of them, and some of them not "properly".
Third, I didn't make my statement as a defensive argument. I stated that I'm concerned that there is tacit acceptance in the reporting of this story that "properly" installed NIDSes would have detected the intrusions. That's not all; there's no discussion of how detected intrusions should have been handled. Read the article with full cognizance of the biased, martketroid nature of the various commentators, and after thinking about what can go wrong in an incident response capability with externally managed and monitored NIDS. Take special note of this statement: "In July 2006, a Unisys employee detected a possible intrusion but 'downplayed it and low-level DHS security managers ignored it,' the committee aide said." See any blame shifting going on?
I don't know what really happened, but I do know that a NIDS is no magic bullet, and that it's only a fraction of the toolset that a real incident detection and response capability needs. Blaming an incident that went undetected for three months on a NIDS installation failure sounds pretty absurd to me. Where was the firewall and audit log monitoring on the compromised systems? Where was the host-based intrusion detection? Why did "low-level managers" ignore an incident report? Was it a valid report, or was it one of hundreds of thousands of mostly noise reports that all went ignored?
This is interesting. I am an infosec professional in the UK. The problem revealed here is no news to me as it is commonplace for most (at least 85%-95%) in the outsourced security service providers. Especially where monitoring is involved.
I used to work in data centres as a security engineer where we host government servers and other private sector business servers. I have seen the problem being discussed here in at least 3 big data centres in the UK. They charge the customers for security, the customers come around for a site visit, they show them a wall full of certificates and accreditation and flashing lights, the customers signs the dotted line, end of story. The infrastructure is either not there or not configured properly or worse still they don't have an incidence response team.The incident management policy and procedure is only for BS7799 auditors. I once worked for a secuity manager who believes in numbers, this managers preferred location for network IDS is outside the firewall right on the internet, (even though he does not have a way of knowing what's gone past his firewall defences) his justification is that he can report potential attacks (90%-95% of which is internet noise and random scanners) to his management.
Whenever any customer queries what they are getting or not getting for security monitoring they are simply told "sorry but you have to pay us more to get that" so it goes round and round.
I believe organisations who outsource should do more to manage their contracts and agreements with third parties, they should carry out spot checks on their infrastructure, challeging the providers claims.
I think Unisys should be taken to court and the managers of DHS should be asked why they did not do their jobs properly.
I think Dilbert must work for Unisys. Do these people ever get anything dome right?
Doug, Dilbert is too smart to work for Unisys, and if he did, his position would have been outsourced to India during the great 'Boombayization' trend Unisys has been going through during the last 3 years. It appears Unisys hired some minimum-wage workers to install this software; someone did not install it, and improperly signed official statements indicating that he did. During my years at Unisys, I have known people to lose their positions for less!
I think the key takeaway here is that Unisys got paid 1.75 BILLION dollars to install SNORT and *failed*.
back in my post 9/11 unemployed days I applied for one of these unisys jobs. securing and hardening the DHS computer systems in teh midwest. They wanted people with BS degrees and offered a whooping $8 an hour...I am not supriesed that this happened
Think about it as a business' risk vs. reward. Who wouldn't be willing to falsely and repetitively "validate" their own or someone ele's compliance with PCI standards, or "certify" a site as secure according to NIST [insert color here] book standards, when you stand to make $1.7 billion in the process? When they know their security is weak they therefore know that the chances of them discovering a breach are low, and even if they do they can pull the cheap shit VISA and Mastercard pulled with Cardsystems and claim that "well they were compliant last we checked, but when this shit happened they sort of weren't we guess."
There is no real accountability here, people. Zero. Try as they might, payment card businesses like VISA, Mastercard and Amex can't really make businesses secure your private data, they can only punish them after the fact in the rare event that a breach is actually discovered. Government regulatory agencies are even worse, GLBA, FFIEC, HIPAA, PCAOB, and AICPA are all steps in the right direction as far as their content goes, but follow a model that is known to be faulty as well as an enforcement structure that has no teeth.
If you use a credit card or a debit card, the risk is all yours, so act like it. Vet your online shopping resources before you use them. Examine your payment card activity daily. We change our passwords regularly to limit their usefulness in the event of inadvertent disclosure, don't we? Do what I do and call your bank every few months and tell them you believe your CC info may have been stolen and you want it changed. They'll do it in a heartbeat.
In 2002 a large bank I was contracting at spent millions of Euros to have HP define and implement a "Secure Unix" solution. When they were done I hacked it left, right and center and wrote an e-mail about it to some folks at the bank. The next thing that happened was that the director of HP in the local country called my company to complain about me and threatening to put us out of business. You see, in their view, their problem was not that they had delivered a sucky solution, but that somebody told the customer about this.
It's good to see that these time honoured practices are still being followed by the industry.
@antibozo - no you've got it wrong.
the proximate issue is that IDS systems that were not installed despite being contracted and paid for had no chance of detecting anything, and improperly installed systems had impaired chances of detecting what they were intended (bought and paid for) to detect.
the big issue is that a major vendor apparently ripped off the American taxpayers.
It's the economy, stupid.
Giving the customer all that they paid for isn't the way to maximize share-holder value. I suspect that UNISYS and IBM are not very far apart on this one.
Consider that a LOT of companies want to cut staffing more and more in order to maximize share-holder value... but, after getting pushed out of IBM back in May, I don't think any of the accountants get the push.
So hearing that UNISYS wasn't delivering on all of their contracted services isn't surprising because CUSTOMER SERVICE DOES NOT ADD TO SHAREHOLDER VALUE THIS QUARTER. I learned _that_ lesson at IBM.
Bottom line is...If DHS had management with half a clue, they would have realized how bad Unisys was screwing up. You can outsource things but you still need to monitor the guys doing the work for you. One good audit would have revealed a lot.
places like unisys just like to suck people's money and place as many warm bodies as they can on any given contract. the security industry knows this, it's a shame the general public and other industries have no clue.
that 1+bn dollars would have been much better spent at any number of smaller boutique security shops that actually care, and love what they do for a living.
guvn'r> the big issue is that a major vendor apparently ripped off the American taxpayers.
No, that's a side issue, and it's something that happens all the time. It's one thing to say the contractor defrauded the government--that's actually something that can be fairly easily assessed. It's something else entirely to assert that had they done everything they said they would, the compromise would have been detected any sooner. It's possible, but it's certainly not a foregone conclusion. And from there to holding them accountable for the compromise is completely absurd. The people responsible for the compromise are the ones who set up a vulnerable service exposed to the Internet. Then there's that network admin whose password was cracked, the people that implemented a password system where the digests were exposed to the attacker, the people who didn't review audit logs on the compromised systems, etc. IDS is no substitute for actual system security.
> I think the key takeaway here is that Unisys got paid 1.75 BILLION
> dollars to install SNORT and *failed*.
I think it's a bit more complicated than that, but that was pretty funny.
> the big issue is that a major vendor apparently ripped off the
> American taxpayers.
There's actually a couple of big issues here. The first is that a major vendor didn't deliver everything in their spec, and then covered it up. The first half of that isn't terribly surprising, but the second half is pretty damning.
For other issue I'll agree with antibozo: we don't really know what the holisitic security policy was supposed to be, here, but it obviously stunk. Failing to install part of a security system should be, as antibozo points out, caught out by the system itself. If nothing else, a proper audit of the system should have caught this (technical audit, not financial one, although that would have done the job as well), and audit processes have to be built into any decent security system.
I'm certainly not absolving Unisys here, but whoever signed off on this project on the DHS side either wasn't doing his/her job, or they didn't assign someone to properly oversee the whole process end-to-end. Oh,wait,that's redundant, that would be part of their job, too.
"government-certified and accredited security programs and systems"
What is meant by this is not a reference to personal certifications (e.g., MCSE, CISSP) but the certification of the IT systems themselves, using one of the four systems for Certification and Accreditation of Federal IT systems. As DHS is a civilian federal agency they are most likely using FISMA (Federal Information Security Management Act). However, as they are involved in national security, they might use NIACAP (National Information Assurance Certification and Accreditation Process). DIACAP (DoD Information Assurance Certification and Accreditation Process) and DCID 6/3 (Director of Central Intelligence Directive) are unlikely to be used at DHS.
Ideally these C&A's result in an improved security environment and a reduction in total number of risks.
Unfortunately the security results are only as good as the efforts of those who implement the process. The Federal government is no different than other organizations in that regard. Without strong IT security leadership within each implementing organization the C&A process becomes a CYA paperchase.
FISMA in particular offers an opportunity to create great security. NIST , who is tasked with developing guidelines for implementing FISMA, has created a useful, comprehensive framework for establishing security. The framework addresses all the different aspects of IT security and risk management, from whether you are running IDS to whether you put your servers under a rusty water pipe. It's flexible to the point of being amorphous, which can be a strength or a weakness depending on the intent of the implementor. It can also sometimes be over broad to the extent that you can interpret requirements right out of the guidelines. So having management that understands and is committed to the implementation of IT security is a precondition to FISMA's success.
It sounds like this lack of leadership on IT security has come back to haunt both Unisys and DHS. The tools to mitigate this risk are there, nobody bothered to use them properly.
Unisys is performing an intrinsically governmental function on behalf of the public trust, and so its CIO governmental "office" should have been confirmed by Congress, as per,
U.S. Constitution, Article II, Section 2, Clause 2 (known as the Appointments Clause)
"He shall have power, by and with the advice and consent of the Senate, to make treaties, provided two thirds of the Senators present concur; and he shall nominate, and by and with the advice and consent of the Senate, shall appoint ambassadors, other public ministers and consuls, judges of the Supreme Court, and all other officers of the United States, whose appointments are not herein otherwise provided for, and which shall be established by law: but the Congress may by law vest the appointment of such inferior officers, as they think proper, in the President alone, in the courts of law, or in the heads of departments."
We are seeing the same disaster with private military contractors in Iraq. By law, Congress can pull the plug on all these contractors, because they have not been confirmed by Congress.
Our diplomatic mission and tactical security in Iraq has been compromised by the actions of private military contractors, acting on behalf of the U.S.
And here in the U.S., our Constitutional rights to privacy are being trampled by private security contractors, again, acting on behalf of the U.S.
None of this should happen, because Congress never confirmed the appointment of these contractors to their high offices.
hspd12jpl.org> Unisys is performing an intrinsically governmental function on behalf of the public trust, and so its CIO governmental "office" should have been confirmed by Congress
Uh, how do you get that? They're not acting as officers established by law. They're supplementing the general government workforce. Should Congress approve every GS-9 employee working for every department?
>government workforce. Should Congress
>approve every GS-9 employee working
>for every department?
No, but maybe they should approve Unisys as a governmental CIO.
Working for a Big 5 consulting firm previously, I saw many incidents or haphazard security controls in place. In some cases my job was to keep them in line and it was very difficult. With cost cutting on contracts and contract vendors hiring subs it is virtually impossible without strict monitoring.
I recently wrote a paper about some unconventional methods that could help increase awareness and protect your organization from such issues. It would have to be modified a little but the same concept should apply for watching your IDS vendor.
hspd12jpl> No, but maybe they should approve Unisys as a governmental CIO.
Again, why do you think that? They aren't acting as the CIO; they provide advice and support like other lower-level employees. And CIOs don't even need Congressional approval; only department Secretaries do.
If you really think a contracting company needs Congressional approval, please explain where you would apply Congressional approval throughout the government workforce as well as contract staff, and why you think a contractor is acting in the role of someone who requires approval while an ordinary government worker is not.
"I truly believe that the type of work we do is inherently governmental," said Deborah Apperson, a senior investigator in the investigations office. "There should be strict controls and strict access. It's not something that should have the profit motive behind it."
If the appointments cannot be done in practice, then the functionality should not be outsourced, period. This is a strictly governmental function.
You can see the parallels with private military contractors in Iraq. Most of those contractors were former military and intelligence, and in most cases, they performed their work efficiently and effectively.
Nevertheless, the mistakes that were made represent the U.S. on behalf of all U.S. citizens. These few tragic incidents where the U.S. abrogated its fundamental wartime responsibilities to the profit sector are totally unacceptable. This is against the values for which America stands for.
If we cannot as a people dispatch soldiers from our populace, who will represent us, if we cannot conduct a war within the laws and treaties that we have enacted as a people, with the funding that we raise for conventional troops though our taxes, then we have no right to be in this war.
Similarly, if it is too inconvenient to conduct security investigations using federal agency employees, then we have no right to contract these important responsibilities to the private companies. The economy of scale that private contractors offer is belying the facts that much of those security clearances are unnecessary, and the outsourcing of those duties infringes upon the right of the people to be free of those unconstitutional intrusions on privacy.
hspd12.org> If we cannot as a people dispatch soldiers from our populace, who will represent us, if we cannot conduct a war within the laws and treaties that we have enacted as a people, with the funding that we raise for conventional troops though our taxes, then we have no right to be in this war.
So, by contorted analogy, you're saying that no government function should be performed by a contractor--that the government must perform every function it requires, from budget reconciliation to web development to food preparation and trash disposal, using solely the federal workforce?
If not, there must be some boundary at which you think it's okay for an agency to hire a contractor. Where would you draw that line?
And having drawn that line, are you asserting that every Federal employee, no matter how lowly, requires Congressional review and approval? If not, where would you draw *that* line?
If you want to argue with people about the ethics of the Blackwater situation, you're in the wrong topic. And holding out one extreme of purported contractual misconduct does not indict all other uses of contract staff in the Federal government. There are a lot of good reasons for the government to use contractors, not least of which is the fact that it is extremely difficult for Federal managers to get rid of underperforming Federal employees, and, believe me, there are quite a few of those.
And, by the way, there's nothing wrong with profit motive as long as it's combined with a competitive field.
Schneier.com is a personal website. Opinions expressed are not necessarily those of BT.