They got a D.
The rest of the U.S. government didn’t do very well. Eight of twenty-four departments (including the Department of Defense) failed. Overall, the federal government received a C- (up from a D+ last year).
supersnail • April 16, 2007 7:10 AM
The scorecard seems to measure lots of things that bureaucrats understand and admire like “standard configurations” and “standard procedures” which really have nothing to do with actual security.
I would be very concerned if the DoD had a “standard configuration” which covered the recruiting officers PC and the weapons control system on a stealth bomber — yet according to the article they would drop two grades for this.
The whole exercise reminds me of ISO 9000 “quality control” certifications which are actually measure cubic feet of paperwork.
BIG AL • April 16, 2007 7:27 AM
I got all D grades in school and today I’m in prison. It figures the government is headed in the same direction.
Cynical Ben • April 16, 2007 7:48 AM
Someone pointed out a couple years ago that these ratings are generally related to the size of the agency. F to A is generally a list of agencies from large to small. This seems to relate to the complexity and amount/severity of bureaucracy in the org. I just think of it as real-life application of Newton’s 3 Laws of Motion… 😉
Tim S. • April 16, 2007 8:43 AM
I’m actually not that disappointed that DoD got an F. They are completely segmented in their operations and their decision making. If the FISMA grades said that they had C, I’d think that someone was lying. How could an agency like that get anything higher than a D with its current infrastructure?
Roger • April 16, 2007 10:41 AM
A number of prominent critics — including several current and former US Federal CIOs — have pointed out that FISMA provides a very poor measurement of IT security anyway. In fact it’s not exactly clear what it does measure, except perhaps the ability to process paperwork quickly. For a specific example, it is claimed that selection of any access control mechanism — no matter how inappropriate — will score a passing grade in the “security controls” category provided the selection process is fully documented.
antibozo • April 16, 2007 11:56 AM
The benefit in FISMA comes from the deeply insecure organizations being forced at least to document their configurations as compared to a somewhat arbitrary baseline (NIST 800-53 and related). The accuracy of a given organization’s documentation is largely dependent on the effectiveness and cluefulness of the parent department’s inspector general’s office (or other oversight office that vets the documentation before the department authorizes the office’s system). The scores themselves don’t mean much since there’s wide variation both in interpretation of the baseline controls and the respective IGs’ knowledge, and the net security benefit may be nil (or even negative) because every organization, regardless of its current security posture, still has to invest man-years in documentation instead of maintaining or improving actual security. This exercise may clue in some of the weak organizations, thus avoiding some compromises, and at least the result is some form of comprehensive documentation. IMO, it’s not an effective use of security money, but not a complete waste either.
Alan • April 16, 2007 12:09 PM
Maybe if they hired qualified people instead of those loyal to the ruling party, they might do a bit better. I don’t expect that to change any time soon.
Pat Cahalan • April 16, 2007 1:16 PM
DHS No Longer Gets Failing Cybersecurity Grade
I don’t know, Bruce, my college called a “D” in a major track course a failure. “C” or better to pass!
I’d call “cybersecurity” a major requirement for DHS (and DoD, FBI, etc). Maybe it’s an elective for the Bureau of Land Management, though 🙂
sooth_sayer • April 16, 2007 1:40 PM
@alan
You seem to be saying that you can’t be loyal to your party and be smart .. wonder if it applies to “other party” too.
Josh Peters • April 16, 2007 2:17 PM
Bruce, is the overall grade boost for the government a result of No Child Left Behind, or is the US Government actually doing a better job at security? 😉
Dave Howe • April 16, 2007 3:42 PM
@sooth_sayer:
You can be loyal to the party and be good at your job; however, its harder to be technically competent in the fields of intelligence and security, but still loyal enough to the party not to go running off to Congress and blurt out any inconvenient facts….
Day1, and counting.... • April 16, 2007 4:28 PM
“The rest of the U.S. government didn’t do very well. … Overall, the federal government received a C- . ”
Yet let’s see how long it will be before Mr. Schneier asks us all to trust those same folks to “pass broad legislation” putatively to solve a social problem.
They’re no good at cybersecurity. So they must be really good at crafting well-thought-out legislation.
Um, what?
antibozo • April 16, 2007 5:12 PM
Day1, and counting…. [sarcastically]> They’re no good at cybersecurity. So they must be really good at crafting well-thought-out legislation.
The FISMA grade is not a measure of the Federal government’s “cybersecurity”. It is a partial measure of compliance with the documentation requirements. Systems can be very secure but poorly documented and thus earn a low grade, or poorly secured but well documented and thus earn a high one.
Furthermore, the grade is of the executive branch only. So you appear to need a remedial government course.
Day1, and counting.... • April 16, 2007 5:32 PM
@Bozo
Right. And none of the U.S. Cabinet Departments craft any law at all. Not a single regulation.
Riddle me this: what’s your guess as to how many regulations are created by non-legislative (that’s the branch where you’ll find Congress) Federal governmental bodies each year? How about the total that now exist? I’ll give you a little hint: go Google something called Administrative Law.
Shall I save you a seat in the remedial government course?
antibozo • April 16, 2007 7:25 PM
Day1, and counting….> So they must be really good at crafting well-thought-out legislation.
Day1, and counting….> … how many regulations are created by non-legislative… go Google something called Administrative Law.
I see–so, in your mind, regulation and Administrative Law are the same thing as legislation.
Better save yourself two seats.
B-Con • April 16, 2007 8:25 PM
As a student, realistically a D and an F are equivalent, because you don’t pass with either. The DHS has improved from failing to thinking about not failing.
From the report, it’s interesting to note that the DHS came in rock bottom during it’s first two years, and second to last in it’s third year.
They’re making me feel more secure already. Maybe they could get some of those dolphins you mention in your next post to come help them out with their cyber security.
NRC-scared • April 16, 2007 11:39 PM
The Nuclear Regulatory Commission’s grade is particularly troublesome. How did they go from the best secured government agency in 2003 (A-) to an F today? How did they mess up that badly?
antibozo • April 17, 2007 1:53 AM
NRC-scared> How did [NRC] go from the best secured government agency in 2003 (A-) to an F today? How did they mess up that badly?
For all we know, their security got better.
Over the past four years, the reporting requirements for FISMA grading have gone up dramatically. Where, back in 2003, an office might have been able to check a bunch of boxes claiming that they complied with all the required controls, now they have to document how they comply with each control, and also document some level of testing to assure that this documentation is accurate. This ends up being a monumental amount of work, so reaching the currently required levels of documentation is extremely difficult now where it used to be a couple of days’ work.
Meanwhile, the inspectors general became much more discriminating about what documentation and levels of testing they would accept, so they routinely reject documentation packages that even two years ago might have been acceptable to them.
The net result is that a huge amount of paperwork is being generated, reviewed by IGs, sent in, sent back, sent in again, buried in soft peat, and recycled as firelighters.
This is what has brought the grades down so precipitously. It isn’t that the level of security went down (not by much, anyway), it’s that the grading difficulty went way, way, up.
It’s important to recognize that these scores don’t reflect actual security–they reflect the security as documented, and to the extent it is documented. Until the documentation is complete, the scores will fluctuate wildly as systems once thought to be fully documented are discovered to be otherwise. And even when the exercise is complete, you won’t be able to compare one department’s grade with another’s because there is no uniform standard applied to all departments–they each have their own IGs with varying workloads and levels of expertise.
Woo • April 17, 2007 2:02 AM
Using XP or Vista should alone be sufficient to award an F, imho…
Thomas • April 17, 2007 2:59 AM
@Woo
“””Using XP or Vista should alone be sufficient to award an F, imho…”””
As far as I know Bruce uses Windows.
If you know what you’re doing, you can use whatever you want.
(I know, I know…. don’t feed the trolls…..)
There’s a bunch of misinformation here.
First of all, grades are no part of FISMA–OMB evaluates agency compliance and congress puts out grades. Each year OMB raises the standard, so that a C this year is better than a C last year.
It is true that you can have secure systems and “fail” FISMA and that sometimes agencies have to choose between doing the right thing and getting better FISMA scores. I spoke to a high level DHS security manager several years ago, who told me that DHS was willing to take hits on FISMA compliance and focus on consolidating and improving networks, rather than C&A systems that were going to be eliminated.
FISMA required NIST to develop security standards for agencies and they’ve done a terrific job–you would have to pay a consulting firm huge amounts of money to get the detailed guidance on how to secure systems that you can get for free from NIST. It is hands down the best security framework available–far more detailed than ISO 17799–or whatever they call it now. Their series of documents starts with determining how much effort should be put into securing each system–based on the information it processes and the effect of damage to it–and then gives you a starting point of security controls and detailed assessment guidance.
Reporting is part of FISMA–and it’s what people mostly complain about, but what are the alternatives? Should Congress and OMB–who are responsible for ensuring that agencies secure their systems–just take the agencies word that they’re secure? I don’t think so–they need some mechanism to verify that the work has been done.
Keep in mind that Federal agencies are huge–at least one agency that I know of has over 200,000 computers. If I asked you to manage a process that did nothing more than find most of those computers and put a little yellow sticky note on them, you’d be faced with a non-trivial task–now think about trying to secure them and keep them secure.
As one OMB official remarked: “FISMA is a tool–and a fool with a tool is still a fool.” An agency that wants to be secure will find the NIST framework–which is the technical guts of FISMA–a wonderful tool. Even people who just want to push paper around are going end up with better security–in spite of themselves–by being forced through the process. Perfect? No. Pretty good? Yes.
antibozo • April 17, 2007 10:21 PM
cm> First of all, grades are no part of FISMA–OMB evaluates agency compliance and congress puts out grades.
No offense, but I think that statement is a bit facile. The grades are based on FISMA compliance, after all. No, they aren’t really meaningful to the agencies being graded unless OMB puts those agencies’ systems on a “watch list” so that their budgets are in jeopardy. The grades are really digested for Congress and the media’s benefit. What matters to individual offices is whether department heads grant their systems authority to operate.
I also take issue with this (sorry, treating your post out of order here):
cm> Even people who just want to push paper around are going end up with better security–in spite of themselves–by being forced through the process.
That’s true of the clueless, not of the clueful. The clueful are wasting a lot of resources on documentation and testing that could be better spent on improving security at the enterprise level. A lot of agencies are blowing major portions of their security budgets this year hiring contractors to document both the strong and the weak with equal effort. It’s this generalized approach that is causing harm.
I’m on the fence as to whether things are going to get better as a whole security-wise, at least until every system has a complete C&A package and the exercise goes from generation to maintenance, at which point people will be able to get back to actually working on security. Now, if people had been doing this all along, things would be different.
cm> Reporting is part of FISMA–and it’s what people mostly complain about, but what are the alternatives? Should Congress and OMB–who are responsible for ensuring that agencies secure their systems–just take the agencies word that they’re secure? I don’t think so–they need some mechanism to verify that the work has been done.
But OMB isn’t really verifying anything. The department IGs are doing most of the so-called verification, and a lot them are not particularly proficient at the technical nuances of security. Ultimately, OMB is still taking each department’s word as to how secure it is; maybe the IG process adds some value to that word; maybe not.
cm> Perfect? No. Pretty good? Yes.
I think we could do a lot better. Yes, a lot of the NIST documentation is good, but a lot of it is verbose, vague, or outright apocryphal. For example, see if you can figure out what 800-53 control SC-10 “Network Disconnect” means in practical terms, or puzzle over the inconsistent concepts of “mobile code” referred to in 800-53/SC-18 and 800-28, just to cite two examples off the top of my head. And the cost of generating the NIST documentation is chicken feed compared with the money being spent on C&A documentation and testing. If FISMA were a program just to produce the NIST documentation, the value would be a lot more obvious.
Larry Kilgallen • April 21, 2007 5:28 PM
From the annual OMB FISMA report (elements chosen by OMB) Republican Representative Tom Davis creates a report card.
But all such reports to date do not measure full FISMA compliance. For civilian agencies, FISMA compliance requires controls from NIST S pecial Publication 800-53 be applied, possibly with additional controls as needed. To date, however, the OMB report and the Davis report card do not say anything about the results of measuring systems regarding whether they implement the 800-53 contr ols. There are just a few of those controls (IS training, contingency planning, incident reporting) that are reported by agencies and thus are available to Representative Davis.
But as far as compliance with the FISMA law of 2002 is concerned, that requ ired the Secretary of Commerce to promulgate rules, which he did on March 9, 2006 to become effective one year later. So it was only on March 9, 2007 that federal civil agencies were required to be in compliance with NIST 800-53.
The test for OMB will b e in June or July, when they issue the FY2007 FISMA reporting guidelines. Watch carefully to see whether OMB requires agencies to actually report their scores on security assessments – to date they have only been required to report the percentage of mach ines on which they conducted assessments.
(No, I would not suggest that public reports indicate exactly which controls are failed by which departments, but a percentage would be good.)c
John K (27001 Blog) • July 1, 2007 12:54 PM
“It is hands down the best security framework available–far more detailed than ISO 17799–or whatever they call it now.”
It’s still ISO 17799… but being renamed ISO 27002 in due course. No other change, just the rename.
To be fair though, detail isn’t everything. There is a bit more to the picture than that.
Subscribe to comments on this entry
Sidebar photo of Bruce Schneier by Joe MacInnis.
wiredog • April 16, 2007 7:02 AM
“C” means “average” so the question is: Is “average” good or bad?