Research into an Information Security Risk Rating

The NSF is funding research on giving organizations information-security risk ratings, similar to credit ratings for individuals:

Existing risk management techniques are based on annual audits and only provide a snapshot of a partner’s security posture. However, new vulnerabilities are discovered everyday and the industry needs a solution that enables a business to continuously monitor changing risk posture of all its partners and proactively manage assumed risks. The Phase II research objective is to build a scalable fully-automated ratings system. The research will focus on identifying and incorporating new data sources, improving the statistical properties of the ratings model, and making the ratings predictive of future behavior.

Historically, credit scoring has been a “cost and time-saving technology” that has provided tremendous value to lenders and borrowers alike by reducing costs, predicting future performance, and improving credit accessibility and affordability. Unlike credit scoring, no industry standard scoring service exists to rate business with respect to their information security risk. With Saperix’s ratings service, businesses and government will have the potential to reap the same time and cost savings that lenders do from credit scoring services. If the research is successful, Saperix’s solution would provide market incentives for improving security outcomes, which would be a significant change in how security investments are viewed by businesses.

I have no idea if this is snake oil or if it actually works, but note that this is a Phase II award. There was already a Phase I award, and the NSF must have liked the results from that.

Tags: computer security, risks

Posted on January 25, 2012 at 6:44 AM • 14 Comments

Comments

Peter Hillier • January 25, 2012 7:05 AM

Interesting research area. I’m sure the insurance companies will love it! Just imagine, your risk rating and rates going up based on the analysis of your online exposure!

Jonathan • January 25, 2012 7:18 AM

I wonder if this would extend to government agencies?

DHS: F-

NSA: We could not confirm the existence of a network.

CIA: All of our auditors died under mysterious circumstances.

Paul • January 25, 2012 7:20 AM

I think the assumption that credit ratings (individual or corporate) is a good measure is a flawed starting point. It has been show that some of these credit rating have no basis in actual risks as firms like Lehman had excellent credit ratings right up to the point they collapsed.

What will happen with this system (like all other compliance models) is that firms will tailor security to meet the checkboxes, which is not always the best or most innovative security required for that firm’s business.

AWD • January 25, 2012 8:12 AM

Something like this could have interesting implications from SOPA/PIPA-like legislation on American cloud service providers..

Bill • January 25, 2012 8:13 AM

Although the idea is good, I see two issues. First, the process of scoring will supersede the process of securing (a la FISMA in the US Government). More money and time was spent on FISMA compliance documentation and scoring than on actual security. Second, managers all have different risk thresholds, so you really could not compare apples to apples here. A financial institution or health care provider will probably have a higher risk threshold than an internet advertising organization, for example. Even within organizations, different people have different thresholds. This looks like it is attempting to apply an objective value to a subjective valuation.

Natanael L • January 25, 2012 8:21 AM

@Jonathan:

Sony: But they were anonymous!
White House: We’re busy with ruling the country, we don’t have time to secure our computers.
The administrators of The Great Firewall of China:
That US military infosec network’s admins: Insiders? What’s that?

Andrew Philips • January 25, 2012 9:38 AM

Whether or not Credit rating is a valid metric, there are third parties whose independent goal is to report accurately on narrow metrics: Did Individual pay bill(s) on time?

From whom will this process gather data? How can that data presume to be quantified and compared?

“Company X’s products are 42 securons strong.”

PrometheeFeu • January 25, 2012 9:54 AM

“Historically, credit scoring has been a “cost and time-saving technology” that has provided tremendous value to lenders and borrowers alike by reducing costs, predicting future performance, and improving credit accessibility and affordability.”

They are joking right? The only people who actually use credit ratings are regulators. Just about everyone else understands the credit rating agencies don’t have anything close to the proper incentives to perform their official function.

I’m guessing that this rating system will fall prey to the same problem as credit ratings. Companies will pay for the rating process and therefore expect the rating agency to walk them through the bare minimum they can do so they get the highest rating they want to have. Everyone will rate perfectly on all the selected criteria and those criteria will stop being a good proxy for security.

Vincent Archer • January 25, 2012 10:03 AM

A company here in France was trying to promote their “security labels” (essentially for websites).

The label was based on results of automated security scans (nessus, mostly), plus a declaration of policy from the audited, and verification of said policy. You’d get labelled ‘Secure IV +’ if you have zero major vuln “on average” and you fix any vulnerability labelled “critical” within 4 business hours of the patch being available, for example. Fail to fix in time, and you downgraded to IV, or III+.

The concept is nice, but you need a critical mass of sites to get recognition. Which most companies will fail to get, since, if the concept becomes popular, each security company will try to launch their own metrics.

NobodySpecial • January 25, 2012 10:11 AM

@Vincent – but this will be administered by the government!
“The Department of Homeland Security classifies this site as secure.”

In other news – the DHS admitted to losing the names, SSN, addresses and inside leg measurements of all it’s employees on a USB key left in a bar.

jmzf • January 25, 2012 11:35 AM

A friend pointed out this article.

As I see it, it’s an attempt to reduce the level of knowledge required to assess risk. You might point out failures within the credit rating system—I add Dun & Bradstreet commercial credit rankings. Any guesses how high Lehman, WAMU, et al were rated despite being toes-over-the-precipice?

There is a de facto standard for risk assessment of third-parties. It’s an amalgamation of SAS70 (now ISAE/SOC2), WebTrust, SysTrust, BITS, ISO270001, etc.
SAS70 was an attempt to democratize such knowledge but it happens that people don’t read. “Do you have their SAS70?” is the first question asked, not “Have you read their SAS70?”

The drive for a de jure risk standard is, I believe, naively built on the false premise that risk assessment is hard because insufficient information is available. In my opinion, Risk Assessment is hard because it requires work, expertise, and judgment. (I suppose this will receive support then fail much like the failed color-gradations for national security.)

gonzalo • January 26, 2012 9:20 AM

I think this is amazing but couldn’t find any paper.

I have being searching some metrics for a while.

I did not see metrics in ISO and neither find them in SAS70, PCI is narrow.

Does anyone suggest more articles?.

Clive Robinson • January 26, 2012 11:27 AM

Before you can make a meaningful measurment, you need reliable, tested and effective measurands. The measurands also need to have a real scale such that differing measurments can be acuratly compared. Such is the basis of Science.

If any body knows of any measurands for computer security that even remotly approach being amenable to the scientific method I for one would be interested in learning more about it and it’s underlying axioms.

Risky • January 30, 2012 2:20 PM

Quantifying risk requires taking likelyhood of its materialzing into equiation as well as the vulnerbility assessments. Insurance companies are good at that. However, they are good when it comes to car insurance an likes where statistic records are mandatory and are well documented. Wonder ever why very few companies insure against IT risks? 😉
Would you take Black Swan theory into calculating your ItSec risk posture?

Schneier on Security

Research into an Information Security Risk Rating

Comments

Leave a comment Cancel reply