Ranking National Cyber Power

Harvard Kennedy School’s Belfer Center published the “National Cyber Power Index 2020: Methodology and Analytical Considerations.” The rankings: 1. US, 2. China, 3. UK, 4. Russia, 5. Netherlands, 6. France, 7. Germany, 8. Canada, 9. Japan, 10. Australia, 11. Israel. More countries are in the document.

We could—and should—argue about the criteria and the methodology, but it’s good that someone is starting this conversation.

Executive Summary: The Belfer National Cyber Power Index (NCPI) measures 30 countries’ cyber capabilities in the context of seven national objectives, using 32 intent indicators and 27 capability indicators with evidence collected from publicly available data.

In contrast to existing cyber related indices, we believe there is no single measure of cyber power. Cyber Power is made up of multiple components and should be considered in the context of a country’s national objectives. We take an all-of-country approach to measuring cyber power. By considering “all-of-country” we include all aspects under the control of a government where possible. Within the NCPI we measure government strategies, capabilities for defense and offense, resource allocation, the private sector, workforce, and innovation. Our assessment is both a measurement of proven power and potential, where the final score assumes that the government of that country can wield these capabilities effectively.

The NCPI has identified seven national objectives that countries pursue using cyber means. The seven objectives are:

  1. Surveilling and Monitoring Domestic Groups;
  2. Strengthening and Enhancing National Cyber Defenses;
  3. Controlling and Manipulating the Information Environment;
  4. Foreign Intelligence Collection for National Security;
  5. Commercial Gain or Enhancing Domestic Industry Growth;
  6. Destroying or Disabling an Adversary’s Infrastructure and Capabilities; and,
  7. Defining International Cyber Norms and Technical Standards.

In contrast to the broadly held view that cyber power means destroying or disabling an adversary’s infrastructure (commonly referred to as offensive cyber operations), offense is only one of these seven objectives countries pursue using cyber means.

Tags: , ,

Posted on September 11, 2020 at 6:15 AM • 26 Comments

Comments

Clive Robinson • September 11, 2020 7:23 AM

@ Ismar,

You must be kidding- no Israel in this list ?

That might well be one of the reasons @Bruce is politely call the authors out with,

“We could — and should — argue about the criteria and the methodology”.

Clive Robinson • September 11, 2020 7:55 AM

@ ALL,

This report appears to be trying to do two things,

1, Net a cloud.
2, Rank clouds by visable fluffyness.

With their expressed intent of trying to measure “cyber-power”.

The reason is “cyber-power” is both ephemeral and ineffable in part due to the “smoke and mirrors of the Great Game” it is a very small part of.

Trying to use “public source” information actually gets you little. As in R&D you can only measure what is chosen to be made public. Thus whilst success is often touted, failure seldom is, and the requirments for secrecy mute even what is said about success, to the little that has to be said, which is little or nothing even to oversight in the Intelligence game.

A cursury glance at the list suggests that available budget, positional membership in an international intelligence community and a desire to be oppressive to national citizens through legelastive means would produce the same list members and ranking.

But you have to think carefully about “Cyber Power” because of the “army of one” problem and plain old chance.

As physical beings in a physical world our thinking tends to be constraind by assumptions that whilst they do pertain to the tangible physical universe, do not apply to the intangible information world we are developing into.

That is time, distance, energy/matter, forces and locality have a set of physical world constraints that do not apply in the same way in an informational world.

For instance to steal a physical object requires you to be local to it when you steal it. Likewise you can only vandalise one building at a time. With both requiring energy and matter resources that have purchase, transportation and deployment constraints even with “force multipliers”, you still need to multiply these physical resources to increase the size of your activities in the physical world. The same is not true for the information world.

Thus traditional measures of “power” do not realy apply to “cyber” activities, be they offensive or defensive.

Winter • September 11, 2020 8:28 AM

Indeed, no Israel. However, the criteria cover a lot of aspects.

I myself was surprised that the Netherlands were on position 5. One reason is that the Amsterdam Internet Exchange (AMS-IX) is one of the biggest internet hubs in the world and of considerable importance for the economy. That motivated the Dutch Government to invest heavily in cybersecurity.

We are not known for offensive capabilities. However, the Dutch police have done some serious coups attacking and infiltrating criminal networks, even in the dark web. And our TLA infiltrated in the network of Cozy Bear.
https://www.volkskrant.nl/wetenschap/dutch-agencies-provide-crucial-intel-about-russia-s-interference-in-us-elections~b4f8111b/?referrer=https%3A%2F%2Fwww.qwant.com%2F

Together, it might be that the strength of Israeli agencies in cyberspace is one-sided and that this reduced their ranking.

The idea that they are just good at hiding their capabilities is enticing, I see too many secrets coming out eventually to make this believable.

No. No • September 11, 2020 8:55 AM

One criteria should be how long crooks can keep servers for malware distribution or C2 in a specific country, or how fast the malware is removed. Most of the countries mentioned could (should) improve to protect their citizens.

Funky RSS? • September 11, 2020 9:07 AM

Anyone else notice that the RSS feed is a little messed up? The mime-type wants to force a page save for all feed articles, not a single page view.

anon1 • September 11, 2020 10:46 AM

I am also a bit perplexed about this list. Israel not in the top 5 (or even 3) is a surprise.

I would also figure on both North Korea and Iran in the listing, although I would say from a point on national cyber power they tend to operate highly decentralized.

metaschima • September 11, 2020 12:33 PM

I agree with the above posts, Israel is definitely in the top 10 if not top 5 or top 2-3. Anyway, for sure visibility is a major factor that was not taken into account otherwise other countries would have been ranked higher maybe even than the US.

Bruce Schneier • September 11, 2020 2:35 PM

Okay, everyone. Read the report. Read the criteria. Read the analysis. What specifically are the errors that result in Israel being lower on the list than you think?

Ismar • September 11, 2020 5:36 PM

@Bruce
Israel is probably best at keeping a low profile and that is why it was not included in this list of top 10 despite Stuxnet, NSO group and the industry where most of the engineers switch working between military, intelligence and civilian jobs enabling cross pollination of technology. In addition, it is privy to secret information sharing with other 5 Eyes countries which has been written about before but not widely reported in mainstream media.

Winter • September 12, 2020 5:00 AM

“Read the analysis.”

It is in the text:

The public is informed of the cyber impacts of only a handful of countries: notably U.S., Israel, Iran, China, Russia and DPRK. Most news coverage reports on only the large-scale or dramatic offensive cyber-attacks. This is a misrepresentation of the full scope of the capabilities, objectives, and the range of actors in cyber space. Additionally, when reporting on these, there is no systematic measure or comparison of even this narrow range of cyber capability.

Look at graph 2. Israel ranks:
intelligence: 4
surveillance: 25
commerce: 11
defense: 23
offense: 6
information control: 4
norms: 17

Anders • September 12, 2020 11:56 AM

First post after blog move.

I was actually very surprised to see Estonia
on that list, at all, and quite high position.

Most of Estonia’s so called “cyber power” is actually
coming from US as NATO Cooperative Cyber Defence Centre
of Excellence is located in Estonia. Without that Estonia
would be hardly on that list. Our neighbors Finns have
order of magnitude better cyber capability, mostly in
defensive area – F-Secure lab has some unique top notch
capabilities.

Also…rumors tell that Stuxnet was developed jointly
with US and Israel in CCDCoE Estonian lab.

Anders • September 12, 2020 12:39 PM

From that report:

“Estonia, often heralded as a beacon of cyber and digital capability, made the top 10 for only two objectives: intelligence and offense. Whilst this is impressive for a country of under 1.5 million, it is perhaps not as impressive as the team were expecting.”

Sorry, but i don’t know even one single Estonian company that is flagship in those fields. Both offensive and intelligence capabilities are coming from US. Finfisher, which is heavily
used here by LEA, is foreign product. We don’t have our own
such industry / companies. We have only some training companies.
Does anyone know ANY Estonian Antivirus company? Really, any?

Sadly, military has captured the “cyber field” world wide and
only fraction of the capabilities is ending up where it’s really
needed – defensive for healthcare.

Internet_individual • September 13, 2020 4:29 AM

I am torn regarding how I feel about the rankings in general. I realize there are too many variables, perspectives, and unconfirmed info that comes in to play for a more “realistic” list in terms of efficacy in capability. Which is why I suspect the indicators were decided on to begin with as they are a measured known. Maybe that wasn’t the purpose, either way im having a tough time seeing the value in the rankings.

So, I thought to myself I could do a much better job. I’ll show them. Ill make a list that conforms to my own perceptions based on the most precise and accurate subjective unknowns available. Sources used: The internet, and what I overheard someone say, I think. I called my paper “The National Cyber Power Index Ultra. Anyways, long story short it turned out to be a disaster for obvious reasons. I had fun with it anyways.

A few of the interesting bits(for me) while creating my superior rankings index were countries potential capability vs actual results. For instance.

Even though America ranks first in the area of controlling information and manipulation, it appears to me at least that we are doing rather poorly in practice. All of the fake news articles, disinformation, fake accounts on social media, and even a large Cult followings with millions of people that were radicalized via manufactured documents and videos, and websites viewed to convert its followers, including politicians. The damage being caused by the apparent lack of control of information is difficult to quantify. If we are doing good in this area I would hate to see what doing bad looks like. The underground darknet marketplaces selling anything from drugs to zero days. I found that there is a difference between having the potential capability and just not using it for one reason or another. If there were no laws or borders can NSA easily assert control?

In the area of commercial gains or protection from the perspective of say NSA being able to monitor domestic infrastructure. Take a country like…..i don’t know, lets just pick one at random,… China. Lets say we have capability to watchh their hackers as they break into our businesses and steal IP, but we don’t stop them either because we don’t have that ability or because we don’t want to tip them off to our surveillance capability. How might I score America here? If we can watch them as they do it, that should account for something right? Maybe negative points depending how you look at it.

Another interesting thought is how Five-Eyes countries capability would play into a ranking system. Intelligence collected by other countries is shared with us and vice versa. On their own any particular country might rank poorly, but because of the partnership the sheer volume from the of information increases effectiveness of capability. By that logic wouldn’t the 5 eyes take the first 5 spots in several of the categories?

OR simply a form of government having effects on capability. For instance, lets say China has the same exact tech and infrastructure as America. From a technical perspective they might have the same score. However, in terms of efficacy since China has the more authoritarian government domestically they likely can control information better, simply from people not having much choice in how they are censored, what information is allowed, and privacy protections they might have if any. So who might get the higher score here? To me, it seems like they have a more hardened position in this regard compared to America.

I petered out about here. Never actually got around to a Ranking either.

Anders • September 13, 2020 5:34 AM

Actually i have far better ranking (mainly for myself).
Flare On challenges results.

http://www.fireeye.com/blog/threat-research/2019/09/2019-flare-on-challenge-solutions.html

Whoever can take the most obfuscated malware apart, can
use the same information for both offensive and defensive
purposes.

Flare On best scores comes from limited amount of
countries. Poland, for example, is there traditionally
on high ranks, however you can’t find Poland in the list
our host here posted. Estonia is not among the Flare On
results at all, btw. Makes you think.

Of course there’s reason for those differences in those lists.
Most Flare On challenge finished people i personally know
just hate that filthy work for the government, they have their
own ethics. So those talents never end up working for the so called “National Cyber Power” and never be on that list.

Winter • September 13, 2020 6:28 AM

“I am torn regarding how I feel about the rankings in general.”

The question I always ask myself is “What decisions are informed by this ranking?”

Oscar’s won are used to chose movies to watch, Michelin stars are used to select restaurants.

I see three potential users of these cyber rankings, businesses selecting a more secure home, students of security looking for a place to study, countries looking for an example to follow.

But there are undoubtedly more.

Clive Robinson • September 13, 2020 2:31 PM

@ Internet_individual,

Even though America ranks first in the area of controlling information and manipulation, it appears to me at least that we are doing rather poorly in practice.

The first part of your statment is true for the obvious when you think about it,

“Location, Location, Location”

Reason, they are after all at the center of the physical architecture that makes up the web. Closely followed by the UK and it’s old Colonies and Dependencies that make up the rest of the “Five Eyes” they do after all sit for historical reasons on the “nodes” of subsea cables and in well placed positions to pick up “dish spill” from geostationary and other communications satellites.

Other countries just do not have every other countries electronic signals crossing them.

I’ve made this point several times in the past on this blog urging people to actually get and look at a map of undersea cable “land fall points” to see where the “physical choke points” are. I’ve also mentioned satellite foot prints in the past and guess what they tend to fall on similar locations.

If you aren’t aware of this look up Bude in Cornwall UK, it carries a very large in fact call it vast amount of International signals. And just up the road from the “official points” there is a less obvious building complex where all the signals go through. All the Telco’s that use Bude are only to aware of this building complex, they have to be for technical reasons, but they don’t generally talk about it as managment do not see it as good for business…

As for the US IC and SigInt agencies apparently not doing anything to protect citizens, businesses, and corporations, and even voting politics, you need to ask yourself the questions,

1, Why should they?
2, What advantage is there in doing so?

The answer to the first is it’s “not in their charters” and to the second the answer is the downside far far out weighs the up side. Or to put it another way more than half the time the inteligence they gather is entirely spurious, bogus, or made up, for various reasons, it’s the reason for the “Two or more INDEPENDENT sources” rule.

The simple fact is only a very very tiny fraction of traffic intercepted is “actionable” in any way prior to an event. It’s why I refere to “collect it all” as “building a time machine” because the real purpose is not “prediction” but “tracing history” it’s a form of “Traffic Analysis” that was first developed in Britain by Gorden Welchman back in the early days of WWII. Basically you take every bit of inteligence you gain and put it in a database (nearly four million cards in a card index by the end of WWII) and you “link build” to “trace actors”. If you know that “Person A” has certain training and history you know what their capabilities are. Thus when they get moved from post to post you know what they bring to the new party and what they take away from the old. This gives you a very good strategic view of the “Order of Battle” or capabilities of those units etc you have under observation. It also historically “fills in gaps” about people you were not watching or were unaware of.

It’s this information more than most others that get woven into whole clorh, then cut and stiched to suit for inteligence reports.

As was reveled by the Manning Cache made public by Wikileaks and Julian Assange and now getting retold in a UK court currently, most drone strikes made on places like the middle east were on innocent individuals. Because they were based on “sightings” and other nonsense from “on the ground informants”. Put basically for various good and very predictable reasons the informants out of fear of tourture or death of them or their families were lying to US intelligence units on the ground and in gittmo etc. Thus the as the US on the ground IU’s did not follow the “Two or more…” rule lots of innocent people got killed, way way more than any real targets. So the US “high tech” “Smart Weapons” advantage that cost billions if not trillions of USD got “pi55ed up the wall” and actuall caused way way more problems than it solved.

To put it bluntly the US military IC was living in a fantasy world based on stupid machismo that gave rise to useless tourture etc that in turn gave rise to usless air strikes etc that cost more money than the GDP of the countrirs being attacked and had little or no effect on the organisation the US were trying to attack, other than turn the populations of those countries towards those organisations.

What some would call “a series of very expensive own goals” which obviously various people do not want known as their empires might cease to be funded etc by the politicians that hold the purse strings.

Which brings us to another point the US IC spys on US politicians very intently. This came to light when Barack Obama got fed up with variois US politicians being under the control of a Foreign Government and revealed the transcripts that proved it to the offending politicians. Who were outraged that their totaly illegal favours they were doing for the Foreign Government were revealed rather than being contrite they actually tried to justify the illegal things they were doing for that Foreign Government.

Imagin if you will what other “dirty little secrets” the IC has on US politicians…

It might explain your second point.

Ismar • September 13, 2020 8:30 PM

@Clive – a bit late to the party but hoping you may answer with some more insights here

“Other countries just do not have every other countries electronic signals crossing them.”

Hmmm, how likely do you think the other countries would be to send their classified information on these lines? Even if encrypted, the messages would reveal the metadata that can be used to identify the sender and the receiver (and no I don’t think that using TOR is good enough against state level players).
This public internet channel might only be useful for communicating between embassies (where metadata does not matter) with some sort of very strong encryption and previously known secret keys kept physically safe.
All other types of communications could be intercepted and harnessed for metadata if not for the data itself.
Hence, it becomes much more important to be able to get the information at the endpoints before it is encrypted as we have discussed many times before on this blog, rendering the internet hubs far less important when discussing information collection at the levels of countries.
Developing this capability does not require major infrastructure to pass through your country but it, instead relies on targeting the other country systems from inside which leads into human resources territory as the weakest link in the computer security chain, or by means of covert electronic channels such as rootkits and zero day vulnerabilities implanted into very low level hardware (hence 5G Huawei backlash) and software infrastructure used by a potential opponent.

Internet_individual • September 13, 2020 8:32 PM

@Winters

Fair point regarding what decisions can be made from the information.

@Clive Robinson

You sir are a living library of knowledge. I enjoy reading your views and insights. Being a rookie, some of what you say goes over my head. Regarding location being a factor taken into consideration. It almost reminds me of the arguement about which operating system is the most secure? It’s likely the one least used. Even if 5-eyes countries are the most advanced and resilient regarding security mechanisms, if 90% of the world hackers are targeting you how secure might you be?

I enjoy reading and learning about the old days, WWII and cold war spy stories. I get the impression back then there was a sense of honor and respect between adversaries working intelligence.

About the insights you gave regarding the political situation. This is quite literally what keeps me up at night. Especially now. The more I learn, the more I wonder what is really going on. It feels like certain people are hell bent on betting the farm. Someone being kept “outside” can only infer so much. But, I guess if the puzzle had an easy solution I might be bored.

Thanks again!

Anders • September 14, 2020 3:40 AM

Just to give to everyone an overview with what kind of
madnesses reverse engineers this year at Flare On
fight.

mobile.twitter.com/daubsi/status/1305161044923473922?p=v

If you ever reverse engineered and used IDA Pro, you’ll understand.

Anders • September 14, 2020 8:49 AM

@Winter

I’m not surprised with Netherlands /AIVD capabilities.

There were 9 persons from Netherlands in the Flare On finish list
last year. Impressive.

And also let’s not forget this:

arstechnica.com/information-technology/2018/01/dutch-intelligence-hacked-video-cameras-in-office-of-russians-who-hacked-dnc/

metaschima • September 14, 2020 9:32 PM

@ Bruce
Admittedly I initially skimmed through the article, but now I have had time to go through it more thoroughly. I believe they put too much emphasis on visible capabilities. They actually list this a (major) weakness of the study. However it is a great study, and actually Graph 3 is the most useful to me. Knowing that we cannot know for sure the actual capabilities of a country I see two possible ways of correcting this. The graph can be interpreted by ranking countries from upper right corner to lower left, OR top to bottom (the least biased). If you do this Israel is #6 both times, but otherwise the list is in largely the same order. Also, we have seen over the years the extremely sophisticated software that Israel can produce, so I have no doubt of their capabilities, and according to the study itself the intent is definitely there.

Leave a comment

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.