Chinese Supply-Chain Attack on Computer Systems

Bloomberg News has a major story about the Chinese hacking computer motherboards made by Supermicro, Levono, and others. It’s been going on since at least 2008. The US government has known about it for almost as long, and has tried to keep the attack secret:

China’s exploitation of products made by Supermicro, as the U.S. company is known, has been under federal scrutiny for much of the past decade, according to 14 former law enforcement and intelligence officials familiar with the matter. That included an FBI counterintelligence investigation that began around 2012, when agents started monitoring the communications of a small group of Supermicro workers, using warrants obtained under the Foreign Intelligence Surveillance Act, or FISA, according to five of the officials.

There’s lots of detail in the article, and I recommend that you read it through.

This is a follow on, with a lot more detail, to a story Bloomberg reported on in fall 2018. I didn’t believe the story back then, writing:

I don’t think it’s real. Yes, it’s plausible. But first of all, if someone actually surreptitiously put malicious chips onto motherboards en masse, we would have seen a photo of the alleged chip already. And second, there are easier, more effective, and less obvious ways of adding backdoors to networking equipment.

I seem to have been wrong. From the current Bloomberg story:

Mike Quinn, a cybersecurity executive who served in senior roles at Cisco Systems Inc. and Microsoft Corp., said he was briefed about added chips on Supermicro motherboards by officials from the U.S. Air Force. Quinn was working for a company that was a potential bidder for Air Force contracts, and the officials wanted to ensure that any work would not include Supermicro equipment, he said. Bloomberg agreed not to specify when Quinn received the briefing or identify the company he was working for at the time.

“This wasn’t a case of a guy stealing a board and soldering a chip on in his hotel room; it was architected onto the final device,” Quinn said, recalling details provided by Air Force officials. The chip “was blended into the trace on a multilayered board,” he said.

“The attackers knew how that board was designed so it would pass” quality assurance tests, Quinn said.

Supply-chain attacks are the flavor of the moment, it seems. But they’re serious, and very hard to defend against in our deeply international IT industry. (I have repeatedly called this an “insurmountable problem.”) Here’s me in 2018:

Supply-chain security is an incredibly complex problem. US-only design and manufacturing isn’t an option; the tech world is far too internationally interdependent for that. We can’t trust anyone, yet we have no choice but to trust everyone. Our phones, computers, software and cloud systems are touched by citizens of dozens of different countries, any one of whom could subvert them at the demand of their government.

We need some fundamental security research here. I wrote this in 2019:

The other solution is to build a secure system, even though any of its parts can be subverted. This is what the former Deputy Director of National Intelligence Sue Gordon meant in April when she said about 5G, “You have to presume a dirty network.” Or more precisely, can we solve this by building trustworthy systems out of untrustworthy parts?

It sounds ridiculous on its face, but the Internet itself was a solution to a similar problem: a reliable network built out of unreliable parts. This was the result of decades of research. That research continues today, and it’s how we can have highly resilient distributed systems like Google’s network even though none of the individual components are particularly good. It’s also the philosophy behind much of the cybersecurity industry today: systems watching one another, looking for vulnerabilities and signs of attack.

It seems that supply-chain attacks are constantly in the news right now. That’s good. They’ve been a serious problem for a long time, and we need to take the threat seriously. For further reading, I strongly recommend this Atlantic Council report from last summer: “Breaking trust: Shades of crisis across an insecure software supply chain.

Posted on February 13, 2021 at 9:41 AM45 Comments

Comments

SpaceLifeForm February 13, 2021 11:09 AM

Silicon Turtles

While I can roll my own software, including building my toolchain from source, I can not create my own fab in my basement.

If you want serious security, you must look at FPGA to manage your keymat. Inside a Faraday Cage.

SocraticGadfly February 13, 2021 12:21 PM

THIS is the fun part in the Bloomberg story:

>Bloomberg Businessweek first reported on China’s meddling with Supermicro products in October 2018, in an article that focused on accounts of added malicious chips found on server motherboards in 2015. That story said Apple Inc. and Amazon.com Inc. had discovered the chips on equipment they’d purchased. Supermicro, Apple and Amazon publicly called for a retraction. U.S. government officials also disputed the article.<<

Lukas February 13, 2021 12:27 PM

That story still isn’t plausibly, and still doesn’t offer any evidence. The description of how this works is implausible (you don’t have chips between layers of a PCB somehow inside a trace), and if this story was true, we’d have thousands of physical pieces of evidence out there that people could easily examine. Yet, since 2018, for some reason, zero actual evidence has appeared.

This story was bs then, it’s still bs now, and it will remain bs until somebody offers actual evidence, which, again, should be super easy, given what the actual claims are.

Vesselin Bontchev February 13, 2021 12:54 PM

Bruce, you were not wrong. The previous Bloomberg story on this subject was bullshit – and this current story is bullshit, too.

M1999 February 13, 2021 12:57 PM

Hmmm.
I wonder, with Apple’s drive for security… is the M1 chip a response to this kind of problem.
Aside from sidestepping Intel chip pricing.

vas pup February 13, 2021 1:52 PM

Related to tag – CHINA:

China’s Tianwen-1 enters Mars orbit

https://www.dw.com/en/chinas-tianwen-1-enters-mars-orbit/a-56524320

“Tianwen-1’s Mars mission

The goal of Tianwen-1 is to survey the atmosphere from orbit over a two-year period.

About 4 billion years ago, the atmosphere of Mars changed and liquid water evaporated. But scientists don’t know why. There may be deposits of water underground, known as subsurface water.

So the mission is intended to go deeper than the Hope probe will. It is scheduled to land a rover on the surface of Mars in May.

A precise location has not been named, but the mission controllers have been looking at the southern part of a region called Utopia Planitia.

The solar-powered rover is expected to operate for a few months, looking for subsurface water — signs of life below the surface of the planet.

It will be roving Mars along with American landers, such as NASA’s Mars 2020 Perseverance rover and its accompanying Ingenuity helicopter.

Instruments on the probe and rover

Tianwen-1’s orbiter, or probe, is carrying 13 “payloads.” A payload can be a communications or Earth observation satellite, but in this case the payload is the mission’s instruments.”

Interesting! Read the whole article.

SpaceLifeForm February 13, 2021 2:08 PM

@ Clive, ALL

You don’t need a hidden chip buried inside a PCB to leak via RF via traces.

Chinese guy February 13, 2021 2:33 PM

People should not be so naive about China’s security capabilities (or any nation player’s for that matter). Whatever you’ve seen in TV/movies, the real spy tools will only be 100 times more creative.

Now being effectively the world’s largest economy and with much higher tax income than most countries, China can spend a lot on this kind of stuff.

Nearly 20 years ago, a professor at my CS department in China was approached by a guy from a large arms manufacturer (all state-owned of course). He wanted to know if it would be possible to modify the microcode of Intel processors (which was already update-able via BIOS) to plant undetectable rootkit. The prof didn’t brag all the details, but from what I gathered, they’ve been focusing on the network exfiltration possibilities.

We both thought this was just theoretical work to justify getting funding to switching to “China-controlled” hardware, but now came to think about it, it could also be used for offensive.

With SMM and ME on Intel, there are a million ways to do this kind of rootkit if you can sign the BIOS image.

TimH February 13, 2021 2:50 PM

I was told by a senior engineer at a very large defence company in CO (reception had a poster up advising an armed incursion drill) that he had seen a couple of these motherboards. So probably the MBs were only modified for targetted customers – hence the evidence could be minimised – which adds complexity to the attack for sure.

da5id February 13, 2021 2:54 PM

I am not so sure you were wrong. This article read very weak with third person sourcing at best, and no further tech details to be had.

I think there is a kernel of truth about a chip on board attack, but it was a very targeted attack against a US government org. The claim last time about Apple and Amazon is obviously BS, and they didn’t bring that back up.

Everything else is whispers and rumors from self serving private security people, wanting to get attention. And some truly abysmal reporting and editing from the folks at Bloomberg.

It would be great to get someone interested in following up on the story from the journalism/media angle. . .

more facts please February 13, 2021 2:55 PM

Multiple forms of attack are described, but the write-up doesn’t say who has actually laid eyes on compromised hardware. Now the add-on chip is embedded in the PCB (per Quinn’s hearsay from the USAF) instead of added later? Is that or is that not what Mott’s FBI colleague said they saw? Altera/Kumar’s statement about being warned about a chip that isn’t supposed to be there — is that something he or his people ever saw? (Apparently not, but he’s corroborating the fact that the warning was being disseminated around industry in “unclassified briefings”.) Likewise Janke reports he was advised about FBI briefings about added chips. Can we really not get the FBI on record saying they saw these added chips, and maybe when what and how many?

Apropos of SolarWinds: were/are the Supermicro BIOS and firmware (if those mean different things here) signed? Intel says the downloaded firmware wasn’t “tampered with” — but (additional?) malware was downloaded from the same Supermicro update site. What does that mean?

Ask the “befuddled NSA” if we’ve found the PLA’s equivalent to the TAO.

Bruce Schneier February 13, 2021 3:01 PM

Independent from this article, I know that the DoD believes this threat is real and is giving away R&D funding for security solutions. So I have more than the Bloomberg reporting to go on.

SpaceLifeForm February 13, 2021 3:03 PM

@ Chinese guy

I do not believe that any signing of firmware (BIOS or UEFI) is needed.

The signing is security theatre.

Silicon Turtles.

Jimmy February 13, 2021 6:32 PM

I will remind people that the PRC has at least the same capabilities and opportunities to insert supply chain hardware attacks as our own CIA & NSA. Snowden’s revelations reveals the NSA doing the same damned thing as Bloomberg said China was doing to Cisco and Juniper network devices as PRC was reported doing to SuperMicro boards. It’s not at all out of the realm of posibility, and there are reports surfacing of it happening on hardware headed towards the US Gov. You better damned well believe the PRC’s MSS doing the same thing to targeted devices headed towards our top line tech companies as well. I’ll also remind you that no one knew about the NSA program outside the government UNTIL Snowden broke the news. Same thing that happened with SolarWinds. It wasn’t the US Government that broke that news, it was FireEye networks. The US Gov didn’t have a damned clue. Apple, et al are gas lighting you. This kind of attack scares them shitless, with damned good reason, and they don’t want to admit it because their stock would take a substantial hit despite it being an obvious attack vector to anyone with half a brain.

The amount of counterfeit gear flooding the US market is almost impossible to stem, and some of them are so good they often can’t be detected by visual inspection. Sometimes it’s only firmware updates that fail that cause a review. I’m willing to be a lot of counterfeit gear is never discovered. Counterfeit chips or designed in counterfeit chips on legit boards are likely in many devices in use right now. It would be trivial for some of that gear to be reporting back to servers that eventually report to the PLA or MSS.

I agree with the article. Schneier was a bit naive when the original story broke.

Clive Robinson February 13, 2021 6:33 PM

@ Bruce,

From the quote you give above,

Quinn said, recalling details provided by Air Force officials. The chip “was blended into the trace on a multilayered board,” he said.

Take it to an actual hardware person and ask them what they think it means?

Look at it this way, three players are involved with that quote,

1, Unnamed Airforce person.
2, Mr Quinn security executive.
3, A Bloomberg journalist.

A hardware engineer will tell you one or more of them does not have a clue what they are talking about.

If you want I can go into details but consider three things,

1, The entire PCB would have to be custom and the chip embeded as part of the manufacturing process.

2, A trace inside a PCB is a passive component that has a transmisson line impeadence Z0, defined by it’s width, thickness, dielectric of the PCB material either side, distance from other traces or planes, and how it is routed. Further it changes Z0 if the trace has more than one branch. Such traces often have to have “delay balance” added to ensure the signals arive together. These signals are in the low UHF range where if Z0 is not right the resulting motherboard will be flaky or not work.

3, Any chip is an active not passive device and will need to be connected to rather more than one trace. It’s unpowered and powered Z0 at any of the pins will be different.

Thus inside the PCB is realy not a good place to put such a chip.

If I was going to do something similar to what Bloomberg are suggesting, I would go about it entirely differently…

One such way is “a chip on a chip” inside a chip package, the technology is in use already and thus the machinary available.

Another way is a “chip swap” you simply but a similar but different functional chip inside the package. In fact you don’t even have to use a different chip… Many chips do quite a bit more than the data sheet suggests, part of this is to do with production cost reduction. You simply reprograme the chip to get the extra functionality. There are various ways this can be done some are quite essoteric and you would need to go to a specialist in chip design, manufacture, encapsulation and more fun bits.

If you did it that way you would then make it “non-obvious it was functioning”.

A fairly dumb way that most can see would be that Intel have certain things like page tables at fixed locations in memory. Think about what you could do if you could change one or two of those bits… You actually don’t have to, it’s been discussed before with regards the “RowHammer Attack”.

But that aside back to the quote you give it’s typical of Bloomberg style… It’s fairly meaningless but sounds very meaningful and redolent with mystic James Bond high tech Q branch to those who don’t know better… But consider even the best of high tech wizzards are constrained by the laws of physics and quite a bit more besides… But even if you could do it you have to ask “Would you do it?” and the answer to that is most likely no due to the obvious reason.

As before, I’m going to say this story is at best cobbled to gether from mostly unatributed quotes, which is what this is. Mr Quinn is spreading hearsay from an unnamed Airforce Source, that actually had no reason to tell him the truth.

The chances are the Airforce had sized Mr Quinn up, then they “sold him a bridge” for some reason, why would be anybodies guess, but as with many bridge sales, the bridge it’s self may not actually exist. The point is neither Mr Quinn nor the Bloomberg journalist or if there are any, the Bloomberg editor either, appear to have “fact checked” or sort multiple independent sources on this…

I could go through the rather dull Bloomberg article from begining to end pulling out what smells of something “in the state of Denmark” as Shakespeare once had occasion to add to the richness of the English language.

But why waste my time? Or those of the reader? The simple fact is that yes a number of countries are capable of doing such things. They could even get the Chinese to make the chips for them… You realy do not know, and from what I’ve read there are no real “facts” just “Fairy Gold” that is trying to be passed as “Dane Gelt”.

The only thing we do know for certain is that somebody is “playing a game” the question is “Who?” fairly rapidly followed by “Why?” and “Where?” the target is.

@ ALL
It just so happens Michael Bloomberg’s Birthday is tomorrow, he will be 79. I wonder if he has found having a birthday on Valentine’s Day to be a bit of a curse, as some do with Xmas day…

Ismar February 13, 2021 6:54 PM

The old saying: “ You get what you pay for “ comes to mind, so , as long as the bottom line does not take wider view of impacts to national security ( sovereignty) and looks only at the immediate financials , these attacks will be inevitable.

On the other hand, given the frequency and the extent of these security breaches (Russia and China) I would not surprised that these 2 countries are getting awfully close to technical and military capabilities of the USA which may, paradoxically, make these types of attacks less frequent as there will be less secrets to look for from now on.

Nicholas Weaver February 13, 2021 9:30 PM

Unfortunately I don’t believe this reporting. Yes it is plausible and concerning, but Bloomberg’s reporting on this has been SO bad, and a really careful analysis by Matt Tait on twitter discussed many of the problems with this.

I will not believe anything that crew writes about security issues unless/until it is confirmed by other outlets.

Alan February 13, 2021 11:40 PM

The “evidence” smells like it came out of game of Telephone, which each person in chain adding their own misunderstandings. I think the truth is that Supermicro shipped one or more systems with a compromised BIOS, and “BIOS had malicious code added” became “malicious chip (by virtue of holding a malicious BIOS)” becomes “A malicious chip was added”.

AL February 14, 2021 1:23 AM

While I’m out of my lane commenting on this issue, I did once troubleshoot an issue where a controller of some drives was compromised by bad “microcode”, resulting in what we called in the IBM world, “abend”.

The issue is not simply whether a chip on a device is doing bad stuff. What also has to be watched out for is whether the programming on the chip can be updated, and if so, by who.

Lukas February 14, 2021 1:54 AM

the MBs were only modified for targetted customers

But consider what this actually means. If you want to embed a custom chip in a subset of PCBs for specific targeted clients, you need to have two different designs for the PCB, you need to have these custom chips in the pick-and-place machines, you need to have custom routines for these machines, you need to know ahead of time which PCBs will go to which specific customer, and then you need to change all the programming for all the devices for that specific customer. At some point, this means that there are hundreds of people aware that something is going on, with direct access to hard evidence – not people working for some kind of shady government agency, but regular workers on the factory floor.

It just doesn’t seem plausible that this is happening, and yet no hard evidence has surfaced.

To be clear, supply chain attacks do exist, and are real. I just don’t think this specific story is very likely to be correct, at least not in the way it is being reported.

Lukas February 14, 2021 1:58 AM

(I think the most likely explanation is that software, rather than hardware, is being attacked here. Maybe there’s some kind of backdoor in some kind of firmware. This got garbled through multiple miscommunications as the information made its way from the person who actually knows what’s going on to the reporter.)

Clive Robinson February 14, 2021 5:26 AM

@ Bruce,

I know that the DoD believes this threat is real and is giving away R&D funding for security solutions.

Yes they DoD amongst others have been getting their panties in a very tight wad over it for most of this century. Especially as they have more than good reason to think that China is actually ahead of the US technically[1] and Russia way ahead on natural resources.

If you look back on this blog you will find @Nick P actually found and raised the original “open” invitations to tender(ITT) and the subject got discussed in some depth then and later.

However over the years as @Nick P noted the “open” went to “closed” to “secret” and who knows where after that, and as happens with that sort of progression nobody realy knows what is going on any longer just that money goes out via various less than obvious or distinctly obfuscated if not hidden off of budget sheet ways.

In essence the DoD know that US,Corps are probably their number one enemy when it comes to the loss of this sort of “secret information” and thus does what it can to keep information out of their hands or in as tiny and well segregated ways as it can with “cover stories” (Which if the Bloomberg Journalist did not dress up and in effect fabricate, is what the unnamed Airforce Person(s) did to Mr Quinn).

Those with a little more domain knowledge can rip through the Bloomberg artical showing up the nonsense contained within.

Thus a problem arises, every one thinks wolves are dangerous, and as the fable has it “the boy cried wolf once to often” and nobody believed him, thus the wolf got to be fed…

Well think of these Bloomberg articles as “crying wolf” and follow the logic.

Which brings us to,

So I have more than the Bloomberg reporting to go on.

You are far from the only one, which is why we take the issue of supply chain adulteration seriously. What we need is reporting that is calm, collected, factual and above all else evidenced based.

I think many would think the Bloomberg article is in many ways the opposite. So much so you have to have in the back of your mind the “Why?” question followed by the “Who?” and “What?” do they get out of it questions…

That is “Who gains from Bloomberg crying wolf, and what does that mean for us?”.

That’s the nub question, and the one we should dig into. Because “Actions have consequences” and Bloomberg’s actions are already having consequences we realy do not need.

Oh and one last point to note, those named in the Bloomberg article like Mr Quinn, ever heard of them before? They are not exactly industry wide recognised leading lights… Which could be a sure indicator that they are being used by the Bloomberg journalist to get the sort of answers they want to spin up their story.

There used to be an expression “yellow journalism” that covered this sort of behaviours. The modern version of “Tabloid journalism” has sort of watered down the meaning so lets just say Bloomberg has a “yellow streak” a mile or so wide and it’s getting wider.

The important thing is not to follow Bloomberg’s line of conflating a very real risk with sensationalism that is in the main disprovable “Hog Wash”. In other words “We do not want to throw the baby out with the bath water”…

[1] Due to the stupidities of various political view points that insisted on COST and killed much high end research, “And you shall reep as you have sown” is somewhat apt. The point being some things can not and most definately should not “be left to the market”, for a number of reasons. Firstly at best “the market” in the US only has interest in short term gain thus research has to pay off quickly within a couple of years or three at the most on average. Likewise it has little interest in colabaration other than by aquisition which gives rise to a distinct lack of plurality in the accompanying research domain. These things others and myself have warned about, I won’t go into what has been going on in academic research, but with many US Universities effectively turning themselves into “Hedge Funds” it’s not hard to work out. But the next stage is also problematic the Corps that drive “The Market” are run in a certain way that encorages certain very reckless if not down right idiotic behaviours. As I’ve continuously warned out sourcing across jurisdictional boundries is a very idiotic thing to do as not just IP theft happens, you are legaly barred from doing anything about it, and even raw materials, tools and finished goods you thought you owned you now don’t, and you might even become a criminal as well. You would think those that “manage” in “the market” would realise this, and who knows they might but their short term behaviour and mantras blind them and destroys the entities they work for in at best the medium term, taking with it not just jobs, skill sets and domain knowledge. Further with such losses there is no incentive to rebuild, build, train or retain knowledge in the domestic economy which then tanks… I could go on, but unless the US changes it’s very short term managment behaviours the downward trend will continue, and it’s grip on technology will wane even further.

Clive Robinson February 14, 2021 5:51 AM

@ ALL,

With regards Nicholas Weaver’s comment,

Unfortunately I don’t believe this reporting. Yes it is plausible and concerning, but Bloomberg’s reporting on this has been SO bad, and a really careful analysis by Matt Tait on twitter discussed many of the problems with this.

The link to Matt Tait’s comments was not included, so to save others the effort,

https://twitter.com/pwnallthethings/status/1360234953011851264

It needs to be noted that Matt Tait is not attacking the technical side, but those supposadly used as sources by the Bloomberg Journalists.

AlexT February 14, 2021 8:14 AM

The return of the microchip – episode II.

Well it would seem Bloomberg are doubling down, but still very short on specfics.

Could it happen ? Most certainly (and DOD should most defnitely be on the watchout).

Is there any concrete evidence ? None whatsover as far I know.

Drone February 14, 2021 10:38 AM

@Schneier said: “…motherboards made by Supermicro, Levono, and others.”

Typo or camouflage?: “Levono” > “Lenovo”

TimH February 14, 2021 11:20 AM

@ Clive,

” Quinn said, recalling details provided by Air Force officials. The chip “was blended into the trace on a multilayered board,” he said.

Take it to an actual hardware person and ask them what they think it means?”

As an actual hardware person, I think you are misreading the quote. All MBs are multilayer, doesn’t matter. The quote says on not in, so not buried inside. If I was going to do it, I would tap onto the SMBus (two wires), so need some part of the board with SDA, SCL, 3.3V, GND all together, on one side or the other, which is likely near an SMBus peripheral.

lurker February 14, 2021 11:46 AM

I’m old, confused, demented, &c, but my recollection of V.1.0 of this story was the exfiltration chip was concealed in the ethernet socket. That’s a big chunk of hardware with plenty of room inside, and some follow-on stories had photos of unnamed boards with an ethernet socket. Scepticism abounded on powering the chip without visible traces, or am I remembering another story? As the hucksters say, “Show me the money.”

JR February 14, 2021 1:35 PM

@Lurker

Whenever I get a new dongle or power supply I obsess over what’s in it. I saved a few no longer used and one day I will smash them open.

I recently had a laptop power supply that was so big and heavy it frightened me.

xcv February 14, 2021 4:25 PM

Supply-chain attacks on consumer device makers who won’t publish the specs for the basic component parts they outsource, and won’t outsource them on a competitive bid basis.

There are communist spies everywhere and they can’t enforce confidentiality agreements on their suppliers, and they can’t or won’t make their own stuff in-house. Is that what this is all about?

So. We’re stuck without jobs in America. We can’t compete with non-immigrant aliens on H1-B visas. Because we’re either underqualified or overqualified if we ask for a “living wage” for high tech work — and it really isn’t that much we’re asking for but the answer is always no because they’re full of $#!+ in a court of law with mental health warrants on top of intellectual property claims. And …

IBM and ExxonMobil are building quantum algorithms to solve this giant computing problem
ExxonMobil and IBM’s researchers have been WORKING TOGETHER to find quantum algorithms that could manage the global fleet of merchant ships.

That’s Chinese 工合 or “gung ho.” There’s a labor union of communist-indoctrinated corporate-hired academic “researchers” at all these big multinational megacorporations who all “work together” for a “common good” of One World Government of self-important “stakeholders” who may or may not be actual “shareholders” but in any case that worldview just happens to exclude individuals and small businesses entirely.

It’s the Trilateral Commission, Bilderberg Meetings, Agenda 21, NATO, the New World Order, and so on and so forth.

JonKnowsNothing February 14, 2021 4:38 PM

@All @Clive

re: USA Malware Hardware

When first published the NSA ANT Catalog had a great number of physical devices that were inserted into various systems. It also lists the firmware implants they had in common usage at the time. The NSA ANT Catalog was published by Der Spiegel 12 29 2013.

The catalog describes the implants, technology, and usage. It including the size of the devices available (many smaller than a US Penny or 1/4 of inch) and pricing. Like all good catalogs you can order the sizes and quantities wanted for the applications.

The COTTONMOUTH Series of implants fit inside the plastic casing surrounding a connector.

One of the implants is a Persistent Backdoor (PBD) called HEADWATER. Designed specifically for Huawei routers.

Another implant is called LOUDAUTO, is an RF retro-reflector and provides room audio from up to 20feet away. Cost to the purchasing agency: $30USD

I dunno if the catalog is still available this was the link at the time.

ht tp://www.spiegel.de/international/world/catalog-reveals-nsa-has-back-doors-for-numerous-devices-a-940994.html

(url fractured to prevent autorun)

xcv February 14, 2021 4:57 PM

@ JonKnowsNothing

When first published the NSA ANT Catalog had a great number of physical devices that were inserted into various systems. It also lists the firmware implants they had in common usage at the time. The NSA ANT Catalog was published by Der Spiegel 12 29 2013.

That sounds like a “boss” at work at the NSA. All that to justify draconian workplace policies of gluing shut the USB ports on employees’ computers.

We’re talking “AboveTopSecret” levels of clearance for rank-and-file NSA employees at work on the job — so what level of clearance must the boss have, with respect to human resources and workplace policies, supervising employees use of their time at the desk, bathroom breaks, smoke breaks etc. It’s just unimaginable!

COTTONMOUTH

It sounds like somebody has a dry mouth and a cold sweat when the boss administers that dreaded polygraph lie detector test at the NSA.

LOUDAUTO

Oh please. Don’t tell me there’s a black sports car with dark smoked glass, low-profile tires, and “loud pipes” at the NSA employees parking lot off the famous freeway exit patrolled by Maryland State Police. Washed and waxed, polished to a spit shine, licensed, registered, and insured, all the right parking decals, etc., etc.

I’m starting to get a certain picture of a dysfunctional government bureacracy with “delusions of grandeur” or something else that’s a little bit off.

Clive Robinson February 14, 2021 5:49 PM

@ JonKnowsNothing,

The catalog describes the implants, technology, and usage. It including the size of the devices available

Those are what you might call “tier one” or “entry level” devices, if you go back on this blog you will find I could describe the technical functioning of them because I’d actually designed manufactured not just equivalent, but better devices.

In short in the main the sort of kit that gets sold to law enforcment from a different catalog in name and supplier as “high end surveillance” and carries a 10,000% mark up on BOM price atleast…

When you can get a small ceramic board made that has three surface mount transistors and a handfull of passives and have a middle ceramic plate with holes to accomadate them and a third as a cover plate and have bonded out terminals, you would be surprised just what some people will pay…

You can get second hand equipment which you can use to make “flip chip” circuits with unencapsulated CPU, synth and RF chips and again put in a custom ceramic package for about the same price… Or if you want on an ultra thin mylar substrate flexible PCB.

With the fun these days of realy tiny MEMS sensors (just keep the cheap ones from helium or hydrogen).

So yes these things can be done, by just about anybody with some technical skills and a back room or basment they can keep dry and clean to work quietly in.

The problem is not in making such devices, heck have a look on alibaba and you will find USB connectors with GSM phone sets in them. The problems are three fold,

1, Getting in the supply chain.
2, Getting at the equipment
3, Having it remain undetected.

As I noted in “round one” of these Bloomberg stories, the easiest way to get in the delivery supply chain is at the beging and just before the end. However some supply chains have multiple delivery chains within them. The big problem is ensuring the modified kit ends up with the right people, not the wrong people.

Again as I noted before the easiest place to actually get at the equipment is before it gets “cased up” then put in packaging with security seals added.

The real hard part is “going undetected” I mentioned one way above, where chips are used that have the desired functionality built in as standard, but not enabled except in the more expensive product line.

I can think up many ways to do what is claimed by Bloomberg, the fact that the backdooring can be done is not the issue. The issue is Bloomberg appears to be in the land of the fairys yet again with what is basically a manufactured story not abything even approaching the truth.

Whilst I care not a jot for “Bloomberg’s good reputation”, I do care about how it effects others.

Thus the second such “shooting down” of Bloomberg “Make Believe” is that it will convince people that such attacks in general are not true… Which is simply not the case.

Thus if Bloomberg’s story is to make people aware of such a threat, then most likely it will have the opposit effect…

Someone February 14, 2021 7:03 PM

To those that don’t believe this…. I do. Parts of the government can be very quiet about things for a long time. As an example, many years ago, a co-work that was an Android fan boy was talking with technical guy from a vendor tat had stopped by our office to help with some issue. The vendor said he couldn’t have a Brand X Android phone because of an agreement with part of the government had worked with. (Yes, I’m leaving out the details). No one knew exactly what the issue was with that phone, and conversation went back to other things.

A few years later that exact phone was in the news as having a nasty security issue. Seems like the coders for the camera driver for the phone were lazy. The OS shipped with /dev/kmem permissions wide open. You could put the permissions back to what they should have been, but the camera wouldn’t work. A complete fix came out a week or so later.

Maybe, just like in this case, you don’t want to let on you know about the problem. In the phone case, NSA can use it to target people with the phone, it was a great backdoor. With the added hardware to MB, if it is being added to to boards that end up out of the US, then if it were me, I’d like to use that backdoor against anyone that has one of those machines. When the s*** hits the fan, NSA can just close up shop on that backdoor and let China take all the grief and say, “Oh my, that’s bad, shame on them.”

David February 14, 2021 8:43 PM

I am sure that there are server boards out there with backdoored BIOS or swapped chips. The NSA won’t talk about them, because they do the same.
Doing that is far easier than making a “special” pcb with an embedded IC. Software based backdoors are the best as they can be remotely removed to destroy evidence.
Wake on Lan Ethernet controllers are an obvious target, presumably a lot of embedded microcode.

xcv February 14, 2021 9:12 PM

@Clive Robinson

Whenever I read about “supply chain” problems, I see a total Nazi holocaust agenda, which fits hand in hand with Communism.

1, Getting in the supply chain.

Oh I get it. Electronic parts and supplies, like guns, must be kept away from mental defectives and social undesirables.

2, Getting at the equipment

Businesses and government agencies have to do instant background checks on NICS and make sure they don’t hire mental defectives or social undesirables or allow them access to any computer equipment.

3, Having it remain undetected.

So mental defective and social undesirables have to be tracked and registered like hex offenders, and kept under constant supervision of state and local cops.

Thus the second such “shooting down” of Bloomberg “Make Believe” is that it will convince people that such attacks in general are not true… Which is simply not the case.

Bloomberg has bought out too many gun control elections. You probably can’t even talk about shooting him down without going to gaol or prison if they ever let you out of Bethlem Royal Hospital. Or Bedlam? Jail? I don’t know how they spell British words anymore.

$1M from Bloomberg group to Washington’s gun background-check campaign

Peter Galbavy February 15, 2021 3:31 AM

Imagine how advanced the US effort in this are must be. Huh.

What, the US is benevolent and innocent of all possibilities or alternative narrative they are protecting freedom and democracy? I have a bridge for sale…

JonKnowsNothing February 15, 2021 10:56 AM

@Clive @All

re:Going Undetected

iirc(badly)

When some of the Snowden files were published, there was an NSA incident where they intercept the routers and telecom systems, carefully open the package without damage (there’s a team that does this) install the devices or re-flash the programs as needed then pack it all up again so that it appears to be an virgin unopened box.

In this case, something went wrong at the installation site and the customer called “support” and the NSA realized they might open up the unit. So they hit the kill switch and fried the entire unit. The equipment provider customer service reps sent out a replacement unit and the customer never found the offending item.

The equipment provider claimed they never knew their units were being intercepted and altered.

In another post about fixing multiple vulnerabilities, imagine the fix it ticket that was entered:

Customer reported blue smoke on power up. Entire System Fried. Replacement system ordered and shipped. Suspected faulty power supply. Verified power supply specs, all within parameters. No other incidents noted. Ticket Closed.

ht tps://www.schneier.com/blog/archives/2021/02/on-vulnerability-adjacent-vulnerabilities.html

Clive Robinson February 15, 2021 11:13 AM

@ JonKnowsNothing,

In another post about fixing multiple vulnerabilities, imagine the fix it ticket that was entered

As a hardware engineer that had responsability for the education and development of other engineers, I used to emphasize certain trade craft.

The most important of which without doubt was “testing techniques and methodology” with a very large emphasis on hands on practice…

I was known to make engineers sweat in the face of their peers by springing other engineers problems on them at meetings. The main usefull side effect was they talked to each other not about the match but problems they were running up against.

The reason they accepted it was because I would take my turn at putting my hand in the hat and pulling out somebody elses problem… It is called “life long learning” after all…

If a tech support or thirdline support person gave me that sort of ticket closing report they would find me not at all friendly and they would find themselves on my bench with the offending item untill the root cause had been found.

JonKnowsNothing February 15, 2021 3:30 PM

@Clive

re: Group Reviews and Proposal Evaluations

In Ancient Times tl;dr

I have always been a fan of “group programming”; most of my employers were not. I am a fan of cross training and cross learning (I am curious); most of my managers were not. I have always found something useful to learn or re-remember during training sessions no matter how many times I’d been to similar sessions; most of my colleagues preferred to scarper. Being part of a team with a Master is an honor and opportunity to exercise the “little grey cells”; the majority of my co-workers just wanted to go home at 5.

As far as the Upper Tiers go, they could care less. The only thing they wanted to know is if we were going to Hit the Deliverable.

Some of the best sessions were mixed groups: HW/SW/UI reviewing stuff I had no idea about. Innocence paid off; I have spotted many major system design faults by reviewing code or design specs I had no idea what they meant.

  If you cannot explain to a beginner what you are doing, you have missed something critical.

There was a story about Richard Feynman: (paraphrased)

During the war project, he was sent to a manufacturing facility to review some problems in production. When he arrived they were waiting for him in a conference room with a pile of blueprints on the table. He did not know what some of the symbols meant. As the others began to explain in detail some of the manufacturing problems, he realized was too late to ask what the symbols meant.

He had to think up a way to ask, so he decided to pick a random blueprint in the pile that had that symbol. He figured that if it was something minor they wouldn’t realize he didn’t know.

He shuffled through the stack until he came to the symbol and pointed at it and asked “What about this?” They rest of the group was stunned. They quickly shuffled up and down the stack and looked at him and said “THAT’S IT! That’s the place with the problem!”

There are a number of takeaways as there are in all of Feynman’s stories and lectures. Much like the exchanges in this blog.

ht tps://en.wikipedia.org/wiki/Richard_Feynman
(url fractured to prevent autorun)

Clive Robinson February 15, 2021 6:02 PM

@ JonKnowsNothing,

They rest of the group was stunned. They quickly shuffled up and down the stack and looked at him and said “THAT’S IT! That’s the place with the problem!”

Many years ago BBC Radio 4 had a program where people of note but not fame were interviewed.

On was an early Female Design Engineer, which was quite a shock for the times.

She recounted a similar story about being “young single and pretty” and fresh faced holder of a first class honours degree in engineering, dressed to impresss. But to find her self facing a bunch of died in the wool flat cap wearing pipe smoking engineers who’s grandfathers had probably been born with both an engineers jacket, collarless shirt, dark blue bib-n-bracez and boots on their feet, in a cold and drafty nissen hut.

She to had the blue print problem, and decided the best plan was to say nothing just nod murmer and ask to see the actual system. Well after a stoney hawk eyed observation by the “men” they showed her along to the machine and she asked for the inspection pannels to be opened and she had a look around and just knew the was well out of her depth… So in desperation she asked to see the machine in operation and so they started it up. In desperation she had backed up against the case and was in effect leaning on it when she noted that for some strange reason the temprature was rising way to quickly. So she asked where the sensor was located they muttered something about it being on the tank so she asked them to show her exactly where. Low and behold it was not connected to the tank but the external casework she had been leaning on… The men looked suitably impressed and she scarpered as quickly as politness would alow, just thankfull that somewhere there must be a godess of engineers that had chosen to smile on her.

And to be honest if there is a godess of engineering she has smiled on me on more than on occasion…

Back in the 1980’s I was given a job interview by a small company that had a large customer (BT) who wanted a system of Torch Computers installed. However they had some technical qurstions and I got asked if I knew of an answer. Thinking it was still part of the interview process I gave a very precise answer with calculations and suggested equipment… Turns out the men from BT were impressed so I got the job.

But before disapearing up to Wales to do the instalation and commisioning, I got called in on another job. And it had a very funny problem in that biphase signalling across twisted pairs were used. One end box that used thr equivalent of a high speed op-amp to convert the bi-phase back to an NRZ clock signal was not working. Whilst I could see a nice clean set of input waves, there was no output wave. Turning it off and running the multimeter around showed there was no apparent circuit issues. I was stumped… So I went and sat on the loo, and stared at the back of the cubical door. And enlightenment came down from above.

Five minutes later I had confirmed the hunch… What had happened was there was a physical break inside the moulded right angle connector. When I had buzzed things out I had done it on the inside of the case from the back not the face of the connector. And that nice waveform I had seen whist being there to the scope probe was not there in reality due to the odd way the grounds had been set up with a transformer…

So chalk that up to a success I demonstrated the problem to both the customer and the supplier who was thankfully local and had a new unit a couple of hours later…

I’ve had a few other “pull the rabbit out the hat” moments in my time, one of the funniest was when I was wearing the green. I’d been dicked with showing a bunch of other technicians how to fault find and repair a load of HF radio man packs. There was about thirty, and the regular techs had marked most of them to be returned to the manufacturer for repair. Which is a long slow and expensive process so the pile had grown untill things were dire and various people with crowns on their shoulders were huffing and puffing and dropping veiled threats… So to my horror when I’d just started showing the technicians how to open the cases without mucking up the seals in walked not just one of the majors but the regimental colonel… I thought I’m doomed but in more squadie like verbiage.

So I just dug in and it was just one of those days, I would describe how to walk the signal path and touched various components as I went. So thirty seconds into the first set the 2N3866 under gental preasure from my finger revealed an intermitent fault. So I got the first tech to start swapping it out. The next set and my finger revealed a cracked trace on the PCB. And so it went on ten minuites later and I’d found all the faults on all the sets and the techs were busy fixing them. The colonel looked at me and enquired who I was, so I explained I was not a regular but from the TA tech unit with HQ Squadron in London and my day job was working in the oil industry. The colonel looked at the major and said “Maybe you should get him up here more often”… The following week I turned up at HQ and got shouted at by the RSM for being improperly dressed… Which was a bit of a surprise, and he chucked me an arm band with another stripe on it, and said “I hear you’ve done good” then “So sort yourself out you look like a sack of 5h1t tied up loosely in the middle”… What god giveth with one hand, he taketh with the other…

Glaurung February 16, 2021 5:48 PM

The specifics of this story now make as little sense as those of the first story two years ago.

Assuming that China has decided to spy by injecting malware into motherboards made in China and sold to the US. Doing it by sticking an extra chip on the board has to be the stupidest imaginable way to go about that. Why go to all the trouble to design a chip that can be detected quite trivially if your target becomes suspicious when for the same amount of effort you can bake your malware directly into the manufacturer’s signed firmware updates?

Or, if you absolutely need your spyware to be permanent even if your target flashes the board with FOSS firmware, why put a chip that’s not supposed to be there on the board when you can create a custom version of a chip that is supposed to be there, thus guaranteeing that your hack will never be detected by physical inspection of the board?

The journalists reporting on this story clearly don’t understand the technical details, and they appear to have talked to a lot of people who also are not technically adept, who attended briefings on security threats, where the technical details of those briefings went over their heads. If after two years of digging they still cannot find someone with direct technical knowledge, I’m inclined to think that they’re on a wild goose chase.

BackInTheRealWorld February 17, 2021 6:10 AM

Whatever snooping China might be doing, the NSA is doing at least 50 times as much.
And the US government is using the data it harvests to kill people. Google for “We kill people based on metadata” – as admitted by a former senior official of NSA and CIA. Or maybe it will just put you on the No-fly list.

The Bloomberg article may be true but it’s just a smokescreen, a distraction. According to Wikipedia (yes, I know, not too reliable, but not bad and the best source I can dig up easily) China has 4 overseas military bases, the US has more than 54.

Grima Squeakersen February 17, 2021 4:51 PM

@Clive re: DoD panties comfort issue – I think you may have missed what to me is another obvious possibility – that it serves some alternative, unstated interest of the US Dod, or of some other agency that can lead DoD around by the nose, to create the appearance of great concern over this (potential) issue.

Clive Robinson February 17, 2021 6:59 PM

@ Grima Squeakersen,

I think you may have missed what to me is another obvious possibility – that it serves some alternative…

There may well be other reasons as well, but lets go through what I said in reply to @Bruce,

Yes they DoD amongst others have been getting their panties in a very tight wad over it for most of this century.

Due to various politicians on the hill, the US Federal Government, State Dept military and intelligence agencies were told to stop developing and using there own systems around the work of US orgsnisations that could be trusted (the I in MIC) and instead use “Consumer Of The Shelf”(COTS) equipment. The excuse was “cost saving” but on reality was thr leading edge of outsourcing government to other organisations that were giving kick backs through lobbyists to those politicians.

The result of this is that the manufacturing of the majority of the components used in these COTS machines were made in the Fat East. For various reasons manufacturing of the higher end components a lot of the DoD frontline and command staff systems ceased to hapen in the US. With further time and for other reasons I’ve warned about before on this blog, the tecunology became under China’s control. Some accused IP theft, but the reality was short term managment thinking gave the Chinese the IP gratis, plain and simple. China unlike the US decided to do as other Far Eastern nations such as Taiwan, Japan, and South Korea had done which is the state invested in building up it’s industry over several decades. Not as the US was doing, giving the work to the Far East in return for a handfull of dollars extra profit in the next quater or two.

There is no doubt about it in quite a few areas of technology China is well in advance of the US especially in communications technology and the semiconductor development involved with it. Which is what the 5G argument is actually all about. The US under the previous administration decided that it had to get back the technology under it’s control. Thus the many lies and half truths to kill off first ZTE and then try to kill off Huawei. The reality is every thing the US accused both Chinese companies of, can be said as equally by China for twenty or thirty US companies. The simple fact is China did tell the US to clean up it’s act, the US refused, China brought in legislation to make the use of equipment by those twenty or thirty US companies. The US executive got upset made bad choice aftervbad choice, and the US economy has suffered badly under the US-China trade War, whilst China has taken some knocks it’s expanded other markets, whilst the US has stagnated…

The DoD however were well aware of the issue of “foreign supply” and how it could be attacked. They were not the only ones, various commenters on this blog had been talking about it for quite some time, because there are examples of things like Apple products being tampered with in the supply chain suchvas Windoes malware on Apple audio players. Also attacks on Epos systems for the Sainsbury’s supermarket chain where plastic cases that had bern welded shut were some how opened and cellphone and Credit Card reading hardware technology added…

Thus the DoD put out “Invitations To Tender”(ITTs) for researchets to come up with ways to spot supply chain poisoning reliably, preferably without requiring destructive testing. The problem is that now semiconductor features are so small they can not be seen by optical or other microscopy… Thus that did tender have kind of “disappeared” which probably means some were successful and others opened newer longer term research.

As for Russia, it’s not realy a secret that they are “Cash poor but resource rich”

Which kind of covers the other parts of my reply to @Bruce ot,

Especially as they have more than good reason to think that China is actually ahead of the US technically[1] and Russia way ahead on natural resources.

As I’ve said you can check that is true in it’s own right.

Thus your point that there may be others, I don’t disagree on we just don’t have sufficient evidence to say, where as we do for what I’ve pointed out.

Thus we need to gather more evidence, even if it does look not even circumstantial.

Leave a comment

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.