The Eavesdropping System in Your Computer
The BMC is an embedded computer found on most server motherboards made in the last 10 or 15 years. Often running Linux, the BMC’s CPU, memory, storage, and network run independently. It runs Intel’s IPMI out-of-band systems management protocol alongside network services (web, telnet, VNC, SMTP, etc.) to help manage, debug, monitor, reboot, and roll out servers, virtual systems, and supercomputers. Vendors frequently add features and rebrand OEM’d BMCs: Dell has iDRAC, Hewlett Packard iLO, IBM calls theirs IMM2, etc. It is popular because it helps raise efficiency and lower costs associated with availability, personnel, scaling, power, cooling, and more.
To do its magic, the BMC has near complete control over the server’s hardware: the IPMI specification says that it can have “full access to system memory and I/O space.” Designed to operate when the bits hit the fan, it continues to run even if the server is powered down. Activity on the BMC is essentially invisible unless you have a good hardware hacker on your side or have cracked root on the embedded operating system.
What’s the problem?
Servers are usually managed in large groups, which may have thousands or even hundreds of thousands of computers. Each group typically has one or two reusable and closely guarded passwords; if you know the password, you control all the servers in the group. Passwords can remain unchanged for a long time—often years—not only because it is very difficult to manage or modify, but also due to the near impossibility of auditing or verifying change. And due to the spec, the password is stored in clear text on the BMC.
IPMI network traffic is usually restricted to a VLAN or management network, but if an attacker has management access to a server she’ll be able to communicate to its BMC and possibly unprotected private networks. If the BMC itself is compromised, it is possible to recover the IPMI password as well. In that bleak event all bets and gloves are off.
BMC vulnerabilities are difficult to manage since they are so low level and vendor pervasive. At times, problems originate in the OEM firmware, not the server vendor, adding uncertainty as to what is actually at risk. You can’t apply fixes yourself since BMCs will only run signed and proprietary flash images. I found an undocumented way of gaining root shell access on a major vendor’s BMC and another giving out-of-the box root shell via SSH. Who knows what’s on other BMCs, and who is putting what where? I’ll note that most BMCs are designed or manufactured in China.
Basically, it’s a perfect spying platform. You can’t control it. You can’t patch it. It can completely control your computer’s hardware and software. And its purpose is remote monitoring.
At the very least, we need to be able to look into these devices and see what’s running on them.
I’m amazed we haven’t seen any talk about this before now.
EDITED TO ADD (1/31): Correction—these chips are on server motherboards, not on PCs or other consumer devices.