GhostBuster

This is a really interesting technical report from Microsoft. It describes a clever prototype—called GhostBuster—they developed for detecting arbitrary persistent and stealthy software, such as rootkits, Trojans, and software keyloggers. It’s a really elegant idea, based on a simple observation: the rootkit must exist on disk to be persistent, but must lie to programs running within the infected OS in order to hide.

Here’s how it works: The user has the GhostBuster program on a CD. He sticks the CD in the drive, and from within the (possibly corrupted) OS, the checker program runs: stopping all other user programs, flushing the caches, and then doing a complete checksum of all files on the disk and a scan of any registry keys that could autostart the system, writing out the results to a file on the hard drive.

Then the user is instructed to press the reset button, the CD boots its own OS, and the scan is repeated. Any differences indicate a rootkit or other stealth software, without the need for knowing what particular rootkits are or the proper checksums for the programs installed on disk.

Simple. Clever. Elegant.

In order to fool GhostBuster, the rootkit must 1) detect that such a checking program is running and either not lie to it or change the output as it’s written to disk (in the limit this becomes the halting problem for the rootkit designer), 2) integrate into the BIOS rather than the OS (tricky, platform specific, and not always possible), or 3) give up on either being persistent or stealthy. Thus this doesn’t eliminate rootkits entirely, but is a pretty mortal blow to persistent rootkits.

Of course, the concept could be adopted for any other operating system as well.

This is a great idea, but there’s a huge problem. GhostBuster is only a research prototype, so you can’t get a copy. And, even worse, Microsoft has no plans to turn it into a commercial tool.

This is too good an idea to abandon. Microsoft, if you’re listening, you should release this tool to the world. Make it public domain. Make it open source, even. It’s a great idea, and you deserve credit for coming up with it.

Any other security companies listening? Make and sell one of these. Anyone out there looking for an open source project? Here’s a really good one.

Note: I have no idea if Microsoft patented this idea. If they did and they don’t release it, shame on them. If they didn’t, good for them.

Tags: , , , , , ,

Posted on February 15, 2005 at 8:00 AM • 38 Comments

Comments

Greg • February 15, 2005 8:43 AM

Flaw: it can’t know in advance all the possible ways that a rootkit can hide as data. Hence it doesn’t know what to scan. e.g. rootkit could be “hidden” in a .png picture, displayed on the desktop.

In fact you could say that the flaw is that it assumes the rootkit will try to “hide”. Instead it may be hosted by an unchanged, but vulnerable application.

Perhaps the point of the technology it to try to prevent rootkits hiding from virus-checkers though, as the latter are already good at spotting data files with known infections.

Israel Torres • February 15, 2005 9:09 AM

One bad thing is that in a production environment all production stops to scan for rootkit(s)… which tactically could be the very reason the rootkit was planted in the first place.

Israel Torres

Nicholas Weaver • February 15, 2005 9:12 AM

A rootkit must hide SOMEPLACE, and the differences must be apparent. Even in a .png, there must be SOME hook somplace to make it start up, and that hook must not be revealed to programs in the running system if the rootkit is to be effectively hidden.

And the good windows rootkits are actually very good at hiding from antivirus systems.

The big issue is the rootkit recognizing the ghostbuster program and treating it differently from all others in the OS, but that could be countered by making the ghostbuster both mutagenic (you clone the disk by an internal function from the safe OS and end up a different copy) and online (download a set of signatures to look for as a way of detecting ‘partial’ rootkits), forcing the rootkit author to do something similar to heuristic metamorphic virus detection (a VERY hard and annoying problem).

Alex • February 15, 2005 9:13 AM

you could easily do the same thing with a knoppix cd, provided you have a piece of software like a ls windows port
Or use bartPE that should work exactly the same as winPE’s software and only takes a few minutes to get a working bootable windows CD

http://www.nu2.nu/pebuilder/

Nicholas Weaver • February 15, 2005 9:19 AM

One more thing. What greg is describing, NOT hiding completely, makes it a partial rootkit rather than a full rootkit. Missing the point of a rootkit: hide in all circumstances from ALL programs in the OS.

Since a partial rootkit is doing camoflage and only some hiding, it is still detectable within the OS. The interesting thing is that partial rootkits can hide from a ghostbuster style programs but NOT from antivirus, while full rootkit programs can hide from antivirus but not from a ghostbuster.

Its not a panacea, but it is a big advantage, as unlike things like running tripwire from a linux bootdisk, you don’t need to know “ground truth” to find the rootkit.

Watching Them, Watching Us • February 15, 2005 9:51 AM

Shouldn’t this “Ghostbuster” at least be like standard anti-virus programmes and check the memory space as well ?

There are several computer viruses which happily skip into a virtual 8086 machine on Intel chips, wait for the unsophisticated antivirus scans to pass them by, and then re-infect. They are are agnostic so far as the actual operating sytstem is being loaded from disk or cd-rom.

Shouldn’t the usage of such a program require full power down rather than a reset ?

Nicholas Weaver • February 15, 2005 10:45 AM

Tripwire requires you knowing what all the binaries SHOULD be when you scan from the bootable CD.

This hack, which you could adopt TO tripwire, allows you to just catch the act of lying itself, without knowing what the ground truth is supposed to be.

Davi Ottenheimer • February 15, 2005 10:49 AM

Yes, open source it and set it free, in a timely fashion…like they have always done on http://www.rootkit.com (http://www.rootkit.com/vault/joanna/klister-0.4.zip).

Here’s an even better simple, clever, elegant idea that can help stop many rootkits: harden the operating system out of the box with Discretionary and System Access Control Lists and include a MD5 snapshot utility.

Patchfinder is one such utility, and it doesn’t even require a reboot, which can be extremely important for a successful investigation. However this, and tripwire, need to be employed early enough to create hashes from a clean set of files. Post-incident, as Alex mentioned, you can use the Knoppix distro or FIRE:
http://www.knoppix-std.org/download.html
http://sourceforge.net/project/showfiles.php?group_id=46038&package_id=56574&release_id=159330

kurt • February 15, 2005 11:11 AM

while ‘fooling’ ghostbuster would be a nice way for the malware designers to deal with ghostbuster, a much more probable approach is to simple not allow it to run….

there’s plenty of malware that already kills anti-virus processes and the average user won’t really know what to think when they pop in their ghostbuster cd and it simply refuses to run…

Stan • February 15, 2005 2:14 PM

Why should MS release this thing? With their new anti-piracy update policy to take place later this year, they should keep the spectre of vulnerability present and future alive and well, as it were.

Jerry • February 15, 2005 2:28 PM

Consider all the files that are updated as windows shuts down (can you say false positives). The registry and user hives, for sure. I think you would have to run GB, flush disks, and then pull power plug before rebooting with GB’s OS to prevent changes once the scan was made.

Also, by analyzing GB’s “checksum” algorithm to see if it has a flaw, a rootkit could hide its own file and replace it with a phantom junk file with the same name and producing the same checksum. A complete secure hash on 100gb of files would take quite a while, in any case.

John Forrister • February 15, 2005 2:40 PM

@Kurt:

Not permitting the utility to run would be a dead giveaway that a system was compromised in most cases – this is actually accounted for (indirectly) on page 3, footnote 1, where the authors state

“The software can also try to “hang??? the “dir??? scanning process so that it never finishes. But that symptom itself would indicate the existence of malware.”

This utility probably wouldn’t be of much use to an average user – but for an admin checking out a misbehaving system, or for forensic purposes, the technique is both valuable, and, as others have pointed out, duplicated fairly easily.

One area that wasn’t addressed in this paper, however, is the use of Alternate Data Streams to hide files, and I don’t believe the methodology documented in this paper accounts for them (although there are other mechanisms which can be used to identify ADS’.)

Morgan • February 15, 2005 2:47 PM

Stan –

Ghostbuster is designed to work against malware that is invisible to the user. Most Windows viruses are far less sophisticated and either run as a visible, though creatively named, process or they replicate via suspicious-looking e-mail. In other words, the types of viruses that lend to the perception of vunerability are not the kind of malware that Ghostbuster would bust, so releasing Ghostbuster would do nothing to curb that perception.

If anything, it would intensify the perception of vunerability, as it would be essentially a corporate admission that a form of malware exists that existing third-party tools do not prevent.

Eli the Bearded • February 15, 2005 4:46 PM

While detecting that Ghostbuster is running does reduce to a halting problem IN GENERAL, in real life it is likely to be the weak point of this process. How many different versions are going to exist after all?

Terry Browning • February 15, 2005 5:00 PM

Pardon me for being thick, but I can’t see this working against a modern root kit.
As I understand it: Ghostbuster scans the disk with the root kit running, then scans again with a clean kernel image; then it looks for differences in the hashes. If there are no differences then Ghostbuster declares the system clean.

Consider a simplified anti-tripwire root kit: The stealthing code sits in the kernel looking for an attempt to open a file (say, /bin/ps). If an attempt is made to open the file for execution, then the load is redirected to another file (perhaps a pr0n .jpg) where the trojaned ps lives. If /bin/ps is only being opened for reading (as tripwire would do) then the load is unaltered and opens the real /bin/ps and everything looks OK.

Ghostbuster would fall prey to the same trick and scan the same actual files both before and after the reboot. The hashes would match and you would be given a false reassurance.

The weak link is assuming that read and execute will be perverted in the same way by the root kit.

The root kit will still need a startup method, which could be found; but Ghostbuster isn’t trying to do that.

kurt • February 15, 2005 5:02 PM

while stopping the ghostbuster process would indeed give away the fact that the system is compromised, it would prevent the operator from gaining any useful information about how it’s been compromised or what to do about it…

Nicholas Weaver • February 15, 2005 5:22 PM

But, terry, the rootkit must load somehow by modifying SOME executable, and hide that presence. There are limited entry points where the rootkit can start (binaries, kernel, registry), and the scan of those areas must be a lie in order to hide from all programs when running in the OS.

John Forrister • February 15, 2005 6:13 PM

Kurt:

Correct, but pretty much all this is intended to do (based on my admittedly quick read) is to return a list of hidden files and their locations. They’re not attempting to identify a particular kit. If the initial scan fails, it would be reasonably certain that something is going on and closer inspection is called for. It’s clearly a limited use tool, but still better than nothing. They did use some readily available kits, as a demonstration, but the intent wasn’t to identify the specific kit, so much as to demonstrate that the process identified the hidden files.

John Forrister • February 15, 2005 6:15 PM

Terry:

You’re correct that they scan the system in both states (suspected compromised and known clean), But instead of comparing hashes or checksums, they’re diff’ing the output from the windows equivalent of an ‘ls -lR’, and flagging files which are only listed in the ls output from the known clean system (as it happens, it also catches differences in file size). I think they’re counting on a virus scanner run from a clean host to catch pr0n.jpg (which, btw, doesn’t count as ‘hidden’ for the purposes of their document).

The document outlines a very specific goal – detection of files present in a file system which are in some fashion completely removed from a directly listing when said listing is pulled while running under the compromised kernel. Of course, the writers of these packages are highly adaptable, and would be move toward your scenario very quickly if this tool were to be released, but that still presents the malware writers with the problem problem of the binaries being found by, eg, virus scanners once said scanners were scanned from a clean boot disk. This really just lets you identify potential ‘hidden’ malware quickly, given the assumptions they’ve made in the document.

Bill Arnold • February 15, 2005 8:48 PM

Before/after system checks are a common technique when analyzing potential computer viruses/worms (typically in a air-gap secured lab, or in a secured virtual environment). When I was doing this sort of thing, we would try to minimize the number of steps between the two checks (typically before and after launching a suspect file) so that there were fewer baseline changes to subtract out of the report. Pressing the reset button to avoid a system shutdown (and the attendant filesystem changes associated with a shutdown) might be novel. I don’t recall using the technique to detect a “stealthed” system compromise, though it may have been used to detect “stealthed” viruses/worms during automated analysis; in automated analysis, we had a baseline that was known to be uncompromised, unlike this system which can discover stealthed compromise with no baseline.

As Bruce pointed out, with this technique there is an arms race between the stealth compromise and the checker running on the live system; the compromise needs to not lie to the checker, which means it needs to recognize the checker or the checker’s behavior, which is (or appears to be at first blush) halting-problem hard.

Vadim Kolontsov • February 16, 2005 2:04 AM

Actually, it’s a very old idea. There is “ADInf” integrity checker, which was developed in Russia since 1991.

Quote: “ADinf also incorporates an algorithm for searching stealth viruses based on their hiding capability. The dodging technique
of a stealth virus is paradoxically the weakest spot in the hiding algorithms and can be led to betray their presence. It is sufficient to compare the size or CRC of an infected file given by DOS and its actual value; any discrepancy between them is a symptom of stealth virus infection”.

(“A Method of Detecting and Eradicating Known and Unknown Viruses” (1993) – see http://adinf.ru/russian/adinf/info/ifip-93.txt).

Sounds familiar, isn’t it..

Mike • February 16, 2005 2:39 AM

Hey, here’s a random idea:

Assuming that the rootkit can’t tamper with the behavior of the scanner program itself (or detect it as being a ‘lie detector’ as opposed to an antivirus scanner), the checksum file may still remain a weak point.

Now, apply some asymmetric crypto to the problem.

The scanner CD has a public key, which it uses to encrypt the hashes.

Then we use a separate “verifyer” CD, which has the private key on it, which it uses to decrypt the hashes.

Now, it shouldn’t be possible to attack the file directly (in an undetectable way, that is) without manipulating the scanner process itself, should it?

berkus • February 16, 2005 6:17 AM

I liked that asymmetric crypto idea, should prevent files from undetected tampering with.

As to the tripwire issue, you can as well use tripwire the way ghostbuster is supposed to work – run tripwire on a running system, then reboot in e.g. knoppix and run tripwire again.. exactly the same effect.

Quadro • February 16, 2005 8:00 PM

While stopping the scan may prevent identification of the type or method of infection, and perhaps also prevent direct elimination of the malicious software, at least it would alert the user that something is wrong. After all, many users only care what they can do about the problem. However, this has been the usual modus operandi of the system administrators I know, some of the most clueless people I have ever encountered, so it might not be a good idea. The safest option remains “format c: /u /y” and reinstall Windows from a clean disk.

In regard to the asymmetric crypto idea, I think there are 2 flaws:
#1, it would be possible to generate a completely new hash, encrypt it with the public key, and replace the true hash.
#2, if there is a CD containing the private key, then what’s to stop malicious users from stealing the private key from that CD? Since it’s on a CD, it can never change. The only solution to this would be to have a unique keypair for each set of CDs, but that would be too expensive.

Euripides • February 17, 2005 1:27 AM

The idea isn’t new. I do the same thing with Knoppix STD and AIDE. And I think, fooling this system from the BIOS would be very hard for an attacker, because Linux makes no use of the BIOS for I/O-Operations. Ok, I use it only on Linux-Systems, for Windows it would be trickier, because the registry is a single large file, one can only detect, if it changed, not if single keys changed (e.g. for the autostart).

Jakub Safar • February 17, 2005 4:22 AM

What about MSWindows updates? Database of files, registry DB would have to be refreshed every time there is a (automatic) update.

ravuya • February 18, 2005 2:38 PM

This tool has existed since the late nineties — tripwire does exactly the same thing. Hell, I think it even has a Windows port.

JB Robertson • February 18, 2005 6:50 PM

hello ? either i’m a security expert (which i’m not) or this simply is an old idea rehashed. i have a couple of script that do just that – you run them on the suspect host, then run your prefered live CD and run again, then compare the results. twenty python line should do, no need for bloatware!

Breno Leitao • February 20, 2005 9:57 AM

Microsoft releasing his research project as open source!? Humm. Improbably.
Let?s improve the Robertson 22 line python code and go on… 😉

[]?s
Leitao

Halvar Flake • February 20, 2005 4:46 PM

Now let’s consider the following rootkit:

1) It keeps all it’s code in an office
document which it found on the harddisk (office documents contain their own file system)

2) It loads itself by overflowing a buffer in a program starts at bootup and that trusts at least one registry key to be of a certain format

Now, all you need for this is
a) One overflow in a program that runs at startup and trusts registry keys (a lot easier than finding an overflow in a networked daemon)
b) An office document on the hard disk

Not only would this construct evade “GhostBuster”, but it would also evade almost every forensic analyst searching for it when mounting the HD from a clean OS.

And I can assure you that the total cost to build this is less than 20k USD 🙂

Cheers,
Halvar Flake
SABRE Security GmbH

Derick Swanepoel • February 27, 2005 7:10 AM

Correct me if I’m wrong, but I see a flaw: Ghostbuster can only compare checksums of files for which it has a database of valid checksums. If the rootkit is hidden in a file that isn’t part of Windows (e.g. some 3rd party service), then Ghostbuster won’t detect it.

How about this rookit: It installs itself as a Windows service, which if I’m not mistaken is analogous to a kernel module. It could even replace the kernel with a rootkitted kernel, and as such it can detect GB. Now since lying to GB would give it away, it replaces the rootkitted kernel with the old valid kernel and stashes the rootkitted kernel away somewhere in an Office document. When the system reboots and GB scans the disk, it finds nothing wrong. When the system boots back, the rootkit service starts up, finds the valid kernel and again replaces it with the rootkitted one.

Quadro • March 4, 2005 12:20 AM

Derick:
No, it does not require a database of valid checksums. The idea is that the checksum obtained under a trusted OS (from the CD) is the checksum of the file as it actually exists on disk and the checksum while running the OS under question is the “valid” one, because the rootkit wants it to be right.

The service rootkit is the same idea others referred to: the problem is strictly that of knowing you’re being watched.

Computer Repair Guy • June 30, 2010 3:26 PM

Though 5y old, it’s a very timely article, as rootkit infections are ever-present and tend to plague Windows clients. Still looking for a perfect tool to fight them off. Thanks.

Leave a comment

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.