Clive RobinsonDecember 15, 2010 6:57 AM

I would encorage everybody to download and learn the tools not just because they can do forensics but because most of them can also be used for other things such as finding things in memory and hard drives that should not be there which many AV tools cannot do and to help put systems back together again.

But on the forensics side the more you know the less chance you have of been "bull541ted" by LEO's and their "guns for hire" who use propriatry tools badly after just one two day training course.

I have sat on the side lines of courts hearing "court recognised Expert Witnesses" for the prosecution troting out a mixture of half truths and down right lies "because that's what the tool and the training taught them".

The sad reality is these people are not experts they are often LEO's etc who are not exactly on the fast track through life trying to carve out a job security niche...

The more you know the more you can employe the process of evoloution on the usless lot.

What would also be nice is jurisdiction by jurisdiction guides to the laws of evidence and civil and criminal procedure rules that are in place there.

The Law of the UK with regards evidence is a compleat and utter mess with bit's tacked on here and there with exceptions and overriders and all sorts of other twists. So much so that most judges and Barristers don't even know the half of it.

You see this all the time with warrants that are issued illegaly and used by various quasi legal organisations effectivly purjuring themselves infront of judges and magistrates but getting away with it because they can bambozal m'learned bretherin then going on to breach UK&EU data protection legislation (The UK's OfCom out of their Birmingham office has been a major offender in this regard).

BF SkinnerDecember 15, 2010 8:09 AM

@Clive "court recognised Expert Witnesses"

This is related to the profile on Kohlmann.
There was the comment on "if his method is sound".

Well what's an expert? Someone who knows what they are talking about.
How can you tell they are an expert? They know more than me.

Kohlmann should be being challenged by the oposition lawyers as to his qualifications and knowledge. But what can a lawyer really know about any experts area? They usually just get the CV and "has testified in many trials of this nature" kinds of ancedotal assurance. While the opposition can try to challange an expert's testimony they really can't try to impeach an expert, can they? They are limited to putting their experts up to testify, to rebutt the other sides expert. So the jury has two sets of conflicting expert opinion. What's needed is an expert cross examining the witnessing expert to reveal those mistatements, lies, distortions, and 'reduction in detail' that technical people use to make complex ideas understandable by executives, lawyers, judges, and their juries.

PhillipDecember 15, 2010 8:47 AM

The Anonymous email tracker -- I did that a few years ago. I had a friend who was receiving menacing emails. We did something similar with the hidden small image to get some information about the mailer.

OlivierDecember 15, 2010 8:55 AM

Scapy (, which is a excellent french network tool, should be added.

RichardDecember 15, 2010 9:57 AM

@Clive "The Law of the UK with regards evidence is a compleat and utter mess"

I fear the laws of evidence are a mess everywhere. In the USA, state rules do not always track federal rules for various reasons, and there is still that pesky problem of "interpretation". What may be set forth in black and white to one judge may appear in shades of gray to another.

More to the point, the rules of evidence determine what information gets to a jury, and as gatekeeper a biased judge (whether consciously or unconsciously) can determine the outcome of a case. There is no cure for this, unfortunately, except by appeal. And even that system is flawed by the overriding interest of the appellate bench to protect its own.

Dirk PraetDecember 15, 2010 10:28 AM

Nice toy collection ! I've been using the FCCU-CD for quite some time and I like it a lot. DEFT is looking very interesting too.

I don't know how this works in the US, but to the best of my knowledge, many more or less civilized countries require quite a rigorous accreditation process before you can be referred to as an expert witness in court. Admittedly, Clive has a point that in spite of that, it is virtually impossible for jurors and judges to know to which extent such a person is for real since most of the time they won't have any clue whatsoever about the subject matter or the persons background. On top of that, expert witnesses - like prosecutor and defense attorney - are just hired guns and thus unlikely to tell anything that will help the opposing side.

IMHO it's up to both parties to vet and rebut each other's arguments, both the legal stuff and expert witness' testimonials. If you can't afford one on top of the stiff fees already charged by your attorney, reach out for help using any available channel. See also "How to vet an Expert" at if you have any doubts about someone's credentials.

BF SkinnerDecember 15, 2010 10:57 AM

Would any of these do code auditing on OpenBSD?

FBI 'planted backdoor' in OpenBSD
Break out the code auditing kit

By John Leyden
Former government contractor Gregory Perry, who helped develop the OpenBSD crypto framework a decade ago, claims that contractors were paid to insert backdoors into OpenBSD's IPSec stack around 10 years ago. Perry recently warned the openBSD's Theo de Raadt of the development, years after the event, via an email that de Raadt has published in the spirit of openness.

MikeDecember 15, 2010 12:36 PM

An expert witness is someone who could be persuaded to come down off the fence for the prosecution/defence, had time in their diary, needed the cash and is known by the peers how well known and for what is always moot.

Gabriel December 15, 2010 7:30 PM

@BF Skinner: I read the version over at Ars but I am skeptical. How would an NDA regarding such a matter expire after ten years? If this did happen, it would be classified and require a FOIA request and review after the classification ended. He would not automatically be able to disclose this even if the period had lapsed. I would certainly think the FBI would not want this known in the open.
I am certainly eager to hear what Bruce has to say about this, perhaps tomorrow.

aikimarkDecember 15, 2010 8:54 PM

Thanks for this post, Bruce. I picked up a little new knowledge about hashing (new/different hash algorithms) in the process.

I had suggested a running multi-hash process to academic detect plagiarism several years ago. Alas, my idea didn't go anywhere.

CarlDecember 15, 2010 11:03 PM

Commenters on Bruce's posts always seem to figure out a way to take the post and use it to slam Law Enforcement, slam the Gov't, slam the legal system, slam anyone in a position of authority.. why?

Clive RobinsonDecember 16, 2010 1:06 AM

@ BF Skinner, Gabriel,

"FBI 'planted backdoor' in OpenBSD"

Just for arguments sake I'm going to play the "cough cough male bovine excrement cough cough" card on Mr Perry's claims.

But also assume that they are taken seriously (they have. to for credability reasons) and another much more indepth code audit takes place on the "giffted" IPSEC stack used on Open BSD.

A couple of questions arise,

1, Will anything be found?
2, Would a real back door be found?

To which the answers are "Yes" and "Probably not".

As you are probably both aware I have mentione three areas of attack that the likes of the NSA et al would be using these days,

1, Protocol issues.
2, Side channel issues.
3, 'Known Plain Text' issues.

There are enough of these that we know that you don't have to try very hard to find bugs in code to excersise them (think SSL for protocol, AES for side channel, MS format docs for known plain text). We also know that even where 'bugs' are known for compatability reasons they are darn difficult to resolve (think Wifi in it's various and still broken but in use forms). So I'm fairly confident that they are going to find quite a few things if the do an indepth right through the IPSEC audit (including finding protocol bugs in IPSEC).

Would they find a specificaly planted 'bug', the answer is 'probably not' for various reasons.

Firstly as I've mentioned several times before there are so many subtal ways of doing it that you are not likley to spot even a very small percentage. Especialy if the fundemental attack vector is unknown to you.

For instance getting access to the key can be done as I've indicated in the past using malloc and friends. A simple example being not using free, but this is fairly well known and memory leak tools will probably pick it up. You can also use malloc and free and then malloc again to get the same block of memory under the new pointer, this is less well known and as it is a correct 'valid usage' of malloc it is not something a tool is going to explicitly flag up though it would not be to hard to write a tool to show real memory reuse within a process.

But importantly there are many further variations that are very subtle and in many ways quite stable due to the use of kernel internals virtual memory manager and the MMU. Such tricks can be worked through automatic paging mechanisms, IPC and buffers etc.

It is the usage of memory paging at the hardware level that gives rise to the three academicaly known classes of "cache" attacks.

As we know "loop unrolling" was the first such attack against AES and demonstrated across the network within a few months of AES getting the stamp of approval. We also know that the NSA certifies AES for some Inline Media Encryptors (IME's) but with the caveat of "only with data at rest"... I've made both these points several times in the past.

The reason AES is susceptible to this is a failure to consider this attack vector during the selection process and this makes it a fundemental weakness just like many protocol errors.

BUT this is just one asspect of "getting the key" you don't have to leak the whole key to get the rest of it with a little work...

And this is the point all encryption systems leak information in one way or another, thus two further questions arise,

3, Can the leak be seen?
4, Is what is leaked usable by an attacker?

Now I can safely say that you can only see what you are looking for, unless your attention is drawn to it in some way. So leak finding is probabilistic in nature (as most things are).

Likewise even if you can see a leak can you recognise what is being leaked is harmfull, and the answer is no you cannot.

For instance black box testing can see 'noise' but you cannot tell if there is a real signal in the noise in some way this is part of the well known "Low probability of detect" radio systems that moved into digital watermarking etc.

If you look at the work of Adam Young and Moti Yung with Cryptovirology they show a way that public key systems can have information hidden in them that can be used to break the private key out. They likewise show how you can use PK systems to secure information leakage.

This means that you could put a backdoor into encryption systems whereby you leak the key but as it is encrypted via PK technology you are the only pearson who can make use of the leak...

Which as a consiquence means if PK is secure you cannot corelate any leaked information back to the key, thus black box testing won't help...

abDecember 16, 2010 2:35 AM

How would an NDA regarding such a matter expire after ten years? If this did happen, it would be classified and require a FOIA request and review after the classification ended. He would not automatically be able to disclose this even if the period had lapsed.


Yea but perhaps he does not care? I mean it is not like the NDA somehow automatically controls his mouth.

And if a backdoor is in OpenBSD, why not in other systems.

Clive RobinsonDecember 16, 2010 2:55 AM

@ Carl,

"Commenters on Bruce's posts always seem to figure out a way to take the post and use it to slam... ...anyone in a position of authority.. why?"

That's an overly obvious question and one most teenagers can answer...

The more authority you have the easier it is to abuse the implicit trust that goes with it.

As has been said many times to be trustworthy you have to be responsible in your actions.

Thus those who are not responsable in their actions would normaly suffer the consiquences via societies justice process.

However as has been repeatedly shown if you have sufficient authority you can game the system not just for you but for others and this gives you further power. Thus you and your friends can evade the justice process.

It's why we say power corupts and absolute power corupts absolutly...

RobertTDecember 16, 2010 4:02 AM

I agree completely, on-line data encryption, is a broken idea and off-line is only marginally better.

The basic problem is that real encryption engines must be physically realizable and ALL physically realized engines leak information about their operation. It is impossible not too leak!

If the programmer actually knows exactly what attack vector is intended then there is a small chance that they can find the associated code.

If they don't know the attack vector the chances of finding it through code review, are exceedingly remote.

What is interesting, however, is that there is a much greater chance of accidentally introducing an error, than there is of finding an intentionally planted side channel compromise. So maybe that is the real intention of this "disclosure"

zootDecember 16, 2010 9:34 AM

@ bf skinnerbox, definition of expert
X mathematic symbol for unknown
spurt a drip under pressure.
unknown drip under pressure.

Leave a comment

Allowed HTML: <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre>

Photo of Bruce Schneier by Per Ervland.

Schneier on Security is a personal website. Opinions expressed are not necessarily those of IBM Resilient.