Schneier on Security
A blog covering security and security technology.
« Who Should be in Charge of U.S. Cybersecurity? |
| Stealing Commodities »
April 2, 2009
DNA False Positives
A story about a very expensive series of false positives. The German police spent years and millions of dollars tracking a mysterious killer whose DNA had been found at the scenes of six murders. Finally they realized they were tracking a worker at the factory that assembled the prepackaged swabs used for DNA testing.
This story could be used as justification for a massive DNA database. After all, if that factory worker had his or her DNA in the database, the police would have quickly realized what the problem was.
Posted on April 2, 2009 at 2:54 PM
• 50 Comments
To receive these entries once a month by e-mail, sign up for the Crypto-Gram Newsletter.
realized what the problem was, or.... convicted the swab plant employee for 6 serial killings.
So I guess what I saw on CSI isn't true: the forensic investigator's DNA *was* on file, so they *could* exclude themselves. Also, there have occasionally been episode plots where some material left by one of the investigators has confirmed that they were at a crime scene, because their DNA is on file.
Step one: get a job in a medical supply company...
The swabs were apparently guaranteed to be sterile from a medical standpoint but not DNA-free from a forensics standpoint. The supplier was up-front about that but the police didn't pay attention.
...in other news, Nike is partnering with the police who want to register everybody with legs and feet, so that in the future when people run away from the cops, they do not have to chase.
Please report to your nearest Foot Locker immediately.
Citizens, compliance is mandatory.
Report for your new Reality Crappy ID Cards (Real ID).
Fortunately, installation of a massive DNA database hasn't been in discussion here in Germany yet (at least with this case). Instead, after the producer stated that their swab were never intented to be DNA-free, the police assured to make higher demands on the products and producers.
Makes me wonder how many other everyday police tools aren't intented to be police tools.
Forensic "science" often barely deserves to be called science. Most of what is used was developed in the field by police and prosecutors. The U.S. National Research Council put out a report about this recently. Actually, DNA analysis is the only forensic method which has been proven valid. The problem now is that finding out what really is scientifically worthy would risk tossing out huge numbers of convictions.
> This story could be used as justification for a massive DNA database.
Or, in a more sensible universe, making sure the swabs are clean to start with.
Even better is the comment of Klaus Hiller, president of the police of Baden-Württemberg:
"Wir haben eine Frau gesucht. Es war eine Frau. Und wir haben diese Frau gefunden."
"We searched for a woman. It was a woman. And we have found this woman."
The alternative is to have control swabs, which are used on areas close by that aren't expected to have the criminal's DNA. There are ways to get good information from imperfect tools.
Yeah, let's have that DNA database. And after they kick in your door at night they can put you in this machine:
.... after a short water boarding session to get your attention...
So, cotton swabs intended for use in collecting DNA evidence, come pre-contaminated with someone else's DNA, thereby invalidating every DNA test ever performed on evidence gathered with said swabs. That should free a lot of criminals convicted with DNA evidence.
Or the people doing the testing could educate the police about precisely what DNA testing does and does not show. It does not point a magic finger to the killer. Science is a compliment to good police work not a substitute for it.
Noun: sarcasm saa(r)kazum
1. Witty language used to convey insults or scorn
"After all, if that factory worker had his or her DNA in the database, the police would have quickly realized what the problem was."
> ...in other news, Nike is partnering with the police who want to register everybody with legs and feet, so that in the future when people run away from the cops, they do not have to chase.
Actually, I'm half surprised that police don't start requiring shoe soles to have ID numbers put into them so they can better identify footprints...
I mean, they actually did put those yellow dot codes into all of our color printers, not to mention how copiers and some image manipulation software go crazy when they think they see a dollar bill...
So, lets see, we have:
Using materials off spec, leading to contamination.
Absent, or at least inadequate control samples, leading to a failure to detect contamination.
Is it just more, or does this put all DNA evidence gathered by these investigators or processed by their labs into significant doubt?
If they can't detect simple contamination of their equipment, what hope do they have of detecting cross contamination of their samples?
AFAIK, German police already has the DNA of their members on a kind of whitelist, to filter out "false" positives. The ideal setup for the perfect crime...
As Craigh Hughes said, a DNA database would only mean the police would have singled out the worker whose DNA was on the swabs and railroaded her into prison.
A police service that can't buy the right kind of swabs isn't a police service smart enough to consider the possibility of misleading evidence.
Its highly unlikely that any 'criminal' will be set free because of this. The system in Germany runs on different rules. It is even possible to get convicted if the police search your house and find hidden illegal substances and the search warrant was for your neighbours house. There is no fruit of the poisennous tree.
J. Edgar Hoover wanted all Americans to be fingerprinted for a large database. There was a promotional film made of Amelia Earhart submitting her fingerprints for this.
Figure out where your local science lab gets their swabs, and apply for a job as swab-packer there. Deliberately contaminate the swabs you pack, and kill your mother-in-law.
I fully support a worldwide DNA database and camera monitoring too.
Would it not make more sense to do some QA on the swabs. Like maybe do a DNA test on a random sample of swabs in each batch to check for DNA contamination?
But I _like_ my mother-in-law.
Can't I kill the bully that kept beating me up in grade school?
> After all, if that factory worker had his or her DNA in the database, the police would have quickly realized what the problem was.
...Or police would arrest her (wrongly) more quickly!
This story could also be used as a wake-up call for the German police.
It has been pretty clear from the coverage of that story that police were looking for some kind of female drug-addicted international James-Bond-Jason-Bourne-serial-killer.
They had found that DNA on 26 (very different) crimes scenes in Germany and Austria spanning the last 15 years. Crimes varied widely of course, from burglary to safe cracking to murder (longish article in German back from the time when they were still looking for the woman: http://www.zeit.de/2008/18/Die-Unsichtbare).
DNA analysis is a very convenient tool, but German police have relied a bit too much on it, it seems...
Funny thing is, before even considering that the swabs were contaminated at the manufacturer, they circulated the story that the culprit might be a transgender or a transvestite, since at different criminal offenses no witness actually saw a woman. It was so ridiculous. German newspaper DIE ZEIT published the contaminated swabs theory at least a year ago, I always wondered why nothing came out of it, as it was the most logical explanation.
this story of the "uwp phantom" should tell us much about the dna fetishism of our legal system(s). it's ridiculous to value such a long chain of evidence which spans over many organisations as highly as this.
(uwp="unknown female person")
You don't need a massive DNA database to avoid this problem, an elimination database is a much cheaper way of avoiding the problem. According to the UK National DNA Database Annual Reprort 2002-03 the UK has already taken this step:
"Supplier laboratories have established elimination databases to help detect contamination from their staff."
I'm actually surprised you haven't written more about DNA false positives: The above report also states:
"The probability of a match between two full SGM Plus(TM) DNA profiles from unrelated individuals is less than 1 in 1 billion and the match probability between two SGM profiles is about 1 in 50 million. The probability of a match increases if the profiles are partial."
see: http://www.homeoffice.gov.uk/documents/... (I'm afraid I cannot find a later version of the report.)
(SGM uses 12 markers and SGM Plus uses 20 markers. SGM Plus was introduced in 1999 and as of March 2003 over 70% of profiles were SGM Plus profiles.)
However even with SGM Plus the probability of a match is 1 in 1 billion. Thus SGM Plus is effectively a 30 bit hash (2^30 ~ 1 billion) of an individual's DNA. Hardly adequate to protect against false positives.
This leads me to the UK football stadium birthday DNA "paradox". Consider a UK football game: there are 22 players and one referee on the pitch. There are about 35,000 fans in the stadium. Thus there is a greater than 50% chance that two people on the pitch have the same birthday and a greater than 50% chance that two people in the stadium have the same DNA profile.
> This story could be used as justification for a massive DNA database.
Well, it could also to the contrary serve as an argument against a massive DNA database. It illustrates nicely the fact that innnocent people can have their DNA found at crime scenes.
If her DNA had been on file, she could have been found (and arrested?) after the first crime.
Just to set the record straight:
The manufacturer did erroneously guarantee some charges of those swabs to be DNA free. They confirmed this, after they first denied it.
And the police did have several types control swabs. (No explanation as to why that did not prevent the error, though.)
In the end this boils down to a fiscal problem.
There are DNA clean sticks but it was stated that they are 14 times more expensive than the ones that were used.
By the way: the part of germany (Bundesland) where this happend istfamous for being more sparingly than the Scots :)
This article from the Register notes that there are almost half-a-million (pairs of) people on the UK DNA database with identical profiles.
Actally neither the reporter nor the minister seem to realise the meaning of this figure as they talk about "removing" them. However given my understanding of UK procedure and the DNA matchng rules (admittedly unsure) and some arithmetic (this bit is more likely to be correct), my calculations suggest that the expected figure of duplicates would indeed be about 530,000 - if three people A, B and C all have the same profile, then this is counted as 3 duplicates (AB, AC and BC)
Please rip this apart if I am wrong:-
I understand that a profile match is obtained as follows. A new DNA sample is collected and examined at thirteen "likely" spots (alleles) for hits. Because the collected DNA may be damaged or degraded, nine (or more) such hits are considered enough for a profile match. The probability of a hit at random at any one spot is about 3/40 or 7.5%. The probabality of a profile match (assuming binomial distribution) at random is therefore 0.000 000 040 or the "better than one in a billion" as stated.
So with 5,140,000 on the register, the chance of you being matched at random is 0.000000040; however the expected number of duplicate profiles (number of profile pairs times probability of match) is (exact arithmetic) 536,599 which, when taken from 5,140,000 gives as near as you like the 4,460,000 from the Home Office. These are not duplicate entries, these are half a million different people with the same DNA profile according to the database.
This phenomena was first noticed in Maryland when an investigating officer found a black man and a white man with the same profile. He later found about two hundred more matches on the (small) Maryland DNA database but was not allowed to investigate further.
@Martin and Mortal's chiefest enemy
DNA is far less simple than "probability of a match " or 3/40 chance of a match at a loci.
First different loci have different numbers of alleles. So some may have 2 other 20. And second it depends on race. Your ethnic background can make a huge difference.
Finally we are in fact all related.
This may sound stupid but its a fact at it totally changes the statistics. First the frequency of alleles can vary wildly. ie some my have a 90% prevalence in a population, then a match is hardly the same as a match with a 1 in 10 prevalence. This is all well know stuff from population genetics.
Assuming "randomness" is a approximation only. The true probability of a match of 2 randomly drawn individuals will be vastly lower.
Last but not least. DNA fingerprinting and old fashion fingerprinting for that matter are not designed to work with nation wide database. They have been designed to work with a small list of suspects. So when a cop suggested a DNA database, its another case of the wrong tool for the wrong job.
For a recent historic example, look at bullet metallurgical forensics recently debunked in the US. (wrong tool, wrong job)
NPR had a really interesting interview with Barry A.J. Fisher, Crime Laboratory Director at Los Angeles Sheriff's Department, and Constantine Gatsonis, Co-Chair on the Committee on Identifying the Needs of the Forensic Science Community at the National Academy of Sciences. They discussed a report released by the National Academy of Sciences which called for the creation of an independent government agency to research and certify forensic techniques.
Man, I love Science Friday!
What else were the swabs contaminated with? I hope no gun owners work at the factory.
>Would it not make more sense to do
>some QA on the swabs. Like maybe do a
> DNA test on a random sample of swabs
>in each batch to check for DNA
They did test with other swabs of the same batch but the contaminated one were rare enough that none of them were used in the test and some of them were used in the real procedure.
"This story could be used as justification for a massive DNA database."
I usually enjoy reading your blog and your clever comments. But I don't believe this is very well thought through. I wouldn't want to be in the workers place when the cops come busting in, very certain they are arresting a murderer... As GermanGuy already said, this shows the dangers of this kind of databases.
Dangerous statement to make even including that word "could", and even in jest, given the way any statement will be spun these days.
I've seen at least three posters taking Bruce to task over his sarcastic comment about a massive DNA database being an answer to the problem. Some of you folks need to get out more. It was clearly a joke.
"The problem now is that finding out what really is scientifically worthy would risk tossing out huge numbers of convictions."
You've got that backwards. NOT finding out what really is scientifically worthy would leave a lot of wrongly-convicted people in jail.
True, but the idea that there may be a significant number of people who are in jail for a crime they didn't commit is deeply disturbing for some people, to the degree that they would prefer to bury their head in the sand and ignore it rather than admit that there is a problem. Unfortunately, I expect that many of the people who would be in in a position to make amends (politicians, senior LEO's, etc) have exactly this attitude.
After a good night sleep and some more reading, I agree. Bruces statement was propably not to be taken seriously. But the statement taken out of context is not so fun anymore.
/Fredrik, who needs to get out more and get better at the subtleties of English.
It's funny we talk about the "science of" with regard to crime and it's investigation as though the words lend credability to the process...
The simple fact is that the basic premise of this "lofty ideal" is "The Lockhart principle", which simply states that,
"Wherever there is contact between two objects, there is a transfer of material between them"
Thus the usual assumption that "contact evidence" shows you where at the crime.
What is not talked about is that it also implicitly says that all investigative methods are known to be contaminated, and thus are prone to be unreliable...
In most forms of science there is the accepted "signal to noise" idea, that is the smaller the signal the more difficult it is to make a determanistic measurment (I know somebody will think Heisengburg but that is not what I'm talking about).
It must be obvious to any that actualy care to think about it, there must be a cross over point between the noise and any signal you are looking for, where you are deluding yourself if you go any further (the same as seeing meaningfull shapes in clouds or the "static" on a TV that is not tuned in).
Sadly as has been witnessed on many occasions even "real science" suffers not just from this "self delusion" but due to "vested interests" (reputation and money being just a two of a long list) it actualy encorages it...
If you accept that and then further think about the well reported methods used to coerce people (plea bargining, reducing sentance etc) you can only start to question if any conviction is safe...
The principle of law in Britain and other parts of the world (sadly not Europe in general) is "balance of probability".
This gives rise to the idea of circumstantial evidence by people that do not understand in any way how to deal with probability (and lets face it if you read enough science papers you begin to think how true this is of science as well).
Just as a point,
In an area you frequent a woman who you think you vaguly recognise is found dead with her throat slashed.
You get picked up and told your DNA and fingerprints where found on the murder weapon.
What do you do when offered a plea bargin or being "fried in the chair".
Now think how you would feel if you find out the weapon is a broken beer glass and that your DNA and fingerprints where found on the unbroken base of the glass...
Then when you think how a bar works when washing and storing glasses.
In a lot of bars they have a machine that you turn the glass upside down and push down between rotating brushes. You then pull it out and whilst still wet it is put on a beer cloth to dry. Because the bar staff do not want to stick their fingers in the machine the outside base of the glass rarely gets cleaned.
If it's towards the end of the evening or a quite period after a rush the glass gets put back on the shelf . The next time the glass gets taken off the shelf to be used is dependent on how busy the bar is from then on.
If you happen to be a visitor to the bar who was only there to celibrate a persons birthday or New Years eve etc then the glass you used which has not had it's base washed could sit on that shelf right at the back and not get used for months...
Now ask yourself if you would belive that as a jury member if presented by the defense?
It is why nearly all convictions on circumstantial evidence are effectivly unsafe but nobody likes to talk about it because it is a Pandora's Box.
In times past most criminals where caught due to their "locality" to the crime and that they where stupid. Even now a very large number of criminals are caught because the do silly things like hide the items they have stolen in their home, post videos of themselves driving a stolen car on UTube or boast about what they have done to others...
And it is this "stupidity" in the majority of cases that realy convicts them not a DNA database, DNA evidence, fingerprints or shoe prints etc, it is realy just a "side show" to convince those who stand in judgment that "justice is being seen to be done"...
However the cost of the "side show" is high but required to counteract the likes of "CSI", "soap operas" etc which the "ordinary person" who might sit on a jury watches.
It is why having the right "legal team" is such an advantage they can "play the jury" to gain "credability advantage".
Sadly that is why (in the UK and US) the system is being changed by those in charge to reduce access to juries and good legal talent...
If this woman's DNA had been on file, her life would have been turned upside down by a baseless investigation. Quite probably she'd have even been convicted.
This is an excellent argument against any DNA database, and for using DNA comparisons only to exonerate people.
To just correct Clive Robinson above in case anyone should want to learn more about "The Lockhart principle" it is in fact Locard's Principle http://en.wikipedia.org/wiki/...
I feel that DNA evidence is better used as a tool during an investigation to eliminate suspects who when tested do not match the profile rather than as evidence used for conviction.
For me, this story illustrates that there is a difference between a science (such as DNA) and the use of that science (police investigating a crime) often resulting in a knowledge gap. Technology will continue to become a larger part of our lives - the rate of technology change is ever increasing. As this happens, there is the potential for the knowledge gap to widen.
A company might buy the best firewall on the market, but if it’s installed by someone from the mail room, or even just a less-experienced IT worker, it probably won’t work as intended. In this case, it seems clear the police needed more training in the science they were using – that’s where I think the focus should be. Something like a national DNA database may have helped in this case, but it still wouldn’t address the underlying problem.
> This story could be used as justification for a massive DNA database.
Or just a DNA database of people who handle DNA samples.
> This story could be used as justification for a massive DNA database.
Or, only create a database for biometric companies' employees. If they make money creating technologies to monitor everybody else, they should be more scrutinized than anybody else.
Schneier.com is a personal website. Opinions expressed are not necessarily those of Co3 Systems, Inc..