For a few months, German police tested a face recognition system. Two hundred frequent travelers volunteered to have their faces recorded and three different systems tried to recognize the faces in the crowds of a train station. Results (in German): 60% recognition at best, 30% on average (depending on light and other factors).
Posted on August 2, 2007 at 1:47 PM • 15 Comments
Dave • August 2, 2007 2:31 PM
I'm just going to come right out and say 'Not Ready for Primetime.'
Oditogre • August 2, 2007 2:36 PM
Google translator mangled it pretty badly, but I got the gist enough that it didn't seem to say how many false positives there were. That would be the biggest issue, to me. If they can achieve 30% recognition rate with 0% false positive rate, that could well be a very effective system for catching fugitives, but otherwise, it's just going to be a bad waste of money.
Mark • August 2, 2007 3:28 PM
Spiegel.de reported about this a while ago. The most amusing part in the article was that the manufacturer of the system made some remarks along the lines of 'It works. Others use it. Why do they have to test it and spend tax-payer money on the test in order to use it?'. The answer should be evident by now ...
Jan • August 2, 2007 3:52 PM
I just searched a little and found the official page with the report http://www.bka.de/kriminalwissenschaften/fotofahndung (the thing labeled "Abschlussbericht", also German).
They have pretty detailed descriptions of the setup and everything in there.
The interesting parts come in sections 4.3ff
They normalized the results to a FAR of 0.1% (in their setup this meant about 23 false matches per day), thats where the percent numbers above come from.
The graphcis for match rates of the different systems are "Abbildung 16" (overall) and "Abbildung 17" (by daytime)
The news article does not give too many details. The original report can be downloaded at http://www.bka.de/kriminalwissenschaften/fotofahndung/
Part of the "summary interpretation" (chapter 5) reads (my translation):
"The project 'photo tracing' confirmed the technical usability of biometric face recognition in manhunt scenarios. Not only the manufacturer's algorithms were tested, but entire systems, including the camera set-up.
"The field test showed that external conditions like lighting and fast movement had a significant influence on recognition performance. Without much effort, recognition rates over 60% with a false acceptance rate of 0.1% are achievable.
"If masses of humans can be spread out, and if cooperative behavior can be motivated, e.g., as part of entrance control, it can be expected that a majority of wanted persons can be reliably recognized by a biometric face recognition system.
"A realistic acceptable false acceptance (misrecognition) rate of 0.1% is a manageable value. But it also shows that the final decision of identity must always be made by a human that evaluates positive recognition onscreen. Otherwise, at the Mainz main train station, about 23 persons would have been burdened with additional measures every day due to mismatches."
Wasn't there an article today about more systems using facial recognition? The gist of it seemed to be facial recognition in lieu of passwords.
How about both? How about facial recognition + fingerprint + password? Other than that will be three technologies to annoy the living heck out of me...never had thumb scanners work very well for me.
Stefan Wagner • August 2, 2007 4:18 PM
Here is a german article, made by an IT-magazine:
http://www.heise.de/newsticker/meldung/92543
Here is the Report as pdf:
http://www.bka.de/kriminalwissenschaften/fotofahndung/pdf/fotofahndung_abschlussbericht.pdf
both in german language too.
The acceptance of false-positivs was 0,1%.
The report states, how the true-positive-rate climbs, when more false-positives are accepted.
One of the conclusions doesn't seem to be accurate:
They asked 200 persons who frequently pass the observation point to participate, and about 23 000 persons pass that point per day in total.
The conclusion was: When we accept a fpr of 0.1%, 23 people are further investigated per day.
But this should depend on the number of suspects you're searching for.
If they would have just one searched person, the chance for picking a false poitive would be lower, and it would have increased, if they searched for 2 million faces.
Alan • August 2, 2007 4:52 PM
When the system can pick between identical twins and people who look similar, then I will have a bit more faith in it.
As for using it for access security, what happens when the person gets "defaced"? A black eye or a broken nose would become a denial of service attack.
JakeS • August 2, 2007 5:00 PM
Yawn. Automatic face recognition again. It just doesn't work except in highly controlled conditions, and as this test shows, not well enough even then: with a self-selecting group of peop;e who wanted to be recognised (or didn't mind if they were recognised) it could only manage 60% at best.
The face isn't even a reliable way to identify people, as personal experience shows. On the one hand, people look like each other; on the other hand, people's appearances change, deliberately or fortuitously, enough to confuse a computer program.
Face recognition is one of the things humans can do better than computers, and even we aren't 100%.
aikimark • August 2, 2007 5:55 PM
Maybe they should have created an audible output, playing the Windows 'ding' wave file when a face was recognized...
bringing new meaning to "your face sure rings a bell" :-)
occasional traveller • August 2, 2007 6:07 PM
The Germans already have a good recognition capability. In 1994 I made a triangular trip from Heathrow to Frankfurt (A-M) (a 3 week interval) and to Manchester. When the TV screen announced the gate number for the Frankfurt-Manchester I wentto that desk and tried to show my documents but he woman refused to even look at them, saying "This is for Manchester but you want Heathrow".
I can only think she remembered seeing me arrive 3 weeks earlier (which amazes me given the number of people who must pass throuh and that I keep all my funny squid arms under my clothing when I travel).
curious • August 3, 2007 5:13 AM
Does anybody know what is the search capacity/performance of the system in terms of how many subjects can be registered as search objects for the system to be usable/effective (identification in near real-time)? With 200 search patterns it's probably quite ok, but what about bigger samples?
Ralph • August 3, 2007 6:18 AM
Like most of these new technologies I see very small upside uses and very large downside potential for abuse.
They will be justified by the scenario in which they work - but then used in scenarios where are proven to be unreliable.
Where does this desire to police each other come from???
Christoph Zurnieden • August 3, 2007 7:38 AM
It's a good field experiment, carefully designed, executed and it's
easily repeatable. Despite of the very low budget.
The result of that experiment is, that all of the three tested products have
a poor detection rate which is also heavily dependent on illumination,
position and speed of the test subjects.
The summary says the products work. That irritated me at first but even this
poor detectionrate is still good for on thing: for detecting groups like a
flock of Hooligans. But the high number of false negatives (that number seems
to be carfully hidden which might have reasons beside my paranoia) does not
support a recommendation of any of these products for the reliable detection
of individuals. A trained person in front of a video monitor is better and
probably cheaper too.
This will change in the future of course. The development of 3D face
recognition has already started; the results of an experiment, similar
to this, will be available in 2009.
> How about facial recognition + fingerprint + password?
The technical solutions to measure biometrics have a very poor detection
rate. Let me give you some numbers:
The odds that two different passwords map to the same has is with a keyed
MD5 algorithms 1:2^80
The odds that two different faces/fingerprints map to the same hash is
unknown, the algorithms of almost all of the products is secret.
But even with a detection rate of 99.999% one of 100.000 detections is
wrong (a false negative or a false positive). 700.000.000 passengers took a
flight in the USA last year, that are 7.000 errors. Not much? Well, it's
the detection rate, so every measure counts. Let's be conservative and
assume 3 measures at every airport, that are 6 measures each flight. Persons
without a boarding pass are also allowed at the airport like families,
business partners and so on, let's asume 1 person on average. That sums up
to 700.000.000 x 6 x 2 = 8.400.000.000 detections which results in 84.000
errors. I couldn't find the number of how many individual people are behind
the 700 millions tickets, so I can't calculate the inconvinience factor.
But I digress.
If we take that 1:100.000 ratio as the result of the calculation of the
birthday problem the cardinality of the set of hashes is 10.000.000.000.
That number together with close to 8.000.000.000 people on the world: please
use condoms and die early to keep these nice products working, thank you.
Conclusion: the difference between a password alone and facial recognition +
fingerprint + password is neglectable. Provided that a good password is
used of course!
No, the "what you are" part of "What you know, what you have and what you
are" is not usable yet and will not be usable for a very long time I guess.
This here http://www.networkworld.com/community/node/18031 made my footnails
curl. Even with a detectionrate of 99.999% it's only good for a PIN of not
more the 5 digits.
Oh, and one little note at the end: these products work with security by
obscurity: if you know the algorithm used or merely get in posession of the
product you are instantly able to develop counter measures. Unlicensed SFX
make-up artists will have a nice holiday in northern Cuba?
> Where does this desire to police each other come from?
It's a leftover from ancient times when we strolled in small groups over the wide fields of eastern africa. A sudden disapearing of a member might be a sign of a way to close carnivore, non-members are competitors in the fight for food and partners and so on.
CZ
thangathai • April 29, 2010 11:37 PM
hai sir
i got 100% face recognition rate, the result is correct or not, please tell me ur opinion
Subscribe to comments on this entry
Photo of Bruce Schneier by Per Ervland.
Schneier on Security is a personal website. Opinions expressed are not necessarily those of IBM Resilient.
Leave a comment