Schneier on Security
A blog covering security and security technology.
« News from the Fingerprint Biometrics World |
| Interview with Me on Cloud Security »
May 29, 2009
No Smiling in Driver's License Photographs
In other biometric news, four states have banned smiling in driver's license photographs.
The serious poses are urged by DMVs that have installed high-tech software that compares a new license photo with others that have already been shot. When a new photo seems to match an existing one, the software sends alarms that someone may be trying to assume another driver's identity.
But there's a wrinkle in the technology: a person's grin. Face-recognition software can fail to match two photos of the same person if facial expressions differ in each photo, says Carnegie Mellon University robotics professor Takeo Kanade.
Posted on May 29, 2009 at 11:19 AM
• 68 Comments
To receive these entries once a month by e-mail, sign up for the Crypto-Gram Newsletter.
What if their last picture had them smiling?
Who buys this crap?
This here is a nice datapoint on the fragility of facial-recognition technology.
It's the same with British Passport photo's.
There are so many don'ts that I recon only a "photo dummy" would pass.
However when I renewed mine the lady looked at me looked at my old passport photo then the new one and just nodded acceptance...
What I don't like is all the noticies that say if anything is wrong with your new passport then it's down to the holder to pay for a new one...
Seeing how badly the RFID and it's coil are inserted, I have grave doubts about it's ability to survive for ten years of "2 week vacation" use let alone occasional business. Carrying it around in your pocket full time it's not going to survive.
Guess the terrorists will now know how to defeat any facial recognition software -- just smile brightly! Of course, this could lead to some interesting new "terrorist profiling" scenarios...
Hey, Bruce -- I sniff the beginnings of a movie-plot... good dental hygiene leads to nice teeth and a bright smile which leads to terrorism! So if we get rid of dental plans, everyone will have bad teeth, stop smiling, and thereby bring an end to all terrorism!
The report I heard about with the Virginia DMV was that the photo application itself would look at the image and decide if it was right. If the person was looking too human another picture had to be taken. ...hmmmm
So replicants, androids and pod people will have fewer problems getting their credentials than real people and therefore get a social advantage pushin real people to the fringes of the city or deep into the hollar's of WVa.
Tsk Tsk. Time to practice welcoming our new simulant overlords.
No smiling for canadian passports either was implemented a few years ago. My children's passport photo look they never had fun in their entire lives to say the least.
The world is ending. How does this (and so many other idiotic ideas) ever even make it off of the drawing board?
I lose more and more faith in humanity with every passing day. I should start reading a different blog, haha!
So only one of set of identical twins can get a driver's license?
Amusingly, I'd hazard a guess that iPhoto's new facial recognition software is wildly more accurate than what these particular DMV offices use...
Time to get somebody like Jill Greenberg to take your kids' passport photos.
Here in IN, I got my OL renewed and they said no smiling. So of course I start thinking about how frail the facial recog thing is and I start laughing. Try not to smile while laughing.
Smiling is a no no, but letting my left eyelid droop as if I had nerve damage was fine.
How widespread, exactly, is the phony driver's license problem this is trying to reduce?
Same for US passports--a "neutral expression", if I remember correctly.
Great, so now driver's license pictures will look even more like police mug shots than they did already. Wonderful. I so enjoy being treated like a criminal. Don't you?
If it can't handle a smile, what the heck happens if I grow a beard?
None of this is idiotic. FR technology does work - it's not supposed to single our a person and completely, positively identify them - it cannot.
But it CAN compare an image against a large number of other images in a database and display the most likely matches for a person to compare.
By enforcing a particular facial expression the matching algorithm gives better results, but smiling or wearing glasses isn't going to necessarily thwart it - just make it less accurate, and possibly saddle the operator with a larger set of images to compare.
It is amusing that people call things they know nothing about idiotic; it isn't about catching terrorists, it is about catching people getting documents fraudulently - and whether you would like to believe it or not, or like it or not, it has worked well.
It does kinda make sense. When you get stopped for speeding or whatever, are you likely to be smiling? Okay, maybe if it's the bar on your 21st birthday, but . . . . ;-)
SomeGuy: "It has worked well".
Cite me something.
By trying to eliminate the human element in security systems with computerized processes, we are becoming less secure rather than more secure. Eventually, the Luddite Effect will take over and people will rebel against this crud or anything else that is computer controlled or influenced. I'm not saying that is necessarily a "good thing", but I think it may be inevitable as people become more and more frustrated, disenfranchised, and repressed by the effects of modern technology and governmental systems.
SomeGuy: And once you have given up all your freedom and dignity in the name of what "works well", you will find that you are living in a police state and, being a fool, you will wonder how you got there.
Doesn't seem like this should matter too much to the DMV. They have a small set of prior photos (in which you're probably smiling, as Tangerine Blue noted). And, if the software can't deal with your smile, a human can verify the match in just a few seconds.
Where this kind of standardization would matter would be using security cameras out in the wild. If the database contains faces that are uniformly grim, the facial-recognition software can better determine how those faces differ from each other in structure.
@Craig, agreed. One by one they are taking away small freedoms in the name of catching that one in a million "criminal". Shit is going to happen one way or another, dont let fear take away the good things in life.
Well, as long as they don't ban having a "look of indescribable horror", then we're okay.
You just need to use the same expression whenever confronted by idiot officaldom. Well, perhaps "bemused contempt" would work better.
OMFG! How could we have missed it??! It's obvious!
It's a movie plot threat!
That's how you stop the nefarious scheme by preventing the perp from getting and ID. The movie? Batman. The criminal? The Joker
It all makes sense now.
This seems like an example of making people change to fit some technology. As a software developer, I can relate several stories where this is a recipe for project failure.
I would think there is a trivial solution that would satisfy both the general public and the states' DMVs -- take two pictures. One picture with a smile and one picture that conforms to the facial recognition software limitations. Store the non-smiling photo and print the smiling photo on the license.
It's a real shame that photography has gotten to be so expensive that they can't take multiple shots with different facial expressions, and even let you pick the one you prefer to actually slap on your license. Curse you, expensive digital photography!
No, wait, the other thing: the DMV sucks.
@Craig: Which freedom and which part of dignity has been given up here?
A Driver's License if a privilege, not a right, and you get to obey the rules or no DL. That is the deal. Talk about making a mountain out of a molehill.
@kangaroo: FOIA it; this is the only answer I can give you unfortunately.
@aikimark: There is a storage issue associated with images - and with the storage issue goes the speed issue, the backup issue, etc etc. It is a nice idea that you have there though.
When I needed pictures to renew my passport last year, the photographer admonished me that new Homeland Security regulations require a "neutral expression" and specifically forbid smiling. I thought a moment, and then struck a pose that I thought would resemble what I'd expect to look like after enduring an international flight in steerage, followed by two hours in a queue waiting for Customs. The result looked like I was in grimacing in horrible pain, which the photographer pronounced "perfect."
I wonder whether the next step is to prohibit men's facial hair, which surely must confuse facial recognition software. Then they'll require shaved heads to avoid confusion with hair styles, followed by a standard projected grid. If the software can't recognize a normal human face, then the security bureaucrats will just have to make rules forcing us to conform our faces to the limits of the software.
I doubt that the FR software requires a high-resolution image, and know that most digital license photo cameras are old enough to predate what is considered 'high resolution'. I think the DMV or software vendor folks are being lazy in their implementation.
Thank you for your support of my idea. (generally)
On the photos for the new German passports which, courtesy of US Government requirements, contain an RFID chip with biometric information, you are not allowed to smile and you're required to look straight into the camera.
The photos are digitized (image plus fingerprints are put on the chip) and a program checks whether the image is acceptable. If it is not, you go back to the photographer. In the beginning, this caused a big mess because a large part was rejected. Now things are working better and most photographers have the official program installed so they can verify right away that their work will not be rejected.
Other Guy: "By enforcing a particular facial expression the matching algorithm gives better results, but smiling or wearing glasses isn't going to necessarily thwart it - just make it less accurate, and possibly saddle the operator with a larger set of images to compare."
I guess if you define "larger set of images" as "up to all the images," you can say it isn't going to thwart it. Of course, that's still just an assertion. I can likewise assert that it probably will be pretty darned well thwarted. In the absence of evidence either way, I'm betting snake oil.
> FOIA it; this is the only answer I can give you unfortunately.
You could reword the answer. How about:
"I think I heard it somewhere, or perhaps I made it up, but in any case I have no particular facts to base it on. And for calling my bluff, I'll taunt you. I dare you to spend the next three years of your life invoking a law that probably doesn't even cover state governments, in a quixotic quest to get the state to tell you things they don't want to reveal. I think that, like me, you'll find it simpler to just believe the claims I fabricated."
** Schneier for Cybersecurity Czar! **
link in my "Posted by" to state farm ins ad "sixteen" with a bunch of kids making faces for their new driver's licenses.
Is there some reliable biometric ID method? Retna scan? Why not use that?
...and just why is it that photos are required on drivers-licenses at all (??)
Do brain-surgeons, plumbers, and barbers need to carry photo ID licenses when they are working ?
@aikimark: The higher the resolution, the better (up to a point where you get diminishing returns)
@Tangerine Blue: What I have said so far I know that I can say. I can't release to you information that I don't know is releasable since I have a clearance to worry about. Otherwise I'd gladly provide statistics - but those are the property of services that use FR, so again FOIA - or perhaps some of them are willing to release the information. Accusing me of fabrication is pretty disingenuous - much like sticking your own head in the sand ;)
I live where "it's against the law" to smile for your DL, so stuck a bunch of cotton balls in my cheeks, used some latex makeup from last Halloween and acted like my jaw was wired shut. I just talked with my mouth closed when I needed to read the letters back. The latex gave me a smirk and the cotton balls helped there too.
I did it because F*ck Them. I can smile when I want to smile.
This isn't going to stop anyone from creating a fake ID and using it, it's only going to make it look like it will stop the states from issuing ID for "id thieves". Please, it just makes them change the tactics, get someone on the inside, just like before the software came into place, game set match.
Waste of my money and my time and it makes going to the DMV just that much worse.
@sp: If it can't handle a smile, what the heck happens if I grow a beard?
Well, Governor Palin, you just go ahead and give it a try. Let me know how it works out.
> Accusing me of fabrication is pretty disingenuous
You are right; I have no evidence that you made it up, and it was hypocritical of me to make claims I couldn't support.
If you in fact have verifiable proof that the ratio of (true positives) to (false positives plus misses) is better than roulette odds, you have my sincere apology.
But when I weigh [Some Guy knows it works] against the public acknowledgment [the system doesn't recognize faces that have facial expressions], I guess I'm still skeptical.
I don't see a need to store the high res image once the biometric data has been gathered.
Just had my PA driver's license renewed. I was specifically told I could smile if I wanted too. Flip flop. Flop flip.
@Snarki, child of Loki
Yeah twoface is never gonna get his license either.
@Shane: "How does this (and so many other idiotic ideas) ever even make it off of the drawing board?"
Easy. It's simpler - and cheaper - to have people conform to the system and its idealised version of 'people', than it is to have the system conform to people. I suppose, in a way, that this is another example of an externality: by going this route, the vendor has shifted a larger burden of the cost of ensuring that people and the system mesh onto the people. People who, incidentally, had no say in the system being implemented in the first place, but that's another story.
This article is just incomplete without a link to the great "Validation" video: http://www.youtube.com/watch?v=Cbk980jV7Ao - a short romantic comedy where (not) smiling on driver's license pictures plays an important role.
@ Some Guy,
"A Driver's License if a privilege, not a right, and you get to obey the rules or no DL. That is the deal."
I'm not sure that having a "driving license" is a privilege any more but a necesity for participating in society in many areas.
Back in the 1980's the UK Prime Minister Margret Thatcher made what at the time sounded like an "off the wall" sound byte about a man in his twenties could consider himself a failure if he was traveling on a bus.
Here we are a generation later and in most parts of the UK outside of major towns there effectivly is no reliable or cost effective public transport, and little or no local services.
Effectivly being able to drive and having access to a car etc is virtually an essential part of being able to live in the modern urban and rural worlds.
I would like it to be otherwise but for hundreds of millions of people in the EU and US the reality is they need their own motorised transport, to get access to the essentials of life such as food, healthcare education and work.
I can only see these problems getting worse with time so at some point (that may well have passed) a driver's license will be a requirment to live not a privaledge, and therefore a right that should not be denied a citizen with out good cause (and a fair trial).
the funny thing is that you can submit with your passport application passport photos that you've taken yourself (e.g. with http://www.ePassportPhoto.com) so no one in a sense enforces this requirement as long as it does not look like a very shiny smile.
@aikimark: Yes, you do - you have to be able to prove you generated a given FR template from a given image.
@Tangerine Blue: http://www.nbc15.com/news/headlines/2684991.html
I was actually looking to find you a link to research done on different biometric algorithms - but unfortunately I forgot the name of the paper. If you are actually interested, I will try to find it later. In any case, there's no public acknowledgment that the system doesn't work if you smile - no one who works with the stuff will tell you this. It can reduce the match probability, but not a whole lot. The algorithms are reasonably tolerant ... they have to be able to match pictures taken ten years apart in some cases, and they do so well. They have been tested with pretty poor quality images and all sorts of expressions, head tilting, fake mustaches and glasses and whatnot.
Concerning fraud: Consider that a fraud case investigation and litigation can cost fifty thousand dollars (I'm not saying it costs that much, it likely costs more), and that a DMV can actually use FR to prevent this fraud (basically, identity theft) to a large degree ... and you see why they want to use FR, at least from a financial angle.
@Clive: You can think it's a necessity, but it's a privilege, period. You have to prove you're worthy of it and it can be taken away for a number of reasons. Likewise, it is a privilege to have electricity - you pay for it. What happens if you don't? Forget about turning on that TV or night light.
Clive, I seriously doubt we will ever gain a right to something now regulated as a privilege, barring revolution. The ability to deny someone the freedom of easy movement extra-judicially is far to valuable a social control tool.
Still, it would seem that you should be dour in your official photo but take on a smiling happy persona in life, especially when dealing with officials. That way they get their photo, you have a pleasant life (as acting happy will make you feel happy) and photo matching you is problematic.
@ Some guy,
"Likewise, it is a privilege to have electricity - you pay for it. What happens if you don't? Forget about turning on that TV or night light."
A very poor counter argument as you are not comparing apples with apples.
You do not require a license to use electricity you only need the supply and the ability to pay.
So I could counter your argument is about being able to ride in a taxi cab, not driving it or any other mode of transport that requires you to be licensed to operate it.
The argument if you like is about a governments right to control a sociatal norm.
In most places in the US driving is a sociatal norm.
I belive it is taught in US high schools as part of the core curiculum to sixteen year olds which kind of makes it the same as mathmatics, reading and writing pretty much something you have to do to be part of society (and I think it is fairly obvious as to what happens to those who cannot read or write).
And that is realy the point. If driving is so important that the state includes it as part of the core curiculum to teens, then the state must reason it is a core requirment for all citizens otherwise in a cost sensitive system it would not be part of the core.
You could argue that the US Gov rates the ability to drive more importantly than all the other forms of education. In that it teaches it to a standard where you should be able to obtaine a "license to operate" they do not do that for any other activity.
I would go on about other "license to operate" the US Government has (guns, radio transmitters, etc) that are arguably not vocational, but it would likly cause the ire of the moderator.
In essence the fact that the US Gov has made driving available to all via the education system and highways it has made it an essential to life. By not putting the resources into alternative methods of transportation outside of cities it has worsened the problem.
There are various papers etc out on the net about the American Addiction to Autos, most are from the environmental issue and most condem the system for the dependance it has forced on the majority of Americans.
I would say that the "part of core education" and "auto addiction" argues strongly that it is very much a part of American life to the point of being a core requirment for participating fully as a US Citizen...
"Still, it would seem that you should be dour in your official photo but take on a smiling happy persona in life, especially when dealing with officials. That way they get their photo, you have a pleasant life (as acting happy will make you feel happy) and photo matching you is problematic."
It was some of my ancesters and their fellow country men that came up with the word "dour" and those still north of the boarder get cursed with it as a reputation along with one for meanness as well 8(
And unfortunatly Gordon "laughing jack" Brown PM just reinforces the image along with his side kick Alistair "tuffty" Darling MP Minister to the Tresury 8(
However with regard to "take on a smiling happy persona in life" although good advise it needs a caution of "in moderation" otherwise some nice gentalmen in white coats come and pop you into a nice new jacket that appears to fasten the wrong way around...
On a more serious note I suspect the problem with smiling is more to do with that akin to a manic rictus than an ordinary plesant or polite smile.
Ordinary smiling only changes the face structure moderatly below the cheek bones. The last time I looked at full face FR it looked for points that where close to the skull or did not change much, most of which are at or above the cheek bones.
Which makes me wonder what if you pulled a Dr Spock raised eyebrow and tried putting the tip of your tounge behind the lower back tooth on the opposit side to the raised eyebrow, all whilst not smiling.
This guy on Brontitall asked "What's wrong with your foot soldier"
I told him "His feet are the wrong size for his shoes."
@Clive Robinson, Some Guy: Driving is technically a privilege in the US. But its almost impossible to get by anywhere but the largest cities without it (and ironically, almost impossible to do there). I visited the UK a couple of times (enjoyed it immensely) and thought it was easier to get by on public transport there; the only way you can even ride on a train in the US is either in an amusement park or by being in a cargo container (terrorists take note - nvm Im sure you already know) coming from China.
Anything invented after ~1860 will be a "privilege" vice "right" regardless of how impossible human life might be without it because the government (plural actually - since other than on an indian reservation you are almost always under the thumbs of at least 3-5 layers of government in the US; all of whom can take your money and tell you what to do - sometimes contradictorily) would not give up its ability to control its vassals and if some people die for lack of (whatever it is), well sad for them, but control is more important in a government-dominated country than human life is. And I cant wait until health care is added to the list in the form of the US version of the National Health; which should be happening any day now.
>"A Driver's License if a privilege, not a right, and you get to obey the rules or no DL. That is the deal."
No. A driving license is an identity token (who are you?)that has mistakenly been merged with an authorization token (what are you allowed to do?). You'll find the rules are the same for an ID card issued by the DMV, and you'll usually get arrested if you get stopped by the police for lacking ID as Mr. Hiibel discovered. 542 US 177 if you want to look it up.
Last I checked the DMV went through hell and high water to make sure you were the person they were giving an ID to (birth records and the like). I'm interested to know how often this system actually does what it is purposed to do and catch people trying to 'cheat the system' and assume someone else's DL identity.
"Likewise, it is a privilege to have electricity - you pay for it. What happens if you don't?"
Is that why rolling blackouts get declared national disasters? Try telling NYC that electricity is a privilege when the city is under a raging hellfire of riots and chaos.
Electricity is paramount to the operation of this country. Labeling it a privilege is just plain naive.
Is drinking water a privilege when I have to pay the city for water service? How about shelter due to rent or property taxes?
I'm afraid to ask just what exactly a 'right' is by your definition, but I'm assuming it'd be equally as nonsensical.
And, as an aside, popping into Bruce's blog and citing vague facts and figures whilst backing it up with nothing more than a 'sorry, my clearance doesn't allow any details', is hilarious.
That'd you'd accuse someone for having their 'head in the sand' because they don't believe a word of it is priceless.
@ Clive (12:57pm) Drivers Ed class isn't a part of the core curriculum, or at least it wasn't when I took it. Pretty much every kid wants to take it because it puts them one step closer to being allowed to drive. Driving in the US is also mostly controlled by the individual states, and the testing and requirements vary by state. It isn't at all the same as reading or writing (or _spelling_!), but more like what used to be called "Home Economics" back when I was in school. (i.e. A practical course on life skills that is handy to take.)
Again, it is not taught by the federal government and is not taught "to a standard" at all except by each individual state. It is a privilege, not a right, and the fact that there is an "auto addiction" doesn't make it any more a right than an addiction to alcohol makes drinking a right. I'm going to stop there, because I think I've made my point.
@Clive Robinson, Grey Bird: Back when I was in high school, driver's ed WAS taught in public schools here. It was optional, but it was probably the most heavily attended class in the entire school. If they had made a "C" average (in the rest of your classes) a requirement for taking it, grades would have gone up like mad. Cars (Ford Granada in my case [US version - no relationship to anything in UK or Europe by same name] ) were donated by car dealerships and teachers moonlighted for extra pay to drive with the students. We got 4 hours of mostly useless and sometimes scary (once on an off-ramp the student driving tried to turn the turn signal off by rotating the steering wheel rapidly, not making the connection that turning the wheel would make the car turn in a manner inappropriate to the road at hand) in-car driving, some mostly useful movies and a surprisingly effective (pre-computer) simulator.
Taking a class enables one to get a license at 16; otherwise (in OH) you are not able to get one until 18, at which age you are somehow magically blessed with the knowledge you'd have learned in class.
Today; however the ultrahyperlitigiousness of the US has made all the public schools (in OH at least) cease teaching it, and students take private driving classes. These are mostly small organizations with a tiny car and perhaps one or two instructors and evidently how to signal, merge, leave an assured clear distance ahead of you or drive without giving priority to your cellphone are not in the curricula. And heaven help them (and the people downhill of them) if they are presented with a clutch.
But as far as the article is concerned; I dont see the problem. In all the dozens of times I've been to various BMVs, none of them made me feel like smiling (well, maybe that first time).
And it seems like if smiling will break the software, then as a terrorist when I pulled off my terrorist coup I'd be grinning my ass off anyway and they wouldn't get a match.
I need to go get my license renewed in the next month or so. I'll bang my head on the desk repeatedly if New Jersey has this crap.
New Jersey, who until mid-2004 issued drivers licenses that were (I kid you not) polaroid photos, with a TYPEWRITER PRINTED card, and both were laminated together.
When I got my first drivers license in 1997 in Pennsylvania it was a computerized photo, with hologram background and mag strip. I switched it out for New Jersey in 2001 and thought it was an elaborate joke/hoax when I saw them produce my license.
And yet we have these same states buying into stuff like face recognition software that doesn't work. I presume they're using this to build a database to find people on CCTVs, etc. - but how useful are they if facial expression changes the match?
Well, I guess it might be better than I thought...
INDIANAPOLIS - Investment in new facial recognition technology at the Indiana Bureau of Motor Vehicles already is yielding returns with the arrest of a man on fraud charges after he sought a seventh Indiana identity.
Well, I always try to make my DMV picture look as bad as possible so when someone uses my license to ID me, I look better than that. Once I happened to be dead drunk (my boss got me drunk on my b'day) on the day I remembered I had to renew -- that one was priceless, and I can almost not believe they didn't just toss me in jail, as my breath was probably a fire hazard.
So I never smile anyway, and it's not like life at the VA DMV would make one smile spontaneously.
This last time I'd just had a very bad accident (not vehicle related), was full of morphine and bandaged and scarred badly -- the best yet, though this finally imposed the eyeglasses requirement; ever try to take an eye test at the limit of morphine overdose?
Any cop who looks at those and then at me will assume I am doing quite well and am very sober by comparison almost no matter the situation.
Works for me, anyway.
I live in a rural county (10k humans, one stoplight), and whether a driver's license is a privilege or a right is moot in at least one sense. If you or someone in your household doesn't have one, you simply cannot survive here for more than a few days unless you're some sort of extreme survivalist and have tools and food stock -- but you needed a job to get there. Even a walk to the nearest neighbor takes a very long time. The nearest place where there's more than one or two jobs at a general store is a 45 minute drive if you break every speeding law there is, an hour if you don't. I got by for years being the original telecommuter, but even then had to do face time with my consulting customers to get the jobs in the first place.
There are plenty of people, however who probably shouldn't drive, license or not, and I have no clue what a really good plan would be. Here we have plenty of old folks effectively DUI at all times, and whether it's their medication or simply the degradations of old age, they sure aren't safe to share the roads with. I speak from experience of being literally run over on my motorcycle in slow motion by one of them. As it was slow motion, I had time to jump off, but the bike didn't fare too well, and this isn't a rare example at all. I've been run off the road numerous times by these folks.
Oh yes, once I had to ID myself for a new license, as the old one had been stolen along with my wallet. I produced my copy of my birth certificate, which was unreadable even to me, knowing what was supposed to be on it. It flew fine, go figure.
Didn't the terrorists who got ID's here use their real names anyway?
This is not true for Arkansas. A local new station picked this up and they asked the DMV. The DMV said this was not true.
@petey, i agree, now if they would keep all these criminals in prision in the first place, We wouldn't need feel like the criminal.
They're lettin out murderers, and others who comitted horendous crimes, hmp! so what? they're allowed to smile.
i just renewed mt D.L. here in ontario, i was told exactly that..don't smile..we're a bunch of criminals now you know.
i don't like not smiling, i smile, i want our old system back.
Schneier.com is a personal website. Opinions expressed are not necessarily those of Co3 Systems, Inc..