Apple's FaceID

This is a good interview with Apple’s SVP of Software Engineering about FaceID.

Honestly, I don’t know what to think. I am confident that Apple is not collecting a photo database, but not optimistic that it can’t be hacked with fake faces. I dislike the fact that the police can point the phone at someone and have it automatically unlock. So this is important:

I also quizzed Federighi about the exact way you “quick disabled” Face ID in tricky scenarios—like being stopped by police, or being asked by a thief to hand over your device.

“On older phones the sequence was to click 5 times [on the power button], but on newer phones like iPhone 8 and iPhone X, if you grip the side buttons on either side and hold them a little while—we’ll take you to the power down [screen]. But that also has the effect of disabling Face ID,” says Federighi. “So, if you were in a case where the thief was asking to hand over your phone—you can just reach into your pocket, squeeze it, and it will disable Face ID. It will do the same thing on iPhone 8 to disable Touch ID.”

That squeeze can be of either volume button plus the power button. This, in my opinion, is an even better solution than the “5 clicks” because it’s less obtrusive. When you do this, it defaults back to your passcode.

More:

It’s worth noting a few additional details here:

  • If you haven’t used Face ID in 48 hours, or if you’ve just rebooted, it will ask for a passcode.
  • If there are 5 failed attempts to Face ID, it will default back to passcode. (Federighi has confirmed that this is what happened in the demo onstage when he was asked for a passcode—it tried to read the people setting the phones up on the podium.)
  • Developers do not have access to raw sensor data from the Face ID array. Instead, they’re given a depth map they can use for applications like the Snap face filters shown onstage. This can also be used in ARKit applications.
  • You’ll also get a passcode request if you haven’t unlocked the phone using a passcode or at all in 6.5 days and if Face ID hasn’t unlocked it in 4 hours.

Also be prepared for your phone to immediately lock every time your sleep/wake button is pressed or it goes to sleep on its own. This is just like Touch ID.

Federighi also noted on our call that Apple would be releasing a security white paper on Face ID closer to the release of the iPhone X. So if you’re a researcher or security wonk looking for more, he says it will have “extreme levels of detail” about the security of the system.

Here’s more about fooling it with fake faces:

Facial recognition has long been notoriously easy to defeat. In 2009, for instance, security researchers showed that they could fool face-based login systems for a variety of laptops with nothing more than a printed photo of the laptop’s owner held in front of its camera. In 2015, Popular Science writer Dan Moren beat an Alibaba facial recognition system just by using a video that included himself blinking.

Hacking FaceID, though, won’t be nearly that simple. The new iPhone uses an infrared system Apple calls TrueDepth to project a grid of 30,000 invisible light dots onto the user’s face. An infrared camera then captures the distortion of that grid as the user rotates his or her head to map the face’s 3-D shape­—a trick similar to the kind now used to capture actors’ faces to morph them into animated and digitally enhanced characters.

It’ll be harder, but I have no doubt that it will be done.

More speculation.

I am not planning on enabling it just yet.

Tags: , , , , ,

Posted on September 19, 2017 at 6:44 AM • 43 Comments

Comments

Mike Scott • September 19, 2017 6:57 AM

My understanding from what I’ve read is that the police can’t just point your phone at your face to unlock it. It has attention detection built in, and you have to actually be looking at the phone for it to unlock. Just look away from the phone and the police can’t force an unlock. Of course it remains to be seen how well that works in practice.

Scott • September 19, 2017 7:50 AM

The linked TechCrunch piece has these two quotes:

“If you haven’t used Face ID in 48 hours, or if you’ve just rebooted, it will ask for a passcode.”

“You’ll also get a passcode request if you haven’t unlocked the phone using a passcode or at all in 6.5 days and if Face ID hasn’t unlocked it in 4 hours.”

How these two add up to each other? I don’t quite get it.

Off. Bruce has a nice, private commenting system here. It should be effective against spammers, I guess. Is it available in an open source form for other bloggers as well, or is it completely custom made, for Bruce’s private use only?

bkd69 • September 19, 2017 7:51 AM

It’s long past time that we had a ‘public use only’ ROM/launcher.

It wouldn’t have access to your contacts or calendar or any such thing, it would only be capable of making emergency calls, and it would only display apps in the market with an ‘official use only’ flag, like car insurance apps. In short, a phone that you can safely hand to anyone because it won’t store any compromising information on there beyond what are already public or semipublic records, like proof of insurance cards.

Peter Brass • September 19, 2017 7:59 AM

I’m curious as to why the theory of police pointing the phone at one’s face is perceived as different than the established precedent of their being allowed to use one’s fingerprint to unlock a phone. While either of these are legitimate concerns, the far more probable security concern is viewing one’s PIN via “shoulder surfing” and/or the pervasive video surveillance in today’s society. Sadly, while a combination of both would result in vastly improved security, the incrementally less streamlined process would likely result in a significant percentage of the population disabling the feature(s). (For instance, the large number of people with no PIN enabled on their phones, whatsoever)

While TouchID has been proven to be vulnerable to (relatively non-trivial) methods to spoof one’s fingerprint and FaceID, those exploits are less probable in daily life than the vulnerabilities associated with weak (or no) PINs.

Additionally, Apple is making it easier and more streamlined to disable Touch or FaceID on an ad-hoc basis. This, combined with now requiring a password when connecting to a computer, further demonstrates their willingness to provide a secure platform with a minimal amount of friction to the user experience.

Carl • September 19, 2017 8:18 AM

“Apple is not collecting a photo database”. They don’t have to. Ppl provide countless selfies to iCloud that can be used for facial recognition.

Clive Robinson • September 19, 2017 9:45 AM

@ Bruce,

It’ll be harder, but I have no doubt that it will be done.

Probably not much harder.

Think of it as any other new system, although it’s been tested the testing will not be as intensive as hundreds if not thoudands of keenly motivated individuals looking to make a name for themselves…

I’d give it about three months before the new software needs a patch or two.

Please not Im not saying Apple is incompetent, it’s just that the odds are against them especially as there might be a $million or two for the first person to market a,weakness correctly.

Robert • September 19, 2017 10:14 AM

I think the real question here is what the legal ramifications are. Can a police officer force you to look at your phone without a warrant? Is there a law against that? It’s not a fingerprint or a password you have to turn over. It’s something you look at. AFAIK there’s nothing in the law that really protects a consumer here. Fingeprints are a very grey area… but at least there’s some concept that it might be/should be protected… Looking at something? I’m not so sure.

Sofa • September 19, 2017 10:24 AM

@Scott

When I read it originally I had the same thought. What it is saying is if you have not used the FaceID in 48 hours or just rebooted it will require the code.

If you have not unlocked the phone using a passcode at all, or for 6.5 days or more, meaning you have ONLY been using FaceID for at least 6.5 days, it will require the code.

The first talks about not using FaceID or rebooting.

The second talks about ONLY using FaceID not the code.

Does that help

-Sofa

Clive Robinson • September 19, 2017 12:12 PM

@ Peter Brass,

<><>

I guess it depends on which jurisdiction you are in.

However your face is usually considered to be “In plain sight in a public place” and ordinarily available to surveillance camera’s etc.

Thus in the US I can see not only some psyco from the DoJ making the case that it’s perfectly legal just as pointing the phone at a poster on the wall etc. Or more specifically on using a password written on a Post-It note attached to the screen of a computer. Likewise I’m sure the FBI etc can find a tame judge to see it there way, thus trigger a long slow legal process up to SCOTUS. Failing that find a judge to jail you on contempt charges, till you unlock the phone using the pass phrase etc. But the other option is to set boarders and customs on you, which legaly means you are stuffed.

I’ve discussed with @Wael and @Nick P in the past about amplifing the “Something you know” factor both temporally and geospacially. That is the phone needs to be at a certain place at a certain time and the right anti duress code entered to unlock it when a full lockup has happened. Thus somebody could set it to be “out of US jurisdiction” etc. If it was publicized then a judge would find it difficult to jail you for contempt because it would be beyond your ability to unlock it whilst in US jurisdiction.

It would not be difficult for any phone to be equiped with such a fail safe. However the OS designers/implementors appear to be taking a very long slow path to get around to this sort of thing.

Further it’s all very well for Apple to say “five is the magic number” for button presses or retry limits, but some people would like to be able to configure a different number to catch out the unwarary etc.

Steve • September 19, 2017 12:42 PM

I wonder if you could use some sort of “pulled face,” such as sticking out your tongue, wrinkling your nose, or something goofy (think “Calvin + Hobbes”), as your unlocking “faceword” to thwart unintentional unlocking.

Dragan • September 19, 2017 1:07 PM

You do know next step is DNA, with personal details, location, fingerprint and current photo, that is the only missing part in the puzzle.

Matt from CT • September 19, 2017 2:00 PM

I wonder if you could use some sort of “pulled face,”

The more interesting use case is as a duress code.

3 right eye winks, one left eye wink, phone bricks.

Northern Realist • September 19, 2017 2:22 PM

Rather cumbersome processes to go thru for someone under duress of either a theft or a police search…

Bruce Schneier • September 19, 2017 7:58 PM

@Scott:

“Off. Bruce has a nice, private commenting system here. It should be effective against spammers, I guess. Is it available in an open source form for other bloggers as well, or is it completely custom made, for Bruce’s private use only?”

It’s just Movable Type.

aklak • September 19, 2017 10:22 PM

@Scott,Bruce

Movable Type is apparently written in Perl rather than PHP, which should be a start.

Reminds me of the I took a couple of PHP classes just for fun at Clark College in Vancouver, Washington. (Having been adjudicated as a mental defective in a court of law, I am essentially unemployable.) At that time, Stefan Esser was blogging the “Month of PHP Bugs,” which was heavily censored by the German government.

Perl has been around a lot longer than PHP, but I’m still not so sure of its security as another one of those interpreted languages with fast and loose automatic memory management.

You could go Ruby, OCaml, Java, Javascript, and on and on. There seems to be a superfluity and overabundance of “high-level” programming languages, and a strange dearth of close-to-the-metal hard logic “low-level” languages like C, C++, Fortran, and Ada, especially for Web applications.

aklak • September 19, 2017 10:38 PM

I was stuck in Kansas City, Missouri for some months, and I noticed that Stefan Esser just tweeted about a dead body found at the airport.

KANSAS CITY, Mo. (AP) — Body not found for 8 months in Missouri airport parking lot

That’s Mob murder, not suicide, and everybody who saw it or smelled it up to that time knew good and well to zip their lips and keep their mouth shut about it because the authorities were no doubt already aware of it for a long time but they didn’t do shit, because the stinking corpse was left as a “warning” by the Mafia in Kansas City, and all the local mollies riding motorcycles out recruiting for the “Piston Annies” and all that piss and vinegar.

They even have a nostalgic display at the Kansas City train station of the local history of the Mafia and La Cosa Nostra.

Anura • September 19, 2017 10:46 PM

@aklak

Movable Type is apparently written in Perl rather than PHP, which should be a start.

Start of what? Night terrors?

Wael • September 19, 2017 11:30 PM

Peter Brass,

I’m curious as to why the theory of police pointing the phone at one’s face is perceived as different than the established precedent of their being allowed to use one’s fingerprint to unlock a phone.

With fingerprint, the police officer doesn’t have to touch the user. With FaceID the police officer can tell the person to look at the birdie and say cheese. FaceID and fingerprint aren’t equivalents: they operate at different distances form the phone, for one. I can think of ways to defeat FaceID, for example: identical twins, or people who look alike. In that case: John Malkovich can unlock Michael Chertoff’s phone in a blink of an eye!

@Clive Robinson,

I’ve discussed with […] in the past about amplifing the “Something you know” factor both temporally and geospacially.

On many occasions. This is a bit off-topic, but https://www.schneierfacts.com/facts/304 I don’t know what made me think of it. Oh, yea… @Bruce can unlock Chuck Norris’s phone too. With the camera covered, mind you – they exchanged a key in the past.

neill • September 20, 2017 12:54 AM

“face ID”

has anyone thought about what happens when there’s an accident (car, motorcycle, bicycle, inline skating, … ) where one has a significant facial injury, BUT is still mentally OK and just wants to call for help???

Simple • September 20, 2017 1:26 AM

@Neill

Simply disable face id and proceed with passcode.

The really trouble if someone is mentally incapacitated or dies, and there is critical need to access the phone for the relatives of the person? Eg access to bank account details or bitcoins etc

225 • September 20, 2017 2:21 AM

Something interesting apple could do with the TrueDepth is turn off a random handful of the “30,000 invisible light dots” each time, and check they are off to guard against replay attacks using a scan of the users face and an IR emitting device.

I don’t know how this would help with the filth demanding compliance vs the Fifth Amendment

Moryn • September 20, 2017 3:14 AM

So eyes have to be open to unlock? Removes one of my concerns which was the mischievous/ill-intentioned/curious partner/spouse/one night stand taking the phone and opening by holding it up to your face while you soundly sleep. Come on, tell me you think people wouldn’t try it.

neill • September 20, 2017 3:15 AM

@Simple

possible though that the injured is dazed and confused, and does not remember the code correctly

or that, if one has not used the pincode for months or years, it was just simply “forgotten”

225 • September 20, 2017 3:45 AM

@neill “You’ll also get a passcode request if you haven’t unlocked the phone using a passcode or at all in 6.5 days and if Face ID hasn’t unlocked it in 4 hours.” so you only go a week without remembering your pincode. Also phones already have an emergency numbers only mode without logging in.

Halloween costumes might also throw off the ID system, but not everyday is Halloween or facial reconstruction accident day

Clive Robinson • September 20, 2017 6:17 AM

@ Moryn,

… holding it up to your face while you soundly sleep. Come on, tell me you think people wouldn’t try it.

Yes it’s an odds on certainty, like a street thug or Gov equivalent doing it whilst you are out cold on the floor…

It does however raise the “gravity” question. Peoples faces under changing “g forces” whilst I’m not talking “4G face ripple” you get a 2G change from standing up to hanging down from your feet. Likewise a 2G change from face up to face down. The older you get the saggier your face gets hence the old saw about “You get the face you deserve by fourty” and an Ig Nobel award for a South London GP (family doctor) who did research on why mens have bigger ears the older they are (thanks to gravity).

Thus if Apples software makes alowance for upto a 2G transformation then that will open up a potential attack vector, that might prove fruitfull…

George • September 20, 2017 9:12 AM

So the thief points the weapon at you and says “unlock it.” How are these measures going to help?

How about press & hold for five seconds disables the phone altogether for 30m (time to remotely wipe it)?

Patrick • September 20, 2017 9:19 AM

Face recognition adds security challenges that are difficult to predict. Apple must give a viable alternative to retain the password by default and only activate facial recognition at will by entering a second code, so that the user may easily opt in and change during safe times to use facial recognition, (I.e., while at home if not exposed to a jealous wife…) or in the office, then disable by a simple gesture or accelerometer movement. Otherwise, all fears expressed will be realized !

A Parent • September 20, 2017 10:11 AM

My toddler figured out that she can unlock my phone when I’m sleeping just by placing my finger on the circle. Malicious hacker’s and thieves aren’t the only ones breaking into phones. How much worse when physical contact isn’t required? “Hey Mom!” “Hey Dad!” “Look over here!” “Thanks for buying me that new toy, new app, new game, etc.”

Ergo Sum • September 20, 2017 10:15 AM

@Clive…

Yes it’s an odds on certainty, like a street thug or Gov equivalent doing it whilst you are out cold on the floor…

That might not work, based on the requirement that the eyes need to be open and looking at the iPhone. Unless the “attention detection” feature had been turned off.

On the surface, the “FaceID” seems more secure than the “TounchID”, the latter one, even a six years old child can overcome:

http://www.complex.com/life/2016/12/kid-buys-250-dollars-pokemon-items-with-moms-thumbprint

Scott • September 20, 2017 11:06 AM

Thank you @Bruce, too! Interestingly, regardless of this comment thread, I just researched Movable Type as a blogging platform. Once it was open source, now it’s a paid product, as I under stand. Which may be fine for well established bloggers, but may turn away folks who just want to try blogging. Perhaps, any other viable recommendations? Movable Type had nice designs, though.

Joomla • September 20, 2017 12:43 PM

Let me remind everyone here, all one has to do on Touch-ID and FaceID is press the power button 5 times to force passcode lock mode. Easy to do when you think you’re going into a place they search or being pulled over.

phry • September 20, 2017 4:00 PM

Now I’d like to know if someone knows of a similar “require pin quick lock” function for the Android lockscreen.

Joao • September 21, 2017 3:23 AM

It would be nice for Apple to add the option to use FACE ID + Finger ID + Passcode/ pin to unlock the iPhone.

You need to see it, you normally hold it, and since you normally also interact with it, this would be 3 things need to unlock it… no one would have their phone unlocked in normal stress circumstances unless they really wanted to, at the very least they could (really) forget the code… due to the stress you know.

The code could be spy over the shoulder, cameras, but that could be thwarted by privacy screens and random position of the keyboard for example.

But thief’s and smart police officers are one step ahead! They many times wait for you to unlock the smartphone and then they steal from you while you are talking to someone for example.
It would be nice to have something else like some device that would be responding to “pings” from the device via some radio beacon but in someway to make it absolutely impossible to know from who, and impossible to replicate. If the device that the smartphone/ iPhone was pinging step out of range or was manually disconnected the smartphone/ iPhone would lock it self and require full login again to unlock it.

Because people could just interfere on purpose on the radio beacon it should allow unlocking and have an auto-lock after some number of minutes to make sure even if someone stole the smartphone/ iPhone it would be locked after a few minutes and some sensitive parts (previously selected by the user besides the normal ones like call logs, definitions) could need re-authentication to make sure they stay private even if the phone go away suddenly. It can be adjusted, for example, while you are looking at it wouldn’t lock. Should be easy enough that people want to use it, but not allow not authorized people to use it to.

Wael • September 22, 2017 1:32 AM

@Bruce Schneier,

I am not planning on enabling it just yet.

Then your only option is to use a passcode. I believe FaceID is more robust than TouchID.

AFAIK • September 22, 2017 6:58 AM

Balance…. There are soooooo many other fundamental components of mobile that need rethinking; face rec is ‘adequate’ for most mobile uses. I&A is the easy part.

Excellent blog, Bruce. It’s a tiny island in an ocean of ignorance and confusion. Thank you.

Sidelobe • September 22, 2017 7:01 AM

It occurs to me that Face ID will be excellent for those who prop up their phone on the dashboard of their car while they drive. They won’t need to set the phone to never lock, and the device will stay unlocked while they are alert and driving. But the phone will still lock automatically when they’re no longer in front of it. This is a similar advantage to those who keep their phone on their desk, as evidenced by the issue at the podium. And, if a proper link can be made between the phone and other desktop equipment, the abilities of auto lock / unlock go way up.

Daniel • September 22, 2017 11:04 AM

Apple has been the leading major corporation in promoting user privacy and security in recent years, and I concur with Bruce’s assessment that they probably aren’t building a database of user facial patterns (or fingerprints). However, this technology is still ripe for abuse insofar as even if Apple has good intentions, it can be coerced into using it for oppressive or Orwellian purposes by nation-state actors all over the world: China, Russia, the US, the EU or elsewhere.

So long as the source code is closed, we can’t know what Apple is doing (or is being forced to do). I applaud Apple for their many great security innovations in recent years, but it’s time to cross the finish line and open source ALL the code to shutdown the backdoor problem in their software once and for all. The market is likely to reward this with increased sales inspired by the newfound confidence people will have in Apple’s integrity.

It should be noted that open sourcing the code does not mean that Apple hands off control to a decentralized group. Apple retains central control (and ownership) of the projects, which are presumably reliant on Apple servers and hardware, so nothing can be added to the official software unless approved by Apple. It just means that researchers could audit the code, look for bugs, suggest improvements, submit code for consideration, and keep Apple honest.

The market for software is moving in two divergent directions: cloud computing / total proprietary control, and genuinely open source. Apple’s middle of the road approach with some open source here and there is on the wrong side of history. The restrictive proprietary controls of Microsoft et al will cause even greater numbers of users to move to Linux and other open source platforms; the trend lines back this up. If Apple wants to get ahead of the game, it must go open source.

If there’s any Apple management reading this, here’s an idea to ease into it: start by open sourcing all the code for iOS and OSX which is older than three years. Create a rolling schedule where all versions of the software are open sourced once they are three years old. This will be slightly-dated technology by the time it’s released so Apple’s latest is still proprietary for a time. Ease into it and see how the market responds.

A Nonny Bunny • October 7, 2017 2:31 PM

@A Parent

How much worse when physical contact isn’t required? “Hey Mom!” “Hey Dad!” “Look over here!” “Thanks for buying me that new toy, new app, new game, etc.”

“Good luck playing that game with broken fingers, son”

But seriously, if you expect trouble, turn the feature off? Make sure the phone’s locked so your child can’t access it. And raise the kid not to do bad things just because it can.
Actually, do give it the opportunities to do (mildly) bad things, and then teach it there are consequences. (But not breaking fingers, that was just a joke).

MarkH • November 14, 2017 1:57 PM

File Under: It Would be Funny, if it Weren’t So Depressingly Stupid

10-year-old boy can unlock mom’s iphone face ID

When the mother set up her face ID in low illumination, he could unlock her phone every time (in the spirit of scientific inquiry, she repeated this). When she used brighter light, he could no longer unlock.

Also, he once unlocked his dad’s face ID, but was not able to repeat this success on subsequent attempts.

It may be a crap security technique … but perhaps an interesting tool for Automatic Detection of Family Resemblance.

I’ve been an Apple skeptic since 1978, and have yet to see any basis for upgrading my evaluation.

Leave a comment

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.