Face Recognition by Thermal Imaging

New research can identify a person by reading their thermal signature in complete darkness and then matching it with ordinary photographs.

Research paper:

Abstract: Cross modal face matching between the thermal and visible spectrum is a much desired capability for night-time surveillance and security applications. Due to a very large modality gap, thermal-to-visible face recognition is one of the most challenging face matching problem. In this paper, we present an approach to bridge this modality gap by a significant margin. Our approach captures the highly non-linear relationship be- tween the two modalities by using a deep neural network. Our model attempts to learn a non-linear mapping from visible to thermal spectrum while preserving the identity in- formation. We show substantive performance improvement on a difficult thermal-visible face dataset. The presented approach improves the state-of-the-art by more than 10% in terms of Rank-1 identification and bridge the drop in performance due to the modality gap by more than 40%.

Tags: , ,

Posted on August 5, 2015 at 6:02 AM • 19 Comments

Comments

Spaceman Spiff • August 5, 2015 8:48 AM

Possibly something that cools the face down, such as Aloe Vera gel, would work as well.

Thrivalista • August 5, 2015 10:08 AM

Would it be more effective to cool only part of the face, to disrupt the overall thermal pattern?
Ice masks, anybody?

MikeA • August 5, 2015 10:39 AM

How about a dazzle-camo balaclava knitted from at least two types of yarn, one highly conductive of heat, one insulating. Bonus points for adding some nichrome wire or Peltier devices in there.

Major "Dutch" Schaefer • August 5, 2015 10:39 AM

I once had to fight an ebe that used similar technology in 1987. It was not easy.
But I accidentally discovered that covering myself in mud acted as camouflage by cooling my skin and blocking my body’s heat signature from the ebe’s thermal sensor. Anyway, I eventually got to the chopper.

8e0905cdaba5cfe48b818b4f34e992f6

Clive Robinson • August 5, 2015 11:01 AM

I suspect that those little plastic bag “Ice Packs” even when at room temprature would smear the thermal image a lot.

Likewise those freezer jel packs used to put on minor injuries.

Better still, you could as others have suggested have insulating and conducting parts. Be nasty and make the left cheek have a ‘U’ on it and the right the sixth capital from the alphabet…

The real problem I suspect is going to be the eyes, there is a lot of blood passing through them, hence high heat radiation potential.

The again for the “home brew” crowd, an Ice Hocky mask with plastic pipe on it going through one of those 12V pumps used in CPU water cooling systems.

I must admit I’ve been thinking about making such a system in a “cooling vest” form similar to the ones developed by NASA for astronauts and their EVA suits, for quite some time now, especialy every year when the British summer gets “sticky”. In theory… you could most times of the year use it for power generation, we do after all generate between 2-2.5Kw/hrs a day getting out an extra 2-5% would be quite usefull for phones and gadgets…

It’s kind of nicer sounding than “poo powered fuel cells” that actually appear to be quite viable…

albert • August 5, 2015 11:07 AM

Y’all are way off the mark. The fix has been in for years: a cap with very bright IR LEDs completely washes out the IR imaging.
.
..
.
..
o

SoWhatDidYouExpect • August 5, 2015 11:42 AM

As one “announced” recognition, identification, scanning technology cools off on the web, another pops up (as seems to happen a lot here). Despite the reality of such technologies, I tend to classify most of them as FUD.

When the spook agencies already have everything they want to deal with U.S. citizens, with regard to classifying them as terrorists, why would they bother with much of this other trash?

The best chance would seem to be in the Windows 10 HELLO process, which indicates it will use biometrics of several forms (fingerprint, face scan, perhaps voice recognition, & whatever), but it still depends on the users capitulating to make it work.

However, keep this stuff coming. We need to be aware of what is happening.

Think of the end game: when everybody in charge (assumption, the overlords) has everything about everybody else (the underlings), then what is left? Nothing. Off the grid may become the default when nobody can afford network connectivity and lack the ability to respond to targeted marketing efforts. Look at TV. We have many sources, hundreds and hundreds of channels, but practically no quality in the viewing that is offered.

Kevin • August 5, 2015 1:12 PM

@albert: IR is not the same as thermal.

Once FLIR Pro actually starts shipping their long-overdue BT/DT entry level thermal IP cameras, we’ll start seeing a lot more use of thermal for surveillance.

The dazzle-camo balaclava knitted from at least two types of thread is likely the most effective option. Adding a layer of highly thermally-reflective material (aluminized mylar film, like the kind used for children’s balloons) would also be useful. To prevent facial recognition from making a match, you need to at a minimum change the relationship between eyes/nose/mouth or just spread the heat enough that all these features are a thermal blur.

If anybody is interested in researching this further, prices of thermal modules and entire cameras are much more affordable than just a few years ago. You can get a handheld FLIR imager with USB interface for about $1K. Less expensive units will have limit resolution and low frame rates (9/15 FPS), but this is more than sufficient for experimenting with thermal face recognition.

Dirk Praet • August 5, 2015 2:30 PM

@ Major “Dutch” Schaefer

I once had to fight an ebe that used similar technology in 1987.

Nice trolling, but we got the reference to Predator from your first post.

albert • August 5, 2015 3:13 PM

@Kevin,
“… IR is not the same as thermal….”
.
Care to explain the difference?
.
..
.
..
o

albert • August 5, 2015 4:40 PM

@Nick P,

Thank you, Nick.

Nonetheless, thermal imagers will respond to infrared, whether active or passive. Active systems can be ‘tuned’ to the illumination frequency. Thermal detectors need to respond to a range of frequencies. The intensity of emitted thermal radiation from the human face should be easy to overcome with IR illumination of sufficient power, and not much power at that. This method has been used successfully in defeating standard nighttime security systems (whether active or passive, I do not know). IIRC, the perps face looked like a white blob. It’s a ‘jamming’ technique.

@Kevin,
You are (I think) talking about altering IR information just enough to foil facial recognition. This might be done in a subtle way (say, if you’re caught in the act, with the necessary materials). The total overwhelming jammer would be an obvious giveaway, even though it’s just a baseball cap with LEDs on it:)

More research needs to be done, not by me, you can bet someone will.

The thought also occurred to me that thermal imaging might be a useful adjunct to visible-light ID photos(not to mention apple-to-apple comparisons for thermal surveillance). Folks with identical facial measurements might have radically different thermal signatures. Since the images are digitized, such information could be encoded into the standard image data. It wouldn’t surprise me if ‘they’ are already working on this.

. .. . .. o

Anura • August 5, 2015 4:55 PM

I wonder how effective this is in daylight; I’d imagine it would lose a lot of detail.

albert • August 5, 2015 6:33 PM

@Anura,
Direct sunlight would obviously heat the face, but in the shade it might work OK. Dry skin probably wouldn’t reflect much IR. How much are measurements affected is the question.
. .. . .. o

asd28 • August 6, 2015 12:49 AM

@albert
Thermal and visible channels simultaneously ?

Add 1 to 50 more channels and you’ve got hyperspectral imagery. I thought those were still sorta expensive ?

From some older info on hyperspectral imaging, one of the bigger problem was interpreting the data. Hard to display 50 different color channels on an RGB display. So it takes a long time to review images by hand.

albert • August 6, 2015 10:50 AM

@asd28,
Yes.

When demand goes up, expense comes down. Since when does expense matter when we’re ‘fighting terrorism’?

The software is doing the lookin’.

. .. . .. o

HKV • August 9, 2015 9:17 PM

I saw that guy on tv demonstrate this led cap of lights
It blocks facial identification, but it flags the person who wears it like a neon sign.
is this how you want to be stealthy
If you approach a common heat/infrared sensor, the little red light will come on,but if you move slowly behind a common corregated brown cardboard shield you can pass.
cardboard with an eye slit would be less obvious than the ir leds

even better, A neck shield like the back of a french kepi and another that is up in the cap and fitted to drape the face with whatever material can block view through (synthetic fibers are very see through for imaging), but you could decorate it with horizontal bars and some fake eyes painted on it perhaps metalic paint, this can easily be dropped from the hat over the face, or when necessary simply put under the cap again much quicker than wiping off camo face makeup
dont put the horizontal bar in front of the eyes though
you want to break up the recognition without attracting attention. and you want to get rid of the disguise fast

Saquib Sarfraz • August 10, 2015 9:44 AM

Everyone, really nice discussion and comments on camouflaging face by using clay and what not!, but are you not missing the point here. these systems are meant for covert surveillance ,that means you do not know if you are being imaged. while in case you want to hide your identity, you would not need clay or other such things, just covering your face by your hand would be enough :).

But sure good to see so much interest on our work.

Leave a comment

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.