Recovering Keyboard Inputs through Thermal Imaging

Researchers at the University of California, Irvine, are able to recover user passwords by way of thermal imaging. The tech is pretty straightforward, but it's interesting to think about the types of scenarios in which it might be pulled off.

Abstract: As a warm-blooded mammalian species, we humans routinely leave thermal residues on various objects with which we come in contact. This includes common input devices, such as keyboards, that are used for entering (among other things) secret information, such as passwords and PINs. Although thermal residue dissipates over time, there is always a certain time window during which thermal energy readings can be harvested from input devices to recover recently entered, and potentially sensitive, information.

To-date, there has been no systematic investigation of thermal profiles of keyboards, and thus no efforts have been made to secure them. This serves as our main motivation for constructing a means for password harvesting from keyboard thermal emanations. Specifically, we introduce Thermanator, a new post factum insider attack based on heat transfer caused by a user typing a password on a typical external keyboard. We conduct and describe a user study that collected thermal residues from 30 users entering 10 unique passwords (both weak and strong) on 4 popular commodity keyboards. Results show that entire sets of key-presses can be recovered by non-expert users as late as 30 seconds after initial password entry, while partial sets can be recovered as late as 1 minute after entry. Furthermore, we find that Hunt-and-Peck typists are particularly vulnerable. We also discuss some Thermanator mitigation strategies.

The main take-away of this work is three-fold: (1) using external keyboards to enter (already much-maligned) passwords is even less secure than previously recognized, (2) post factum (planned or impromptu) thermal imaging attacks are realistic, and finally (3) perhaps it is time to either stop using keyboards for password entry, or abandon passwords altogether.

News article.

Posted on July 10, 2018 at 6:18 AM • 36 Comments

Comments

meJuly 10, 2018 7:00 AM

"thermal imaging attacks are realistic"
"recovered by non-expert users as late as 30 seconds after initial password entry"

i don't find this realistic, quite the opposite...
if you enter a password it's to do something (say check an email or whatever)
unless we are taliking about some pin protected gate/door, in that case the only thing you do is enter.

but still... just placing a camera and filming the password seems lots easier.

meJuly 10, 2018 7:02 AM

i would say that could be more realistic to hide a microphone somewhere so you don't need to watch the keyboard like with a camera. but you can hide it better.

and then by using inter-key delay time you reconstruct what has been typed. with some guessing.

bobJuly 10, 2018 7:38 AM

@me

Body eclipses keyboard? Hand eclipses PIN pad? 3rd party enters a password or PIN for your access?

neillJuly 10, 2018 7:42 AM

you can also use non-thermal methods to get more info about the password:

1-clean the keypad
2-wait for the user input
3-swipe every key with a cotton swab (maybe pre-arranged in a grid that matches the keys)
4-analyze the swabs for dirt/grease/oils from the user's fingers

combine that with visual (camera) intel and you get a good guess what the PIN is

neillJuly 10, 2018 7:49 AM

PS ... or sprinkle flour/dust on the keypad (might be seen), UV ink etc

however where i live 99% (my guess) of users do not wear gloves (that might render thermal useless)

echoJuly 10, 2018 7:58 AM

Is there a formula to guage security akin to the Drake formula and if not why not?

Todd ArnoldJuly 10, 2018 8:10 AM

I agree with the post saying it is unlikely someone will enter a password, then immediately leave so that a thermal scan can be done. On top of that, it's likely that after entering a password, the user will immediately begin typing on the keyboard to do whatever they logged in for - obscuring the characters typed for the password.

There is another interesting aspect of this attack. In high-security cryptographic systems, it is common for cryptographic keys to be manually entered in multiple cleartext key parts (dual control, split knowledge). Entry of those key parts would be subject to this same thermal attack. (However, key part entry is usually done in tightly controlled rooms.)

WinterJuly 10, 2018 8:15 AM

This will not affect computers much. How about PIN pads on ATMs, doors, and safes?

Petre PeterJuly 10, 2018 8:29 AM

I imagine it's also easier to harvest passes from touch screens.Just sent a picture of the touch screen to the thermal lab.

David RudlingJuly 10, 2018 8:34 AM

This can be fixed for the future by good security design. Non-thermally-retentive keyboard/keypad materials need to be specified as a security consideration for equipment in high risk scenarios. Like all security design the cost of the change will be judged against the risk assessment. For normal home or office use my guess is that probably nothing will change.

goodmanJuly 10, 2018 10:11 AM

I'm sure I saw exactly this in a movie/TV plot (maybe Mission Impossible; the paper mentions it without explanation)... and I've always tried to touch all an ATM's keys to prevent it.

They don't say what to replace keyboard password entry with. They list it separately from "abandon passwords", hinting at an alternate entry method—maybe touchscreens? They'd have the same problem, likely, unless randomized.

It would be interesting to have a phone-based login option for ATMs. (And for that matter, a PIN/password-based option when calling a bank's voice line.)

Santa ClausJuly 10, 2018 10:42 AM

Nothing new. Edmond Locard 1877-1966 said (shortened as "every contact leaves a trace") - longer version: "Wherever he steps, whatever he touches, whatever he leaves, even unconsciously, will serve as a silent witness against him. Not only his fingerprints or his footprints, but his hair, the fibers from his clothes, the glass he breaks, the tool mark he leaves, the paint he scratches, the blood or semen he deposits or collects. All of these and more, bear mute witness against him. This is evidence that does not forget. It is not confused by the excitement of the moment. It is not absent because human witnesses are. It is factual evidence. Physical evidence cannot be wrong, it cannot perjure itself, it cannot be wholly absent. Only human failure to find it, study and understand it, can diminish its value."

albertJuly 10, 2018 10:50 AM

This is a non-solution in search of a non-problem. Really. Is there no computer-security related news today?

How about heating the keyboard to finger-tip temperature? I'm going to patent that. Beware, copycats!


@echo,
"...Is there a formula to guage security akin to the Drake formula and if not why not?..."

Because it would be as useless as the Drake equation. (I assume you meant 'equation')

. .. . .. --- ....

echoJuly 10, 2018 11:33 AM

@albert

Oh you know what I mean. It would stop at least half a dozen media panics over nothing each year.

While you are at it could you make a mascara and eye liner warming machine. I hate it when it shocks my inner eyelid and blinking when its too cold and its a bit chilly putting it somewhere to warm up. Speaking of which I daresay eyeshadow could be lightly painted onto the numbers of keypads or keyboards to reveal what keys have been pressed and I can't imagine any security guard having much of a career left if they put a halt to women carrying make up into work.

albertJuly 10, 2018 12:06 PM

@echo,

A makeup warmer is an idea you should pursue. Get a patent. Don't forget a backup-battery-powered one as well.

. .. . .. --- ....

meJuly 11, 2018 2:08 AM

@David Rudling
"This can be fixed for the future by good security design. Non-thermally-retentive keyboard/keypad materials"

Or... we can use a standard touchscreen and display numbers every time in random order instead of:
123
456
789
-0-

like:
921
354
607
-8-

and partially fix also camera problem.

@Winter:
i was thinking about atm too but again, after you type pin more than 30 seconds pass before you go away and you are not even indoor, maybe wind cancel thermal image faster.

again, i find normal camera (or dust, as suggested by someone) quite simpler

HermannJuly 11, 2018 3:46 AM

@David Rudling
"This can be fixed for the future by good security design. Non-thermally-retentive keyboard/keypad materials"

It will be probably cheaper to use gloves or a stylus

David RudlingJuly 11, 2018 3:59 AM

@Hermann

"It will be probably cheaper to use gloves or a stylus"

Absolutely. As I said, a proper security design fix could be justified only in situations with a high security risk assessment, rather as in the old days of TEMPEST equipment. Simple mitigaton steps such as you so ably describe will undoubtedly be the preferred option for other concerned users.

meJuly 11, 2018 6:21 AM

@Weather
showind digits on a touch screen in random order every time you unlock the phone, it's *not* the same as having the digits always in the same place.

if you see thermal map like this:
low heat top left, mid heat at top center, high heat at top right
you can deduce that the pin was 1,2,3 (3 pressed last so more heat visible).
even without thermal imaging and supposing a clean screen you will see three digital fingrprints so you know that numbers 1,2,3 are part of the pin but you don't know the order (and the keyspace is reduced a lot).

now, if you show digits on screen in random order, knowing that you pressed top-left, top-center- top-right you gain nothing because next time you try to unlock, the numbers will appear in a random, differnt location, so repeating top-left, top-center, top-right you will write a different and invalid pin.

the same can be done for a keyboard but given the huge ammount of keys people will waste an hour just to find out where a letter is.

albertJuly 11, 2018 10:17 AM

@Sed Contra,

"Technology giveth and technology taketh away." - albert

New battery technologies (Lithium-ion) have given us very high energy storage in very small packages. Are you familiar with 'fuel guage' ICs? There is no protection against shoddy production of the batteries themselves. Besides explosions, Li batteries 'carry their own' oxygen supply, so conventional firefighting procedures don't work.

Your point is valid.

Maybe @echo should start with the plug-in version first.

. .. . .. --- ....

echoJuly 11, 2018 10:30 AM

@albert

I have given up and going with the established low tech version. It's a faff and I hate it when I forget but it works.

Why has everyone forgotten that not every battery is a lithium battery? If I built a make up warmer for mascara and eye liner I would likely prefer ordinary rechargeable batteries. Because they are produced to a universal standard they can also be used in other devices too which might be handy. One tiny fridge/warmer which is commercially available runs off USB which together with US charging plugs is another emerging universal standard.

albertJuly 11, 2018 10:35 AM

@David Rudling,

"...Non-thermally-retentive keyboard/keypad materials..."

I'm interested in the 'Non-thermally-retentive' materials part.

Do you examples? A link or two perhaps?

. .. . .. --- ....

vinnygJuly 11, 2018 6:19 PM

@echo re: mascara warmer - Well, you could try warming the eyeball itself via microwaves. Could have serious side-effects, though...
@albert In the highly unlikely event that this actually became a "thing," think of the fun to be had by deliberately warming the "wrong" keys. A message could be sent to the would-be pw snatcher, content left to the reader's imagination.

echoJuly 11, 2018 7:17 PM

@vinnyg

Re-softening gel eyeliner in a glass pot is possible in a microwave. Broken powder makeup can also be fixed with surgical spirits (USA: rubbing alchohol).

Ooooh. Sneaky! Configrable warming keys. I like it! There is also the new thermal camoflage which can block heat or project a heat map. It's all a bit Mission Impossible but perhaps camoflage window covers could be used one day to project false data?

Sancho_PJuly 12, 2018 3:26 AM

re thermal image of keyboard

I wonder why they haven‘t shown (and used) the „Enter“ key image.
For sure the last one pressed, it may give a valuable reference for timing and pressure.
- And in some apps it would show that thermal imaging is useless.

Or was the test done without sending the pwd?
Ouch.

raspy sturgeonJuly 12, 2018 6:22 AM

Er... let me weigh the risks:

1. I could type in a password on my keyboard knowing that someone could be hiding under my desk with a thermal camera, hoping that today I might suddenly stand up and leave the room (for some unfathomable reason) 15 secs after I type my login password, manages to capture an image of the keyboard in the remaining 15 secs and leaves the scene without being seen.

2. Or I could adopt an authentication method that is not protected by the 5th amendment and can be forced out of me by any schmuck with a uniform.

I think I've made my mind up.

B33FJuly 12, 2018 6:53 AM

This has been known about for a long time, there's even videos on youtube demonstrating actual implementation in a public space using a FLIR ONE attached to an Android phone for shoulder surfing.

Leave a comment

Allowed HTML: <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre>

Photo of Bruce Schneier by Per Ervland.

Schneier on Security is a personal website. Opinions expressed are not necessarily those of IBM Resilient.