Gaze Tracking Software Protecting Privacy

Interesting use of gaze tracking software to protect privacy:

Chameleon uses gaze-tracking software and camera equipment to track an authorized reader’s eyes to show only that one person the correct text. After a 15-second calibration period in which the software essentially “learns” the viewer’s gaze patterns, anyone looking over that user’s shoulder sees dummy text that randomly and constantly changes.

To tap the broader consumer market, Anderson built a more consumer-friendly version called PrivateEye, which can work with a simple Webcam. The software blurs a user’s monitor when he or she turns away. It also detects other faces in the background, and a small video screen pops up to alert the user that someone is looking at the screen.

How effective this is will mostly be a usability problem, but I like the idea of a system detecting if anyone else is looking at my screen.

Slashdot story.

EDITED TO ADD (7/14): A demo.

Tags: ,

Posted on July 14, 2009 at 6:20 AM • 39 Comments

Comments

John • July 14, 2009 6:51 AM

Unfortunately, only the simplier “PrivateEye” version of the software is shown on the web site. It’s still an useful demo for the blurring of the screen when you turn away. But leaves one question unanswered…..

What if you walk away from the computer leaving it unlocked? Will it unblur the screen when someone else sits down in front of the computer?

Yes, I know you shouldn’t leave the computer unlocked and unattended. But I suspect more than a few users will feel that it’s acceptable. After all, “The screen is blurred, nothing sensitive is showing so I can take a quick trip to the water fountain, rest room, snack machine, etc.

jeff • July 14, 2009 6:52 AM

@bruce,

I agree its interesting, but I’m not sure I’d want my webcam on 100% of the time. I’m still upset about when they built-in a microphone that I can’t physically disconnect (without opening the case).

Jeff

Versvs • July 14, 2009 6:58 AM

How good will the software be in order to detect that i’m just looking to the computer beside yours?

And making use of that special mindset: Isn’t it annoying that i can prevent you from reading just standing by your side in a casual position?

Edward Kmett • July 14, 2009 7:00 AM

What worries me is that their demo doesn’t seem all that secure. They advertise it as protection for TEMPEST attacks, but it would seem to me that if I can record someone reading it, I can hunt for real English works on the screen / the same word appearing in the same location (unless the garbage is cyclic) and read along with the user.

Clive Robinson • July 14, 2009 7:01 AM

@ Bruce,

The first use of “bio-metrics” for general use I like the sound of 8)

I guess there are a number of questions of identification -v- blur out and user recognition etc

But this one sounds like bugs/features or no bugs it’s a start in the right direction.

JRR • July 14, 2009 7:12 AM

I just went and looked at the demo, and honestly, I don’t have much trouble reading the scrambled version.

AppSec • July 14, 2009 7:12 AM

Must be a pain to use if you want to have someone help you with a problem on your computer….

JRR • July 14, 2009 7:14 AM

Ah, OK, the demo is misleading. The REAL demo with the moving eye is actually garbage. The “this is what the attacker sees” part of the demo is just scrambling letters within the words, and that still leaves it fairly readable.

They need to be more careful when making their demos.

Nick • July 14, 2009 7:35 AM

As with all defence mechanisms, the threat model matters. In this case, the threat model is someone reading over your shoulder, hence they only need to defend against something with a very small focal area that moves around rapidly (i.e. the human eye).

Something which can have the whole screen in focus at once (i.e. a camera) is a different kind of threat that this technique won’t defend against, but using a camera is also still much harder to do discreetly than just looking at the screen.

vedaal • July 14, 2009 7:39 AM

“After a 15-second calibration period in which the software essentially “learns” the viewer’s gaze patterns, anyone looking over that user’s shoulder sees dummy text that randomly and constantly changes.”

how does this work for videos? (or does it at all?)

pixellate the image to the unauthorized viewer?

does a person’s gaze pattern change enough from reading text to viewing videos, that the software will blur even the authorized user from viewing a video
because of a change in gaze pattern from the calibration done using text ?

that said,
it seems like an interesting idea,
and something i’d like to try on a plane flight

(no, it’s not your drink, or motion sickness from the flight, or a TIA,

it’s my software…, really,

look, i’ll disable it so you can see …8^) )

Clive Robinson • July 14, 2009 7:48 AM

What we need to know is how it recognises eyes.

If for instance it uses internal reflection under IR then it will also be able to detect video or other cameras with the focus set at the detectors range or infinity (red eye effect).

HaFue • July 14, 2009 7:52 AM

It still will not help, if someone uses a camera to watch/film your desktop, or just uses a periscope 😉
but the idea is good and expandable

Clive Robinson • July 14, 2009 7:53 AM

There is of course one minor issue to deal with…

Some computers (laptops) put out some or all of the image on the aux video port for projectors etc.

Unless it can reliably detect a connection to the aux video port or disable it then all you have to do to see what the user is seeing is connect a monitor etc to the aux video port….

peri • July 14, 2009 8:04 AM

I could be wrong but it sounds like it tracks where your gaze lands on the monitor and blurs everything outside some box whose bounds are probably adjustable by a user. I have not yet owned a monitor that would be larger than the bounding box; which makes me suspect I wouldn’t be interested in this software.

In a crowded area like a restaurant or coffee shop, I would imagine everyone would be looking at your monitor just because the moving unblurred box is interesting. So you would be subject to an unintentional DoS attack of alerts from the program freaking out.

On a plane, I would imagine the eyes of the people sitting on either side of you would be out of the webcam’s field of view and would thus get to see what interests you on your screen. I would certainly find that amusing on a long flight.

I would certainly find this more compelling if one could dynamically change the center of your monitor’s viewing angle to your eyes and drastically limit the viewing angle when there are prying eyes around.

peri • July 14, 2009 8:21 AM

I just saw the demo and it would seem that instead of a bounding box they use an oval and instead of blurring the screen they jumble letters. I don’t think that changes my mind about the drawbacks. In fact I think Eve would be even more entertained and the gaze of onlookers would be more strongly drawn by all the shifting letters.

Paeniteo • July 14, 2009 8:36 AM

@Clive: “Unless it can reliably detect a connection to the aux video port or disable it then all you have to do to see what the user is seeing is connect a monitor etc to the aux video port….”

However, I guess I would notice if someone tries to connect external devices to my laptop while I’m working at it…

Derob • July 14, 2009 8:46 AM

@Clive 7:53: That thought occurred in my mind as well. It can be worse however. We have an advanced projector in a meeting room down the corridor which ‘features’ a wifi connection to project, rather than a traditional cable. Since it is the only wireless access point in our office my laptop tends to connect automatically to it, and me accidentally hitting a button could start the projection allowing everyone in the meeting room to see what is on my screen… I would not notice…

kashmarek • July 14, 2009 9:17 AM

This just seems like a trick to get cameras on everyone’s workstation or laptop just so they can monitor you, rather than watch for other users looking over your shoulder. Is that really a security problem? After all, most computers can be broken into and the files viewed directly (take FindLaw for example).

k • July 14, 2009 10:34 AM

would someone please hack this so I can move a cursor and/or click using an eye blink or other specific facial tick. track pads/balls are an improvement, as a finger doesn’t weigh much, but I never understood why one would want to have to move significant amounts of meat, pushing a mouse around a mouse pad, in order to manipulate a weightless cursor on a screen.

Bill • July 14, 2009 11:05 AM

Hey, this could (eventually) solve the issue of people looking over our shoulder and so allow us to enter our passwords in clear text (to tie in an alternate subject).

🙂

Jason • July 14, 2009 11:29 AM

@k

I can move my mouse about 3″ and get from the upper left corner of my left dual monitor to the lower right corner of my right dual monitor. The effective resolution is 2560×1024.

I can accomplish it with less movement if I move it faster (since I have acceleration turned on).

I like using a mouse. When I play games, mouse is essential. I tried playing Half-life years ago with a trackball. It was terrible.

Tanuki • July 14, 2009 1:19 PM

I presume it fails spectacularly if the intended viewer happens to be wearing Ray-Bans. Or, in all probability, any other variety of spectacles!

Loadmaster • July 14, 2009 1:29 PM

Since this only does background letter scrambling, it probably can’t be used for the potentially more popular application of playing video games or viewing porn while at work.

sabik • July 14, 2009 2:23 PM

@kashmarek, or to monitor that you not only spent the regulation 15 minutes reading through the new memo from head office but also that you read through it carefully.

Anon • July 14, 2009 4:41 PM

Funny how no one spots the incredible opportunity for DRM. Build this into a mobile reader and you get a “pay per reader” licensing model.

Filias Cupio • July 14, 2009 8:59 PM

Like others, I suspect that a video recording of your screen will provide enough information to defeat the system.

I’ve heard of a similar system for accelerating internet high definition TV transmission. Most of the image is sent in low resolution, except for the bit you are actually looking at now. A (rather major) privacy drawback is that your eye movements are being sent back to whoever is providing the video. All they have to do is display some erotic pictures and your eye movements will reveal whether you are attracted to men, women and/or children.

Clive Robinson: I think the scrambling is done in software, not by the display, so connecting to aux video port would be no more useful than videoing the primary display.

Michael Seese • July 14, 2009 9:01 PM

What about a feature which shoots out a little poison dart when it detects someone looking at my screen? (Woe to the barista at Starbucks who decides to be kind and walk my mocha over.)

BF Skinner • July 15, 2009 4:52 AM

All those banks I walk by that orient their monitors away from the inner office I can see from the street…they need this or something like.

Bill Anderson • July 15, 2009 8:24 AM

Hi Folks, I’d like to try to clear up a few questions that have come up in the comments.

  1. There are two distinct products. PrivateEye is software for consumer and enterprise use. It only requires a webcam to work. It’s designed to stop a range of social engineering threats by reducing opportunities for eavesdroppers to read the screen without getting in the valid user’s way. In the Professional version it has a feature that detects and warns if an eavesdropper looks at your screen by (optionally) opening a little video thumbnail showing his face. Very effective tool for discouraging bad behavior.
  2. Chameleon is a high end security system that uses an additional hardware gazetracker. I want to stress the Chameleon demo on the web is only a poor simulation to give a general idea. The real product is gaze controlled, and is much more clever about replacing real content with false and misleading content on the fly. There is quite a lot of research and technology behind it. If anyone wants to know more about it please feel free to contact Oculis.

Since folks are interested, here’s a discount code that will get you 25% off on PrivateEye. You can get the product at. http://www.oculislabs.com/Products/PrivateEye.html. When prompted enter the ‘Coupon Code’ OCUL-HVFF-INTR for a 25% discount.

Adrian Ramsey • July 15, 2009 9:37 AM

The MPAA and TV companies will love this if it ever gets into TV sets.

“You have someone not registered as an approved watcher viewing $BIG_FILM. Add them to our approved watchers’ list for only $9.99…”

fuchikoma • July 15, 2009 1:04 PM

They forgot something in their demo…

Arocdnicg to rsceearch at Cmabrigde Uinervtisy, it deosn’t mttaer in waht oredr the ltteers in a wrod are, the olny iprmoatnt tihng is taht the frist and lsat ltteer are in the rghit pcale. The rset can be a toatl mses and you can sitll raed it wouthit pobelrm. Tihs is buseace the huamn mnid deos not raed ervey lteter by istlef, but the wrod as a wlohe.

I read the first paragraph of “this is what the attacker sees” scrambled… I’d hope it doesn’t rely on the letters really present in each word!

Billy Crook • July 15, 2009 2:38 PM

I hereby post a bounty of $500 ($1000 if completed in one year from now) for a Free Software Xorg/X11 module that does something similar to this. It must:

1) divide the screen into randomly shaped, moving, borderless polygons. Average the color within them. Soft-blur their edges. Track authorized user’s eyes. Clarify the area said user is looking at.

2) Retain snapshot and/or video of any unauthorised persons faces, and breifly display their picture on an unscrambled, warning bar visible from a distance. (So they know they’ve been spotted) Optionally blur the entire screen at this point.

3) Release under GPLv3 or later

This might actually fit in as some compiz fusion plugin fairly nicely.

Anyone else think this is worth some money to develop? Pitch in your bounty. I don’t think Oculis is interested in pursuing this market/use case. This might change their mind, or inspire someone else to obviate them.

Tom • July 16, 2009 12:33 AM

Why are any of you doing anything remotely sensitive in any sort of publicly-visible area?

What about the peeker watching your keystrokes?

Why on earth do people buy computers with built-in web cams and microphones (both of which have been turned on by remote exploits)? Plug them in if and when you really need them, and when you’re in a safe place.

THINK!

Bjornar B Larsen • July 23, 2009 3:28 AM

So, everything but what you read on your screen is supposedly jumbled around, making reading over the shoulder impossible. I guess how well this works depends on how accurately the equipment can pin-point the gaze, and how unaffected humans are by motion around what we read.

Is it possible to make this work in a more hostile environment? Let’s assume that the goal is not only to protect from a casual co-worker, but also protect from a hostile video recorder that we assume can record everything on screen perfectly. Let’s simplify by assuming the villain video recorder cannot record anything but the screen (not head movement etc.). Let’s say people need to see at least three words at the same time in the sentence they’re reading. Let’s also assume humans unaffected by moving elements around what’s focused on, and perfect measurement of where the gaze falls.

So my eyes (“lense”) will move from left to right, and all of the fake decoy lenses would do the same, confusing the enemy. The task is to prevent the real text from standing apart from the decoy texts, so proper words must be used for the decoy texts. Still, the number of words on screen will be limited to, say 2000. Assuming we’ll have 2000/3 lenses moving, if the reader reads everything in one flow, it should be trivial (by analysis of perfect video) to single out which of the texts contain interesting data worth protecting (a bit harder if the decoy lenses display text that actually make sense, remembering to add the occational spelling error and questionable grammar expected in texts). The occasion where entropy may be introduced is when the reader breaks the normal reading flow and puts his gaze somewhere else on the screen (“jumps gaze” to a different sentence), because the decoy lenses could jump in random directions at the same time. So to conclude, this system may work if:
1. The reader (our hero) jumps gaze very often
2. Our hero doesn’t read the same words several times

If I’m not missing something.

Leave a comment

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.