Identifying When Someone Is Operating a Computer Remotely

Here's an interesting technique to detect Remote Access Trojans, or RATS: differences in how local and remote users use the keyboard and mouse:

By using biometric analysis tools, we are able to analyze cognitive traits such as hand-eye coordination, usage preferences, as well as device interaction patterns to identify a delay or latency often associated with remote access attacks. Simply put, a RAT's keyboard typing or cursor movement will often cause delayed visual feedback which in turn results in delayed response time; the data is simply not as fluent as would be expected from standard human behavior data.

No data on false positives vs. false negatives, but interesting nonetheless.

Posted on March 9, 2015 at 1:03 PM • 19 Comments


MarkMarch 9, 2015 1:09 PM

" ... will often cause delayed visual feedback which in turn results in delayed response time; the data is simply not as fluent as would be expected from standard human behavior data."

Or the user has had a couple of drinks.

DaveMarch 9, 2015 1:19 PM

Mark: in that case, once again the user probably shouldn't be operating the computer. :-) I can see someone installing this as a "save me from myself" measure....

ArthurMarch 9, 2015 1:46 PM

One must wonder what the false positive rate would be in real-world use, with slow internet connections and connections from 3G smart phones.

Mark WMarch 9, 2015 2:35 PM

So then a slightly more sophisticated RAT sits around collecting the same metrics and emulates them... doh.

AndrewMarch 9, 2015 4:50 PM

Why someone operating a RAT whould start typing or moving the mouse to get disclosed when he can get the file list then draw any file? Or get the typed characters list recorded by keylogger, so the passwords?
The only reason whould be to impersonate the user, like sending some mails or other nasty actions when he left the computer for a while... One can easily can get in big troubles this way...

FrancisMarch 9, 2015 8:46 PM

So people who are slow in operating computers (seniors for example) would be misidentified as a RAT? Keep in mind "Standard" is subjective and not everyone falls into the same range of response time.

jdgaltMarch 9, 2015 11:00 PM

The trouble with this test is that people with various handicaps (bad eyesight, coordination, etc.) will always register as false positives.

No Such AgencyMarch 10, 2015 2:58 AM

Instead of analyzing biometrics, if the computer in use is usually accessed via keyboard/mouse, then why not simply compare the software buffer to the hardware buffer?

Any inputs that are emulating keyboard/mouse inputs will not be triggering hardware inputs, so you can immediately see that software generated inputs are occurring.

If the system is to be used via a network, that gets harder to deal with.

Best method is to be paranoid, and log everything (or at the very least, date/time of any actions triggered by any form of user based input).

After that, "if they're in your machine, it's not your machine anymore".

As always, defense in depth.

65535March 10, 2015 3:15 AM

This is an interesting project but it will need some work.


False positives will be a problem. Further, a faulty piece of hardware, a corrupted driver, or a memory hogging program would have to be checked and removed from the false positive list.

Next, what happens when a true “Remote Access Trojan” is identified? Is the computer shut down – or just a pop warning issued? Is the wireless access card deactivated [possibly along with the NIC]?

Is there a way to roll-back the computer to a “good working” image is the know latency base line when a RAT is found[or what ever they call it]? Is a re-format of the HDD or total junking of all hardware necessary?

UhuMarch 10, 2015 3:34 AM

Too all the commenters saying that false positives would be because of a slow user:

I can't comment on the mouse part, but have you ever used SSH over a slow connection? I usually type the command just as fast, then wait for visual feed-back before hitting enter. So if you measure average typing speed and get a high typing speed, you know that the person using the computer is proficient at it and not drunk or old. An attacker probably knows what she is doing.

seheMarch 10, 2015 4:17 AM

Or... you can train a classifier with the actual metrics of the main user(s) and say goodbye to false positives.

You can do a guaranteed-local thing (change vt to a console and prompt, present local lockscreen on other OSes) to allow the local user to manually override and unlock.

In my opinion it could be easy to make this usable. However, it's unclear to me what the gains would be. Unless a RAT would be "likely" to happen. Perhaps that is the case for an average computer user - I don't have the data on that

mike~ackerMarch 10, 2015 8:24 AM

instead of guessing you could just run a software inventory,-- check the CRC on all the installed code. this has to be done using an alternate o/s such as a Linux "live" stick .

cbMarch 10, 2015 11:12 AM

Even assuming a perfect implementation - good code, no false positives, no missed RATs, modelling the activity of the actual primary user instead of some theoretical average user - this is doomed because it has to run on the system. You know, the system that was compromised so badly that keyboard, mouse and screen activity are under remote control. Evading detection would be trivial with that level of access.

marsupialMarch 10, 2015 11:31 AM

A few online payment / banking sites already use this technique and it's a RPITA for those of us who use password managers.

SomebodyMarch 10, 2015 2:45 PM

What happens when the user's computer decides it's time to index the disk/install updates/recalc a spreadsheet/play a dancing baby video in the background?

These things can introduce jitter in my response time on the order of seconds.

Leon WolfesonMarch 10, 2015 6:49 PM

One key use for this would seem to be collecting a signature from the individual user, and demanding additional verification measures if there's a significantly different usage pattern after the initial period.

Scott "SFITCS" FergusonMarch 11, 2015 5:50 AM


So people who are slow in operating computers (seniors for example) would be misidentified as a RAT? Keep in mind "Standard" is subjective and not everyone falls into the same range of response time.

I suspect that determining a RAT would be a comparative process - if so the usual (local) operator would be the "Standard" compared against. In which case the process would be:- useful; subject to a lot less mis-identifications than you propose.

I believe it would identify a remote screen/VNC session - but would it identify a tmux session? I suspect not.

It's also likely(?) that in order to identify the standard (local) user the program would not only measure the speed at which individual words or letters are typed - but also typing patterns. This process is nothing new and likely already deployed by three letter agencies to try and identify anonymous intertube users.

I don't know if various remote access tools can be identified by buffer sizes or whether tmux is unique in how it display results of keystrokes before they are remotely displayed.

Given the likely-hood that 5eyes can monitor not only at a backbone level but also at the ISP level this capability might be somewhat redundant if keystrokes/mouse movements/screen updates to and from a RAT would be matchable to those from the local machine unless the encryption included padding(?).

If I used a standard blowfish encrypted tmux session to use a remote machine as a proxy - could monitoring of that remote machines ISP and my ISP identify the session?

DrewApril 22, 2015 1:47 PM

It sounds like someone who is not good with a computer could be confused as a RAT but still interesting.

Leave a comment

Allowed HTML: <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre>

Photo of Bruce Schneier by Per Ervland.

Schneier on Security is a personal website. Opinions expressed are not necessarily those of IBM Resilient.