Using a Smartphone's Microphone and Speakers to Eavesdrop on Passwords

It's amazing that this is even possible: "SonarSnoop: Active Acoustic Side-Channel Attacks":

Abstract: We report the first active acoustic side-channel attack. Speakers are used to emit human inaudible acoustic signals and the echo is recorded via microphones, turning the acoustic system of a smart phone into a sonar system. The echo signal can be used to profile user interaction with the device. For example, a victim's finger movements can be inferred to steal Android phone unlock patterns. In our empirical study, the number of candidate unlock patterns that an attacker must try to authenticate herself to a Samsung S4 Android phone can be reduced by up to 70% using this novel acoustic side-channel. Our approach can be easily applied to other application scenarios and device types. Overall, our work highlights a new family of security threats.

News article.

Posted on September 5, 2018 at 6:05 AM • 27 Comments

Comments

WarrenSeptember 5, 2018 8:30 AM

This is a pretty strong argument for fingerprints, facial recognition, and things like Steve Gibson's forthcoming proposed SQRL (https://www.grc.com/sqrl/sqrl.htm)

Impossibly StupidSeptember 5, 2018 10:13 AM

This isn't an argument for anything, let alone biometric identification, which is even easier to "eavesdrop" on. Everything has side-channel attacks. The only real question is how to mitigate them.

I wouldn't even consider this to be a "new family" of attacks, since hand positions and/or keyboard input has already been inferred from audio input, and even WiFi signal strength.

DentonSeptember 5, 2018 12:34 PM

"unlock patterns...can be reduced by up to 70%"
That doesn't sound terribly accurate. Correctly guessing just one number in a four digit pin reduces the search space by 90%. It's impressive that they can get any information at all though.

Gunter KönigsmannSeptember 5, 2018 2:29 PM

My desktop speakers always make noise that seems quite specific for the task the CPU is busy with. Perhaps that would be a more promising side channel.

But: normally smartphones and such use delta sigma ADCs and DACs. If one could configure their digital filters to have higher limiting frequencies I would bet one could get a much higher resolution sonar and an ultrasonic communication channel with a high bandwidth: Smartphone microphones and speakers are small enough to feel like they would work fine with ultrasonic sounds...

HmmSeptember 5, 2018 2:38 PM

"The only real question is how to mitigate them."

Ultrasound emitters worn around your neck and from your wristwatch that chirp modulated garbage.

Saturate the channels. If they're getting low-ish results like 70% this is no doubt sensitive to it.

Clive RobinsonSeptember 5, 2018 6:07 PM

@ echo,

I wonder if it's possible to 3D map space around the phone.

It depends on what you mean by "map" but yes there are ways of transmitting signals to get range information by chirping frequency, phase and code.

@ hmm,

Ultrasound emitters worn around your neck and from your wristwatch that chirp modulated garbage.

There are ways of removing jaming signals using MIMO techniques. But you can also use the jaming signal to provide information as well.

The point to remember is that with the likes of unlock patterns and passwords are not "one time" events, thus you can have many bytes at the apple and average out the jaming.

That is for any number N of the patterns when averaged together the wanted signal goes up by N and the noise only by sqrt(N). Thus for sixteen averages you have effectively taken the jamming signal down to a quater of what it is in a single reading.

echoSeptember 5, 2018 6:29 PM

@Clive

It depends on what you mean by "map" but yes there are ways of transmitting signals to get range information by chirping frequency, phase and code.

I have no idea what the resolution would be. I just wondered if this technique could be used like FaceID or Kinect to grab a 3D image of what is near the phone. If so I imagine the image would be fairly coarse. Depending on time maybe a finer image could be built like the ghost image thingy?

They did character detection with desktop speakers and keyboard didn't they?

I spotted a data analysis topic the other day which follows on but will place this in the squid topic.

Jesse ThompsonSeptember 5, 2018 6:38 PM

@Waren: I had to go look up SQRL. I think that QR code ⇒ mobile app ⇒ back to web server is an interesting approach to explore, but this Gibson guy apparently still hasn't released an implementation after like 5 years.

It does sound like it would defeat all "eavesdrop on the users' fingers"-related side channels though at least, and it's cost for adoption is potentially less than for HOTP/TOTP smartphone apps.

Clive RobinsonSeptember 5, 2018 7:20 PM

@ echo,

I have no idea what the resolution would be. I just wondered if this technique could be used like FaceID or Kinect to grab a 3D image of what is near the phone.

Whilst you could get a reasonable range estimate the problem is getting direction. Using two speakers and a single mike, you could build up some information but it would have a cardoid type error response (squashed figure of eight).

To get something acceptable you would need a third source or second detector.

Alyer Babtu September 5, 2018 7:52 PM

I’m thinking a double walled glass chamber with the inner wall acoustically (vacuum ?) and vibrationally isolated from the outer wall and so the larger exterior; put hand and mobile device inside, seal it up, and swipe away. You can attach a lanyard for carrying purposes.

JoeSeptember 5, 2018 10:12 PM

I would prefer biometric and log in code. I don't use biometric (fingerprint) because LE can require me to open the device. But I wish I could use a fingerprint with a log in code. I think it would make it a little more difficult to be broken into.

ArclightSeptember 6, 2018 12:54 AM

It seems like the "Scramblepad" feature that randomizes the PIN pad key locations every use would largely mitigate this attack. LineageOS on Android implements this.

Erdem MemisyaziciSeptember 6, 2018 12:56 AM

Yea, so why don't we all carry a powered microphone, speakers, and GPS on us 24/7? We don't need hard switches on those modules, they're probably fine, software will keep it safe. Keep'em all powered while not in use, it'll grease up the battery industry as well. *sigh*

Clive RobinsonSeptember 6, 2018 5:09 AM

@ Alyer Babtu,

I’m thinking a double walled glass chamber with the inner wall acoustically (vacuum ?) and vibrationally isolated from the outer wall and so the larger exterior; put hand and mobile device inside, seal it up, and swipe away

I think what you are trying to describe is a very wide necked thermosflask for taking hot meals or cold deserts to work in.


There is a problem with the idea,

    Speakers are used to emit human inaudible acoustic signals and the echo is recorded via microphones, turning the acoustic system of a smart phone into a sonar system.

You need to read it backwards to realise what the actually mean aboit the speakers and microphones.

The point is it's the phones speakers and mics that are being used so they would all be in the thermos with your hand...

Also it's not exactly a drop in your pocket item, nor hand/man bag, which is why such items usually come with a fairly large handle attached to the screw top lid or side. Also they are quite fragile at the best of times, aand you would not want your hand in there when it implodes and then shards effectively explode all over the place...

Alyer Babtu September 6, 2018 11:29 AM

@Clive Robinson

the phones speakers and mics

Thanks for the correction. I indeed completely missed the essence.

echoSeptember 8, 2018 4:03 AM

@Clive, @Erdem Memisyazici

Also it's not exactly a drop in your pocket item, nor hand/man bag, which is why such items usually come with a fairly large handle attached to the screw top lid or side. Also they are quite fragile at the best of times, aand you would not want your hand in there when it implodes and then shards effectively explode all over the place...

I know smartphones are easy and tend to encourage laziness which makes paper based maps redundant but service can be disrupted and batteries can go flat. I bought a waterproof smartphone bag and some ziplock type bags to contain my smartphone during wet weather and contain paper maps among other things. I have a biggish compass but may buy a small compass too.

As well as the small umbrella which is an everday carry in my handbag I also bought some very light plastic raincoats too including disposable raincoats which fit in a case the size of a golfball. Just in case I also bought disposable plastic ankle high shoe covers.

Clive RobinsonSeptember 9, 2018 5:12 PM

@ echo,

... Just in case I also bought disposable plastic ankle high shoe covers.

OK that will get you through an average British Summer, but what have you got for after the biblical fourty days and forty nights, we had a few yrars ago?

Somebody I know has an inflatable canoe in the back of their car, that they actually use quite a bit.

But the point is that technology it's self is a problem. If you have a watch you can use it as a compass almost anywhere away from the equator and you can use a compass to make a sundial clock. So in a day you have the basis for working out your position on the earths surface and thus be able to navigate your way home. Whilst a mobile phone has a clock and GPS it's battery is to short lived to be of any real use.

The greed of phone manufactures is such that you can not change the battery on most smart phones these days and those external chargers are very inefficient. Worse some do not function as smart phones without a SIM and working network due to "Cloud Must be Present" issues of the software[1].

Because of this "every things a phone now" idiocy carrying anything other than a wallet and phone is deemed suspicious.

The thing is when I was a youngster carrying a pen knife string and other usefull bits was not a reason to be held by the Police on suspicion of "going equiped to commit a crime" as it is these days in the UK... Under the EU Treaty the right of free trade actually covers you to carry a toolkit, but UK Police especially in the Met will still fight you all the way to court having taken your tools away. And they will conveniently loose or break them just so you get the message.

[1] I won't name names but some phones have "online predictive spell checkers" that are I suspect an excuse to send every key press you make "Back to the Mother Ship" in China etc. The native applications do not work unless network connected because of this, and you actually have to change the software keyboard to stop the ET Phone Home behaviour even with non native apps...

Alyer Babtu September 9, 2018 8:01 PM

@Clive Robinson

battery is to short lived

A few smartphones still have removable main batteries, e.g. soon (maybe) releasing Librem 5. Some people make a gesture towards security by removing the battery. But even if the battery is removed, there is still some power in the phone, e.g. to run the clock. Can this power be hacked to nullify the effect of the removal ?

echoSeptember 10, 2018 4:40 AM

@Clive

Apart from me i seems nobody noticed the cop, who tazered the old lady for carrying a knife she used to gather dandelion, was carrying a pocket knife.

The UK breaks a lot of European law on the sly before we even count the UK state breaking its own laws.

The Uk is brainwashed. I havealready chatted with a neighbour and told her about the last incident. She basically said they have a haye thing going. I also told her I am plannign to leave the UK and claim asylum when I can. It's a struggle because a lawyer became abusiveand threatening and stopped me getting a document counetrsigned I need to get my paperwork in order. This has caused weeks of ill ehalth. I'm also strugglign to afford the travel. I wish things weren't coming to this but I can't cope.

Apart from staving off depresison this blog is no help. As and when I think I will just disappear.

Clive RobinsonSeptember 10, 2018 5:18 AM

@ Alyer Babtu,

Can this power be hacked to nullify the effect of the removal ?

It depends in part on the circuit layout.

Years ago it would have been one battery for the Real Time Clock chip and in way more expensive computers a small part of the memory such as a single CMOS SRAM chip. In both cases carefully designed to stop other parts draining the battery.

The advent of the IBM PC brought the RTC and SRAM chips together under a single battery. Since then more and more devices including full microcontrolers have been brought under the same battery.

But batteries have poor charge life for various reasons and thus first rechargable batteries and now "Super-Caps" are used.

What often happens is the microcontroler is not turned off but put into a low power state often called "sleep mode" which can be woken by a hardware interupt.

But because of the very integrated nature of such microcontrolers software can change the way the interupts work thus the microcontroller can be brought out of sleep mode fairly easily.

What can then be done depends entirely on the capacity of the backup power device. The larger the capacity the more that can be done, likewise the less power drawn the more that can be done.

Usually physical size then price determins what capacity the backup power device has. Which in many cases makes it very small thus not much can be done...

However not much is relative when you consider the likes of car key fobs and other "key chain" devices that will run for years on a single hearing aid sized battery. But also consider medical electronics, those hearing aids now have DSP functionality that runs continuously in them. With a level of computing power alone that back in the 1980's people would have killed for in the defence sector.

Clive RobinsonSeptember 10, 2018 7:37 AM

@ echo,

Apart from me i seems nobody noticed the cop, who tazered the old lady for carrying a knife she used to gather dandelion, was carrying a pocket knife.

As I mentioned a few days ago there are loop holes that go back to the early 1800's where by all that is required for you to be arrested is a police officers suspicions.

The Met Police for instance think this is OK to delegate this to civilians with no training at those moronic "bag check" security points.

A few years ago I happened to be carrying a small tool kit which contained a pocket knife with a blade well within the supposed legal limit. An idiot on a security check point who could barely speak two words of english grabbed my tool kit and as far as I'm concerned stole it. His boss called the police when I tried to retrieve my property and some jumped up Community Police officer turned up. Who had at that time no powers of arrest or for that matter right to collect and hold evidence. This idiot did not know the difference between inches and centimeters and new little or nothing about the law. Unfortunatly instead of doing what I should have done which was retrieve my property and leave I made the mistake of waiting for a police officer with appropriate lawfull authority. From which point it all went down hill.

So no it does not supprise me in the slightest. It is all driven by political incompetence that derives very specifically from the curent UK PM when she was Home office minister.

I know I will probably upset you when I say that she and the previous female prime minister are examples of the certain sorts of people who should never ever be given any kind of power. But worse than that the pair of them have made life incredibly difficult for other female politicians who in effect just get tarred with the same brush, due to the pairs prominence and obvious failings writ large by the media world wide (thus giving impetus to prejudice).

As for your own issues whilst I sympathise a lot having been through a little of what you have been. Other than offering sympathy and very general advice there is little that I or others can do. As you yourself have noted for legal reasons you can not talk about specifics or in some cases even generalities. This sort of gaging as you might have noticed is one way that the authorities have a significant advantage, in that they are given way way greater lee way on public commenting than those actually personally involved in cases.

Unfortunatly there are one or two commenters hear that take advantage of their effective anonymity to make at best unhelpfull if not down right derogatory remarks. You can fairly easily spot them because either they are hiding behind a new "nom de plume", or have been warned in the past by @Moderator for their behaviour. I see that one has poped up already in the past few hours on another thread, that you have probably seen.

The one thing I have learnt in life is that those who abuse their position, are generally venal in nature and quick to take punitive action to protect not just their possition but their rice bowl. As a general rule they have a vengfull and thugish nature only occasionaly not expressed physically when pushed beyond their immediate cognative limits. That and an inflated feeling of self worth, few if any of these people when in authority, get the censure not only they deserve, but that also acts as a cautionary tale or deterent to others that are likewise minded.

Unfortunatly as you have seen "guard labour" in the UK is a job that few who can find alternative employment would even remotely consider doing. Especially as the current UK PM over sat the defenestration of nearly all police officers that had acted as a balance against the "canteen mentality" types. Another of her cronies and their predecessors over saw a similar scytheing of the armed forces. Thus ripping out needed experience and replacing them with "rent a thug" types that are effectively unemployable in society. Which in effect is a return of the "Black and Tans" and "Auxies"[1] which history shows is an incredibly bad idea. As it could easily become another tie up between guard labour and terrorist organisations again or much much worse corporate thuggery. There is mounting evidence against the Met Police for the latter activities, where they have handed over what is in all probability illegal surveilance to corporations. Like Sony and various construction companies where it ends up in quite illegal databases and some very unplesant organisations like Kroll Ascociates. Thus having dire consequences on many quite peacfull and moderate people who have minimally excercised their right to legitimate peacefull political process. I know that I've ended up in more than one of these data bases because the Met Officers realy are not that bright or cautious in their behaviours.

You are far from the only person who wants to leave the UK and take up EU member state citizenship, due to seeing what you are seeing. However the EU does not want UK citizens judging by the barriers they are putting up due to the stupidity of Brexit[2] and UK Government Departments[3].

But the fact that some of what you post does apparently get ignored is that this blog started out as a highly technical blog that developed into one on a slightly more general security blog. However in the past few years various political interests have used the very open nature of this blog to try and push certain agendas that the older blog members know from sometimes bitter experience turn into battle zones. Whilst our host is more permissive than many he has had to clamp down sometimes quite hard. An unwanted but necessary consequence has been a "chilling effect", the number of posters has considerably diminished some to other invitation only or heavily moderated technical blogs others to just lurking and I assume some have left altogether. Thus many are cautious or sensitive about subjects that are contentious or to far off topic.

[1] The behaviour of the Auxies was absolutly deplorable and the Black and Tans whilst not exactly innocent did get blaimed for many of the Auxies violent criminal actions. Whilst both were suppodadly disolved the Auxies behavioir and tactics got subsumed into the Royal Ulster Constabulary (RUC). In the 90's under Ronnie Flanagan due to politicians the RUC effectively "arms lengthed" the Auxies style behaviour to the Ulster Volunteer Force (UVF) that was a known terrorist organisation, with not unrxpected results. Ronnie in effect got rewarded for his sedition activities. Other members of the RUC close to him got simillarly rewarded for their sedition activities. One went on to the Met Police where he totally mismanaged Operation Ore, and sickeningly is still proud of all the harms he caused not just to adults but the child protection services and the children they were trying to protect. At best they were not just pridefull and sinfull but also seditious and dangerous criminals in all but name.

[2] Way to many UK MP's are secretly glad about Brexit you can see it in their voting record and lack of support for the EU. Worse many deliberatly conflate the EU with ECHR in the publics eye to thus get anti-EU sentiment, likewise UK Government departments have for many years quite deliberatly implemented EU legislation in the worst possible ways[3]. That's not to say I'm all for the EU, there are parts such as the Council of Ministers that need to be cut out like the cancer they truly are, likewise those that are growing fat off of the extrodinary rates of corruption in the EU much of which has lead directly to the Euro Crisis. But hidden away behind this is a master economic plan that is little different to that formulated by Nazi Economists, who continued to have significant influance long after WWII ended.

[3] Just one example was DEFRA implementing the worst "slaughter house" rules they could, thus decimating the UK independent slauter houses. The results of which were not just BSE, but Foot and Mouth, and the Horse Meat scandals as a directly linkable result. Oh and the effective decimation of British Farming that is not controled by Agro-Corp.

Alyer Babtu September 10, 2018 12:41 PM

@Clive Robinson

one battery for the Real Time

Or one battery to realtime them all ...

It seems then someone wanting confidentiality would not want to put much reliance on removing the mobile device’s battery. And from the ubiquity of powered devices, it would be hard to find an “energy gapped” and unobtrusive spot anywhere, no matter where one goes or what one does. Traffic analysis will find one out.

echoSeptember 10, 2018 9:19 PM

@Clive

I can't disagree with the overall content of what you say.

There is a critique of Theresa May not unlike the article I posted explaining the background to Serena Williams outburst. The article on Theresa is much less orgiving and does break down her personality and record very well which exposed the gendered psychology behind everything. It also exposed the behind the scenes attitude of men who are not wasting an opportunity to peddle their cynical agenda. If this critique was written by a man about a man it would likely be a front page item. Because it is about a woman and requires very careful reading through and consideration I don't beleive it will achieve prominence. Not only would any discussion turn into a polarised squabble the cynics behind the throne are a toxic mess well adept at manipulating attention and misdirection much like they were with Margaret Thatcher.

The state individual (and others who colluded) wereacting outside their competence. I actually have an audio recording of one meeting where I warned one individual of this and talked through policy and the expert authorities and regulating authorities before we discussed the item on the agenda. Needless to say I was bullied out the door. This happened on a second attempt with another person. I have audio recordings of both. I know this particular state institition has been accused of this in the past which they have denied in court and were cleared. My audio recordings carry the proof they are guilty of gross professional misconduct on a systemic and individual and persistent basis. While this caused me a huge degree of personal distress and a tenfold increase in problems to solve and a lot of personal harm and loss other people have died because of this.

I could post on obscure "invitation only" forums but they are small and many have themselves suffered from abuse. Under those circumstances people can rarely see the big picture. People who do have money to buy their way past problems or leave the country as some in the past have done have expressed horror at how bad things can be, so I have been told. Pretty much any feedback about how things really are was erased from a series of public conferences on this issue. The media never report the quality or quantity of low level abuse.

I have obtained a letter at some personal and financial cost to myself a letter by an accredited expert (who has a high public profile) who notes abuses and unlawful behaviour he is familiar with which happens within the state system. Like Bruce they are one of those people who if they want they can write an article and it is published by a national newspaper. Unfortunately, they have never shown or expressed any inclination to do so. The fact their predecessor was publicly ruined by a handful of establishment with a rice bowl to defend and carrying personal grudges may have something to do with this.

I am aware of the DEFRA issue. It's a personal annoyance of mine too. I agree. The problem I have isn't obviously or strictly comparable but structurally similar. By chance a recent immigration case showed the lawyers were trying to develop this kind of "code equivalence" argument but the judge nitpicked their example into the ground and unravelled it. I can think of at least one hugely damaging case where a judge tried to be clever like this before.

I'm on a couple of lists too. Both have since been ruled unlawful. I have been advised I am on a third list by a person who was told by someone in the security services. They were not told why but I suspect as an asset or a threat. I can only suppose the motivation is paranoia or fantasy, or misogyny.

Oh, I don't know what to do. I am dealing with idiots who don't know what they are doing, and have the expert authority to prove it because this is exactly what they said although much more diplomatically.

Leave a comment

Allowed HTML: <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre>

Photo of Bruce Schneier by Per Ervland.

Schneier on Security is a personal website. Opinions expressed are not necessarily those of IBM Resilient.