Detecting Adblocker Blockers

Interesting research on the prevalence of adblock blockers: “Measuring and Disrupting Anti-Adblockers Using Differential Execution Analysis“:

Abstract: Millions of people use adblockers to remove intrusive and malicious ads as well as protect themselves against tracking and pervasive surveillance. Online publishers consider adblockers a major threat to the ad-powered “free” Web. They have started to retaliate against adblockers by employing anti-adblockers which can detect and stop adblock users. To counter this retaliation, adblockers in turn try to detect and filter anti-adblocking scripts. This back and forth has prompted an escalating arms race between adblockers and anti-adblockers.

We want to develop a comprehensive understanding of anti-adblockers, with the ultimate aim of enabling adblockers to bypass state-of-the-art anti-adblockers. In this paper, we present a differential execution analysis to automatically detect and analyze anti-adblockers. At a high level, we collect execution traces by visiting a website with and without adblockers. Through differential execution analysis, we are able to pinpoint the conditions that lead to the differences caused by anti-adblocking code. Using our system, we detect anti-adblockers on 30.5% of the Alexa top-10K websites which is 5-52 times more than reported in prior literature. Unlike prior work which is limited to detecting visible reactions (e.g., warning messages) by anti-adblockers, our system can discover attempts to detect adblockers even when there is no visible reaction. From manually checking one third of the detected websites, we find that the websites that have no visible reactions constitute over 90% of the cases, completely dominating the ones that have visible warning messages. Finally, based on our findings, we further develop JavaScript rewriting and API hooking based solutions (the latter implemented as a Chrome extension) to help adblockers bypass state-of-the-art anti-adblockers.

News article.

Tags: ,

Posted on January 5, 2018 at 9:00 AM • 35 Comments

Comments

Mike • January 5, 2018 9:19 AM

I’m not opposed to online advertising, I’m opposed to abusive/malicious online advertising. Until online advertisers understand that distinction, the adblocker anti-adblocker war will continue.

As a follow-up thought for any lurkers who deploy anti-adblocker technologies: If you block my visit to your site because I use an adblocker, I will just go elsewhere – your content is not that valuable or unique.

Penguat • January 5, 2018 9:37 AM

Right now, I am using an adblocker because I have found in the past that ads are not effectively held to a code of conduct – and there are currently (always?!) some extant vulnerabilities which could be exploited by running untrusted code on my system.

Unfortunately I don’t see a way for advertisers to run only trusted code on my system whilst getting the metrics they have come to expect.

Rhys • January 5, 2018 9:54 AM

This is just a negotiation in process. Attrition is end game.

Individual security & privacy are subverted for the unwitting.

The employers of this coercive technology demonstrate their Values & Ethics which should be a warning to their prospective clients. Not just about this individual act but, their conduct in delivering a product to you.

And they demonstrate the extent of larceny they are willing to visit upon others rather than address the issue of their value proposition. (Which is generally found- not compelling.)

What ever happened to the witticism that ‘if you’re not paying for the product, then you are the product’?

echo • January 5, 2018 10:00 AM

I have used an adblocker for so long now that viewing the internet without an adblocker is a ghastly experience not to mention abusive ads perhaps serving malicious software.

I don’t know where the ad industry believes it is losing revenue or where these lost sales come from. My view is a battery can only hold so much charge and there can only be so many competing drains on that charge. If the capacity isn’t there it isn’t there.

The numbers cited in the paper are interesting. 30% of top sites with active blockers and 90% overall with inactive blockers? These seem like very familiar numbers found in economics and psychology.

Adrian • January 5, 2018 10:34 AM

Even when I’m not using an adblocker, many sites accuse me of doing so, presumably because I have some atypical browser settings, like no 3rd-party cookies.

Grauhut • January 5, 2018 11:28 AM

Its easy for me, if they block me for securing my property, okay, then i dont need them! 🙂

I accept first party hosted text and grafical ads, i can see @bruces Data and Goliath book ad here quite well, no problem.

But i will never ever accept to run untrustable code from unknown third parties just to be enabled to read someones text.

Whoever insist on running bs from shady sites on my systems in order to be able to see his crap is hostile and i dont need hostile sources. There are enough info sources out there.

hmm • January 5, 2018 11:32 AM

That’s a big ol’ link.

“if they block me for securing my property, okay, then i dont need them!”

That’s right. Steve Forbes and the rest can go soak. There’s a way to monetize the content without exposing people to malware or crap they don’t want, frankly the whole ad-network model is lazy.

If they’re willing to spend money trying to trick browsers into doing things the user doesn’t want, then they’re not getting a dime of mine – EVER. NO MATTER WHAT.

No quarter to highwaymen.

Dr. I. Needtob Athe • January 5, 2018 12:27 PM

I don’t understand what the difficulty is in implementing an undetectable ad blocker. Why can’t a browser retrieve an entire web page from a server, display the portions on the screen that the viewer wants to see, and not display the portions that the viewer doesn’t want to see? How can the server detect that?

hmm • January 5, 2018 12:37 PM

Dr. I, 3rd parties are the ones sending the ads.

You download the “website” as you say and it actually opens up connections to some 10-50 other sites, in most cases, where the actual ads come from and the analytic packages and the magic cookies and invisible pixels and canvas scripts and font profilers, etc.

So they each have their own code and logical regimes to make sure they’re being actually displayed and to the profile/person they think they’re targeting, in situ. It gets complicated to fool them *(all, individually) into thinking they’re being displayed when they’re not really. They have countermeasures, analytic javascripts, all kinds.

So it’s not “the server” of the site you’re visiting that “detects” that per se, but since they’re in cahoots with the ad campaigns (duh) they get informed that you’re blocking their monetizing and they would rather kick/piss you off than allow you to willingly give them money for their content, deciding for yourself if it’s worth that to you.

So if you want an undetectable ad blocker you have to short-circuit hundreds of scripts, basically.
What you’re actually suggesting is that they each get reverse-engineered. It’s a huge task.
And they change all the time, hence the ‘arms race’ aspect.

Greg • January 5, 2018 12:57 PM

I mainly use a script-blocker and, sometimes will turn on the script for the primary site.

For example, this site only shows a script for its site. Others I have found will have several and opening one script allows a dozen more, and so on and so on.

As a result there are some sites that are just unreadable and if they go that route I don’t read them. In some cases I’ll use the “Inspect Element” and start deleting certain sections of the page to view it properly.

I also block images, flash, activeX, applets among others. I’ve used this method for years and have never been infected with viruses etc.

mark • January 5, 2018 1:03 PM

I have a simple response to anti-adblocker software: if I land on a page, and it’s blank, and I look at noscript, and the page wants me to allow 20 websites, most of which have the letters ad in their name, I choose to decide that I can find what I’m looking for somewhere else, and close that tab… and I’ll not only not see their ads, but I will tend not to want to ever visit that page again.

Here, have another bite of your nose, guys, your face isn’t spited enough.

Mike V. • January 5, 2018 1:04 PM

I want to step up my ad-blocking game, so last night, I went out and bought my first Raspberry Pi. This weekend, I plan to use it to set up a Pi-Hole, which acts as a DNS server for your network and filters out content from known ad networks. Thus, browser-based ad-blockers are no longer required.

Ads that come from the same domain as the content are unable to be blocked, but there are relatively few sites that do this (YouTube apparently is one). I wonder how (if?) ad networks will effectively counter the Pi-Hole.

Dr. I. Needtob Athe • January 5, 2018 1:11 PM

Mike V, according to the response to me by “hmm”, your Pi-Hole isn’t going to make any difference. You’ll still get complaints about your ad blocker.

echo • January 5, 2018 2:32 PM

I’ve been experimenting with a ram disc to help speed things up and cut down the number of SD writes. This made me wonder… Would it be possible for a browser to buffer a page without ad blocking then in a second step read the page off the buffer with ad blocking? How could they detect this?

Grauhut • January 5, 2018 3:12 PM

@echo: “Would it be possible for a browser to buffer a page without ad blocking then in a second step read the page off the buffer with ad blocking?”

Thats workable, but not trivial. You could set up a proxy chain that includes a prerenderer. You would need to use something like phantomjs as the “victim render engine” and you would filter the html output of this prerenderer afterwards.

I don’t know about a prepackaged solution, but i know this should work.

hmm • January 5, 2018 3:32 PM

“Ads that come from the same domain as the content are unable to be blocked, but there are relatively few sites that do this ”

You are misunderstanding the anti-adblocker scripts.

They detect that the ad is not displaying and don’t allow you to see the content based on that.

If you black hole all ads, unless you’ve somehow (this would be hard) tricked them to run emulated on the Pi instead of your HDCP screen, passing the cookies and script data between or something, such that the scripts THINK the ad is being displayed and gives you the credentials to the content, it’s not going to defeat this problem.

You can block ads or you can get to the protected content as-designed, or the third option is a hack.
Blocking with a Pi or browser ext or routing table or pfsense or hosts.file, it does the same thing.
It does not defeat anti-adblockers to get to the protected content, it just blocks it all.

Jan-Willem • January 5, 2018 3:36 PM

It is not about showing ads. As long as they don’t monopolize my screen, I don’t care so much. It’s about the fact that they follow my browsing, my interests etc. There are even technologies around to follow all my typing and my mouse tracking. As long as these kind of technologies are used, I will use an adblocker.

hmm • January 5, 2018 3:40 PM

Some cheapo popup-style adblock-blockers can be defeated with a click :

Reload the page, but click stop about .5 sec (YMMV/net speed) into the loading.

You get the content because the text and whatnot is cached and comes quickly, but the script checks take a second to load and complete so you can “beat” them to the page and display the content on an “incomplete” page load. If this is the case, a purpose built script can strip that content out and save it before the popup blocks you out.

Ed Hurst • January 5, 2018 4:18 PM

There are lots of ways nobody has mentioned yet. I favor crippled browsers for most surfing, like Links2 from Twibright Labs (they have a Windows version); or you could use Dillo on Linux/Unix. They don’t block content; they simply don’t respond to it — no scripts, limited graphical elements, etc. You do have to get used to reading the simplified page layouts, which includes a lot of scrolling down. Not many websites fail to display at least the text, which is mostly all I care about.

On just about any browser I use, I typically clear the cache before navigating to the next site. You can get really brutal with Lynx (text only), but some sites actively block that. You can twiddle the settings for Lynx and it will lie to the servers. There are other minimal browsers out there.

echo • January 5, 2018 4:21 PM

@Grauhaut

Using phantomjs as a pre-render proxy is a good idea.

I was wondering if a more tightly coupled solution was possible. I did a quick search and couldn’t find anything obvious about the (Firefox) display pipeline and know nothing about what api hooks are available for add-ons. I expect this is none trivial too.

Reader view is also available in a pinch. I keep forgetting this exists.

VinnyG • January 5, 2018 4:41 PM

I must ask, what form does anti-adblocker intervention usually assume? I use Firefox v56 with AdBlock Plus, NoScript, Privacy Badger, Better Privacy, and Request Policy, all of which have some control over ad display and/or script execution. I run NoScript and Request Policy pretty much in lock down mode, where I must explicitly allow individual domains to execute scripts. I do not have any optional whitelisted domains in either add-in (no domains remain that I am permitted to remove.) I do not see ads, with rare exceptions in “walled garden” ecologies where the host domain is the ad source (e.g. Facebook.) I also do not see messages complaining about my ad blocker tech, nor am I blocked from accessing content on sites with any frequency. I admittedly do not have tools that allow me to analyze whether the rare page load failure is attributable to ad blocker detection or something else, but the frequency is low enough that I’m not concerned about it. As others have noted, there are few web monoplies on a particular story or data set – the synopsis nearly always contains sufficient information to find an alternative source. I have found that occasionally when site content doesn’t load, turning off page styles will fix the issue. If that reflects an attempt at ad-blocker blocking, it is pretty lame. I should also probably mention that I frequently leave scripting turned off and scroll down to read unformatted text, and have no issue with doing so. I do recall ~2 years ago seeing some complaints about my ad blocker, but nothing recent. I browse every day, follow many links from news aggregator sites to source sites, and also frequently search for product and service information leading to on-line purchases. Except for the fact that I spend relatively little time on social media, I think I am fairly typical in site requests (non-phone reqeusts, at least.) Am I lucky, or atypical (assuming those aren’t the same thing in my case 😕

hmm • January 5, 2018 4:44 PM

“You do have to get used to reading the simplified page layouts, which includes a lot of scrolling down”

That only works if there is plaintext content somewhere, as opposed to a separately loaded module.
To your point, a lot of sites (media, blogs) and probably most sites are lazy like that.

If they do it deliberately / correctly you don’t get the content until the adcheck is successful.

Michael Vincent • January 5, 2018 9:24 PM

The real advantage of the Pi-Hole is that it will block ads on smart TVs and the like, which aren’t able to use browser-based adblockers. I didn’t mean to imply it could defeat anti-adblockers.

Jim • January 6, 2018 7:03 AM

I work for a publishing company which for the past couple of years has been using adblock-blockers to try to get people to whitelist our sites. Recently we abandoned the process as it just wasn’t working. With programmatic advertising delivering diminishing returns anyway, we are now rethinking our strategy.

Ed Hurst • January 6, 2018 8:29 AM

@hmm

Quite correct; sites that rely on scripted modules to deliver content itself won’t work with some of my favorite browsers. I’m not a developer, only a power user, so I can’t explain it, but often enough I’ve gotten Dillo to display content from some modular sites (for example, PJ Media). Dillo hasn’t implemented Javascript, so I don’t know what magic they use to get it.

Schnagle • January 8, 2018 6:03 PM

The Ad-blocker problem is a moving target. They’re constantly devising new ways to track visitor activity. One of the latest techniques in use involves DNS cookies. Some sites use nothing but DNS cookies (they don’t resolve to anything and aren’t supposed to resolve – they carry a unique tracker/content ID, and that is picked up by the DNS server and simply correlated with the associated web server data in order to make a complete picture.

The easiest ad blocker is your own. Simply run a proxy of some kind (maybe Squid) – and collect all the urls in its log. Run a local caching DNS resolver like Unbound and load it with the urls gotten from the proxy. Make most of the domains for the urls point to a black hole (can be done with Unbound), and let Unbound cache the rest, so they get no DNS fingerprint at all. This can be up-to-date without relying on third parties, and cost nothing. Also may be somewhat better in soft-fuzzy security feeling, by not using 3rd parties.

Fredric • January 9, 2018 12:24 PM

“… to remove intrusive and malicious ads…”

Most ads you see today on the Internet are blatant frauds, quack medical frauds, “free energy” frauds, you name it, almost all ads are frauds, THAT is one major reason why people block all ads.

hmm • January 9, 2018 6:02 PM

@ ED H

Thanks, Dillo is a nice little thing. No javascript at all would be a smart feature. (I wonder how..)

Halatinous Himself • September 21, 2021 7:26 PM

MITM proxy for nuanced adblocking bypassing NX or nulled connection detection [by ads serving platforms]

Hosted for self or as a service to receive spoofed domain requests FOR PRIVACY.

https://mitmproxy.org/

Implementing this will provide more sophisticated users fine grain power to circumvent “blocking” detection by using DNS resolution spoofing and “content” manipulation. Yes, there are privacy concerns as a paid service is a shift of trust.

.

A basic implementation will serve transparent pixel gif for image requests, else a valid empty html document. Default Spoof HTTP header (200 ok). Generate targeted SSL cert. Provide options for other web status codes like redirect flavors, or errors. Combine with content filter rewriting for best effect. NextDNS provides resolution spoofing in the settings menu labeled redirect. It is not url redirection.

This is a war of attrition that collaborative genius will win every time even against cost no object adversaries like google.

Leave a comment

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.