Schneier on Security
A blog covering security and security technology.
« Identity Theft and Methamphetamines |
| A Minor Security Lesson from Mumbai Terrorist Bombings »
July 13, 2006
Click Fraud and the Problem of Authenticating People
Google's $6 billion-a-year advertising business is at risk because it can't be sure that anyone is looking at its ads. The problem is called click fraud, and it comes in two basic flavors.
With network click fraud, you host Google AdSense advertisements on your own website. Google pays you every time someone clicks on its ad on your site. It's fraud if you sit at the computer and repeatedly click on the ad or -- better yet -- write a computer program that repeatedly clicks on the ad. That kind of fraud is easy for Google to spot, so the clever network click fraudsters simulate different IP addresses, or install Trojan horses on other people's computers to generate the fake clicks.
The other kind of click fraud is competitive. You notice your business competitor has bought an ad on Google, paying Google for each click. So you use the above techniques to repeatedly click on his ads, forcing him to spend money -- sometimes a lot of money -- on nothing. (Here's a company that will commit click fraud for you.)
Click fraud has become a classic security arms race. Google improves its fraud-detection tools, so the fraudsters get increasingly clever ... and the cycle continues. Meanwhile, Google is facing multiple lawsuits from those who claim the company isn't doing enough. My guess is that everyone is right: It's in Google's interest both to solve and to downplay the importance of the problem.
But the overarching problem is both hard to solve and important: How do you tell if there's an actual person sitting in front of a computer screen? How do you tell that the person is paying attention, hasn't automated his responses, and isn't being assisted by friends? Authentication systems are big business, whether based on something you know (passwords), something you have (tokens) or something you are (biometrics). But none of those systems can secure you against someone who walks away and lets another person sit down at the keyboard, or a computer that's infected with a Trojan.
This problem manifests itself in other areas as well.
For years, online computer game companies have been battling players who use computer programs to assist their play: programs that allow them to shoot perfectly or see information they normally couldn't see.
Playing is less fun if everyone else is computer-assisted, but unless there's a cash prize on the line, the stakes are small. Not so with online poker sites, where computer-assisted players -- or even computers playing without a real person at all -- have the potential to drive all the human players away from the game.
Look around the internet, and you see this problem pop up again and again. The whole point of CAPTCHAs is to ensure that it's a real person visiting a website, not just a bot on a computer. Standard testing doesn't work online, because the tester can't be sure that the test taker doesn't have his book open, or a friend standing over his shoulder helping him. The solution in both cases is a proctor, of course, but that's not always practical and obviates the benefits of internet testing.
This problem has even come up in court cases. In one instance, the prosecution demonstrated that the defendant's computer committed some hacking offense, but the defense argued that it wasn't the defendant who did it -- that someone else was controlling his computer. And in another case, a defendant charged with a child porn offense argued that, while it was true that illegal material was on his computer, his computer was in a common room of his house and he hosted a lot of parties -- and it wasn't him who'd downloaded the porn.
Years ago, talking about security, I complained about the link between computer and chair. The easy part is securing digital information: on the desktop computer, in transit from computer to computer or on massive servers. The hard part is securing information from the computer to the person. Likewise, authenticating a computer is much easier than authenticating a person sitting in front of the computer. And verifying the integrity of data is much easier than verifying the integrity of the person looking at it -- in both senses of that word.
And it's a problem that will get worse as computers get better at imitating people.
Google is testing a new advertising model to deal with click fraud: cost-per-action ads. Advertisers don't pay unless the customer performs a certain action: buys a product, fills out a survey, whatever. It's a hard model to make work -- Google would become more of a partner in the final sale instead of an indifferent displayer of advertising -- but it's the right security response to click fraud: Change the rules of the game so that click fraud doesn't matter.
That's how to solve a security problem.
This essay appeared on Wired.com.
EDITED TO ADD (7/13): Click Monkeys is a hoax site.
EDITED TO ADD (7/25): An evalution of Google's anti-click-fraud efforts, as part of the Lane Gifts case. I'm not sure if this expert report was done for Google, for Lane Gifts, or for the judge.
Posted on July 13, 2006 at 5:22 AM
• 39 Comments
To receive these entries once a month by e-mail, sign up for the Crypto-Gram Newsletter.
Clickmonkeys? Commiting fraud? You are not serious and I just forgot my sarcasm detectors, right?
Can't the bad guys write a computer program that automatically fills out the surveys?
The military have been trying to solve problems of this type since the second world war, and I am not aware of any reliable solutions yet.
In fact the issue goes further than the person being present, how do you know they are the right person, not somebody with the right credentials (which may be undetectable fakes).
I beleive the conclusion was certainly for the control of high risk/value items (think nukes) not possible, therefore other means where required.
"and isn't being assisted by friends" or foes.
Yes... i discussed this many times, and i always disadvise companies to use Google adwords because google uses a very bad network in which the ads are being presented.
Webmasters put up a website place the adsense ads they earn some money with it, nothing wrong with that, but sometimes users are being forced to click on them. This is what i call Adsense Spam. Go look arround, many sites are forcing users to click on the ads, like make the whole document clickable etc.
And what about the users who sympathise with a webmaster?, that they click on some ads: "just to support the webmaster"
Really, there are better advertisement programs and if i may give some a tip: use overture.
Bruce, clickmonkeys is a hoax site. Do you think there is really a Ukranian tanker full of monkeys anchored off the coast?
The pranksters other site can supply you with freshly clubbed baby seal, or a bbq at a zoo. They'll cook up a gorilla for you. Not.
"Trusted" computing supposedly includes provisions for authenticating devices like keyboards, to ensure that input comes from a human, not from software simulating input. And that precious, licensed A/V goes to "trusted" hardware only, and not to "unsafe" software like virtual devices that might dump the data streams to a file. All only for the good purpose of saving us from evil trojans, of course.
You could argue that Macrovision has served a similar purpose for over 20 years: to ensure that the VCR is not connected to a recording device, but to a screen presumably watched by humans.
I just ran a test. I went to a legit news story about the major league all star game. The google ad box was running text ads for all star tickets. The game was on Tuesday. They are still selling tickets on-line Thursday. The airport was jammed on Wednesday, authenticating people who were missing flights home.
I guess you need to buy a Google One Box to really get the good stuff and avoid the fraud and Rube Goldberg routine.
This is from Google.
"Google Search Appliance GB-1001 - New Version
Now the same reliable results you expect from Google web search can be yours on your intranet or public website with the Google Search Appliance.
This combined hardware and software solution indexes all forms of content on your intranet and websites, offering a robust, scalable and cost-effective solution to your enterprise search needs.
Starts at $30,000 for search across 500,000 documents, and includes 2 years of support, software updates, and replacement coverage."
If the adsense biz crashes, I guess they can push more One Boxes out the door. Maybe they will start selling a personal one box version that plays songs and kills the ipod. Google Radio next? 1pod!
I think Jim is a bot because his post makes no sense.
Well you are wrong not a bot. Keep trying to think though.
>Can't the bad guys write a computer program that automatically fills out the surveys?
Human surveys are sometimes sampled by the person who paid for them calling the participants to confirm that they really happened.
Regarding online games and click fraud, there's a more perniciuos form where players use software to create macros for the game that, in limited circumstances, handle 90% of the gameplay. They run several computers with the game, macroing each, generating in-game wealth /loot that's then sold on eBay. There's no cheating involved in the sense of exploiting weaknesses in the game engine (by comparison, the aimbots that Bruce mentions exploited the fact that the client required full knowledge of the game's geometry and the location of other players, knowledge hidden from the player himself).
The problem is subtle: game designers carefully balance the world, placing just so many monsters here and just so many resources there so that normal players have to put X amount of effort into getting Y amount of reward, which translates to a certain level on continual subscription fees. Automating the play upsets that basic balancing equation, flooding the virtual economy with money and precious loot, as well as monopolizing rare entities.
On the other hand, gold farming (or macro-mining) increases the size of the overall in-game economy, bringing with it stability and making difficult to achieve goals easier, which attracts players. For doctrinaire reasons, game companies have to frown on it because, individually, a player paying $15.95/month feels disadvantaged competing with a well-organized macro miner for the same in game prizes; but game developers must recognize the stabilizing benefit to the whole world.
Another problematic axis is that gold farmers are monopolizing a market that the game companies would like to capture themselves: incrementally charging players in-game for advantages like gold or loot. Players as a body are generally against the concept, and look down on other players who buy gold from eBay, because it advantages the players with a lot of disposable income. Nonetheless, gold farmers are making money, and Blizzard/Eve/Sony must look enviously at that revenue stream.
The upshot is that game companies are also looking for ways to distinguish between real human players and automated clicks, for exactly the same economic reasons.
Google has value-based conversion tracking. Money talks, BS walks. Yahoo isn't quite up to snuff, but they'll get there.
> Years ago, talking about security, I complained about the link between computer and chair.
Bruce, you had a perfect opportunity to bring PEBKAC back into common usage here. How can we keep the newer generations interested in the jargon file if we don't keep the lingo alive?
i suspect the problem is much worse than advertised. air-conditioned trailers full of illegal aliens in the border states, all clicking on ads for sub-minimum wage.
if your business model is transparent except for two problem areas, here and there, here and there are where the scam will evolve to maximum size.
google is a short. as jimi hendrix observed, "castles made of sand wash into the sea, eventually."
Exercise for the student:
Design an advertising system where:
a) advertisers do not pay a fee unless they actually make a sale.
b) advertisers cannot defraud Google by claiming that a click-through did not result in a sale, when, in fact, the sale occurred.
Actually, I think I stated the problem in an unnecessarily restrictive fashion. The goal is not necessarily to prevent advertisers from defrauding Google. The goal is to make it unprofitable for them to do so.
Brian- there's the rub. Right now all the advertisers have to trust google that they are paying out only on legit clicks, and that they're doing something to stop fake clicks. Except there is no third party auditing of Google's records, unlike every other advertising media on earth (magazines, newspapers, TV, radio, all have third party auditors)
If you go to a "pay per sale" model, that means that the ADVERTISERS suddenly have all the aces, and can defraud Google by simply saying a sale didn't take place. The Google/advertiser role would be reversed, with all the power placed in the hands of the advertisers.
To put it simply, this would utterly destroy Google's business model and would no longer allow them to be a "black box" in terms of pricing. If Google is ever forced to go to this model, except their revenue to fall 90% or more, unless they plan to send CPAs out to every advertisers' place of business to comb through their books.
Hell, if I was an advertiser you bet your ass I would cheat Google by telling them "nope, I didnt get any sales this week from all those leads you sent me" even if I had made a million bucks.
Which is why Google is terrified.
Here's how I would do it.
Google assigns each click-through a unique tracking number.
Using something like a SOAP web service, advertisers are required to notify Google of a sale immediately before the sale is completed. They include the tracking number in the notification.
Google's web service can either sign-off on the notification (in which case the advertiser is billed), or they can return an error code telling the advertiser that the sale was a just a test and should not proceed.
Google then does random spot checks of advertisers. Every now and then, a google employee goes through the process to generate a "fake" sale for an advertiser. If the sale succeeds, they know that the advertiser is defrauding Google, and can take appropriate action. If the sale fails, they know the advertiser is being honest.
The spot checks need to be frequent enough, and the penalties for failing a check high enough, that the advertisers know they stand to lose money if they try to defraud Google.
The spot checks also need to look real. Maybe Google can hire some click monkeys for that, their rates seem reasonable.
Are you sure you are going to turn away one of the largest places to reach customers instead of allowing Google to audit you?
I honestly feel this is a symbiotic relationship where unfortunately, one party holds a much stronger hand. Sure, Google depends on that revenue, but you know there are advertisers will take that chance. Advertisers, however, don't have another media that can reach as large of an audience.
Now, if those law suits scare Google enough, it might prompt them to go point of sale and work with advertisers to find a solution.
Isn't Google trying to invert some kind of inverted Turing test ?
When they have finished, does anything that is accepted by their software class as intelligent?
I confess - I've used automated mouse-clicks to gain an advantage in an online roleplaying game. The game was "Dark Age of Camelot". The activity was practising fletching. Once you'd purchased sufficient materials, there was a very repetative "click a button to start making some arrows, wait 10 seconds for it to complete." So I used Lego Mindstorms to make a robot which would push my mouse button every 10 seconds, while I went off and did something else.
Brian wrote: "b) advertisers cannot defraud Google by claiming that a click-through did not result in a sale, when, in fact, the sale occurred."
Not a problem. Google calculates their expected benefit from displaying an ad as: probability that ad results in action x amount advertsier is willing to pay for action. If the advertiser underreports actions google estimates the probability of the action lower, therefore google's expected benefit is lower, therefore google displays other ads instead.
I like this approach by google. It means everyone can try out their own click-fraud prevention systems. I bet there'll be a whole market in non-intrusive Turing tests opening up because of this. Stuff like is the user scrolling and moving the mouse in a human fashion? All you need is ingenuitity and a dash of AJAX.
Walmart does something similar with their shelf space. Clever. I like it.
As an advertiser I notice our ads get hit by slurp, a bot, and we seem to get charged for them, although googlebot is not hitting the adverts.
I think google has a way to go before it
can claim progress in click fraud if the main activity is not to charge for its own bot.
Mr. Turing, where are you now?
Google AdSense + Google Checkout = Proof that the ad caused the customer to buy product. Isn't this where they're going?
I was in the search-engine business for one job, and there were 3 advertising models; (1) pay-per-impression (cpi), (2) pay-per-clickthrough, and (3) fraction-of-sales.
They are listed in increasing order of favorability for the merchant/advertiser (and decreasing for the search engine/web site). Basically to do the last one, you need to cooperate in the design of the web site so that both parties have a record of the transaction; that is, put a tracked image served by google/doubleclick/whatever on the purchase page. If they don't serve it, they don't get paid. You can play webtracking games with cookies or frames or all kinds of stuff. There are firms that audit web logs, we were paying one such company to do so. The silly thing was that they had a short 32-bit checksum on each web log entry, and the calculation was made by linking against a custom library, and it hashed the .text segment of the application which created them, so that they'd know if we made logs with a different program. However, our CGI was written in Perl, and the code was actually in the data segment, so we could have forged an unlimited number of entries. Anyway, they spot checked them, it wasn't perfect but it was sufficient to keep the honest businesses honest.
Actually there's two fraction-of-sales techniques; one involves prorated and one involves actually tracking through to the sale. The prorated version says that 10% of the people that visit the site spend $100, so if you send them 20% of their visits you get a fraction of $200.
What happened with the dot-com bust, among other things, was that advertisers could, for the first time, measure the number of impressions and sales with great accuracy, and guess what? They found out that their ads weren't nearly as effective as they had believed! They teach in classrooms that ads don't stimulate demand, but they do increase awareness of the product, but probably there is a lot of fooling oneself involved in typical advertising. The cold hard truth is that people hate ads, and try their best to ignore them.
If a group of people are in varied locations around the world and each or using dynamic IP addresses or different proxies, and conspire to click on a competitors ad - there may very little one could do.
The only possible solution for alleviating click fraud MIGHT be to limit the number of times one IP address could successfully CLICK on an ad and go to the ad itself via a REDIRECT script - within a given time frame.
And also to make accepting a tracking cookie MANDATORY so one could NOT use different IP Address ...as a tactic
and possibly allowing the Ad site to be appear in an IFRAME of the Search Engine's.
But even that strategy has loopholes
I'm glad that you have commented publicly on the click fraud problem. It is good to see that security experts are becoming aware of the problem and expressing (informed) opinions. The more the general public is aware of the issues, the better decisions they can make about how they spend money on ads, purchases, etc.
I worked in the search engine business as well. I was very unhappy when my company switched to a CPC model because I have always felt is was very undesirable due to the degree of fraud that could be perpetrated through it. I spent a lot of time fighting click fraud, which essentially amounted to a cat-and-mouse game between myself and the fraudsters, which was a highly unproductive use of my time. CPA shifts risk from the advertiser to the search engine or ad network, but does not eliminate other types of fraud (e.g. when the advertiser pays less than they should). I think that with CPA, there is an incentive to be honest, because breaches of contract can not only lead to termination of one's account (e.g. if the SE or ad network computes an expected amount of earnings based on clickthroughs and/or impressions), but branding as a bad business risk among customers. But some savvy merchants might find ways to withhold some payments.
Of all models I know of, my opinion is that fixed fees offer best combination of simplicity of implementation and lowest risk of fraud to all parties. There is a sentiment among some ad professionals that fixed fees are too "old media" for today, and an arguably justifiable complaint from advertisers that they don't want to pay if they get insufficient clicks or impressions. Ultimately, there is no "silver bullet"; at some point, whoever is paying for advertising should expect to pay at least the cost of the service of handling the ad campaign, reaching the audience, and some profit for the SE or ad network.
The other thing I'd like to point out is that it has always struck me as odd that as talented and bright as the Google engineers are, that they did not realize how much damage from CPC click fraud could ensue. I would have thought at the very least, they would have delayed releasing AdWords/AdSense until they had tools in place (such as smartpricing) to minimize the effect of click fraud. Or that they would have waited until developing other payment options such as CPA or fixed fees.
Take it from an industry insider...
1. CPA will never replace PPC at Google.
2. The scope of the PPC fraud problem is bigger than most people think.
3. The PPC fraud problem can be fixed, but the networks and search engines are dragging their feet. They trying to walk the line between a large, immediate reduction in revenue, vs a steady long-term decline in revenue as a result of loss of confidence and trust among advertisers.
4. Want to cut your fraud click risk by 80% in 5 minutes? Easy, just remove your campaigns from the search engine's affiliate network and only run on the SE itself. Of course, you may be losing a large percentage of your traffic at the same time, depend on the network.
5. The networks have loads of data that could easily be available in reports, but is not.
My most recent article on the subject:
Many articles are linked on our web site:
I don't understand how pay-per-action (PPA) is going to solve the click fraud issue. One solution I heard was that whenever ad gets clicked, user will be presented with a some kind of challenge page where he/she has to enter email address or any other stuff. Even if this is done, how difficult it is to simulate this action via a bot? I don't think click fraud is a problem which will get solved ever.
Bruce, the observation about the link between computer and chair is excellent but I think your ending is overoptimistic because you have left out of your analysis the factor of agenda.
I do not know exactly the details of how AdSense and AdWords work but I think that in both cases of click fraud it would be reasonable to assume that google gets a commission for each ad click, whether it is fraudulent or not.
As a result click fraud is profitable for google, up to the point of course, that it is
"controlled", so that the system does not become unusable. So it is in google's interest to do the absolute minimum to hinder click fraud and that's
another reason (besides the problems you mention) to expect it to be with us for a long time.
While it is in Google's short-term interest to to minimize the problem of click fraud -- and they're doing that -- it is in their long-term interest to solve the problem. That is, it is in their long term interest to ensure that the advertising model works on the Internet.
What is it that actually makes click fraud _fraud_? I assume that when surfing, I have every right to click on every link in the net that I encounter there, including e.g. links to my competitors ads. As far as I know, it is not forbidden even to write a program that clicks on some link, or even use a third party to do this, as long as my click volume does not generate any DoS attack in the receiving end.
And even though this behaviour could be viewed illegal based on some
harassment paragraphs or whatever, where is the _fraud_?
Fraud is defined as "obtaining a financial advantage by dishonesty". The financial advantage (money) is the key thing distinguishing fraud (a crime) from regular dishonesty (which is arguably immoral, but legal except in certain specific situations such as testifying under oath).
In this case, someone is getting paid for people looking at ads displayed on his website. It's fine for him to click on anything he likes, but if he represents those clicks as being real prospective customers (and thus gets paid for them) when they were actually his botnet, he is committing fraud.
I don't know if i can say this here or not but i faced a real problem and that, is i used one ot the dvertised for house in gumtree website and i send money by MoneyGram to my husband to show this person i have money for rent ( his condition) and show him online the confirmation so he went and get money from post office instead of my husband with invalid ID.is there anyone here could help me!!!!!!!!!!!?????
Schneier.com is a personal website. Opinions expressed are not necessarily those of Co3 Systems, Inc.