Schneier on Security
A blog covering security and security technology.
« Reading a Letter from the Envelope it Was In |
| Censorship in Dubai »
November 12, 2008
The Economics of Spam
Excellent paper on the economics of spam. The authors infiltrated the Storm worm and monitored its doings.
After 26 days, and almost 350 million e-mail messages, only 28 sales resulted -- a conversion rate of well under 0.00001%. Of these, all but one were for male-enhancement products and the average purchase price was close to $100. Taken together, these conversions would have resulted in revenues of $2,731.88 -- a bit over $100 a day for the measurement period or $140 per day for periods when the campaign was active. However, our study interposed on only a small fraction of the overall Storm network -- we estimate roughly 1.5 percent based on the fraction of worker bots we proxy. Thus, the total daily revenue attributable to Storm's pharmacy campaign is likely closer to $7000 (or $9500 during periods of campaign activity). By the same logic, we estimate that Storm self-propagation campaigns can produce between 3500 and 8500 new bots per day.
Under the assumption that our measurements are representative over time (an admittedly dangerous assumption when dealing with such small samples), we can extrapolate that, were it sent continuously at the same rate, Storm-generated pharmaceutical spam would produce roughly 3.5 million dollars of revenue in a year. This number could be even higher if spam-advertised pharmacies experience repeat business. A bit less than "millions of dollars every day," but certainly a healthy enterprise.
Of course, the authors point out that it's dangerous to make these sorts of generalizations:
We would be the first to admit that these results represent a single data point and are not necessarily representative of spam as a whole. Different campaigns, using different tactics and marketing different products will undoubtedly produce different outcomes. Indeed, we caution strongly against researchers using the conversion rates we have measured for these Storm-based campaigns to justify assumptions in any other context.
Spam is all about economics. When sending junk mail costs a dollar in paper, list rental, and postage, a marketer needs a reasonable conversion rate to make the campaign worthwhile. When sending junk mail is almost free, a one in ten million conversion rate is acceptable.
Posted on November 12, 2008 at 6:52 AM
• 54 Comments
To receive these entries once a month by e-mail, sign up for the Crypto-Gram Newsletter.
But when I suggest that after your first 100 e-mails a day, you be charged a penny an e-mail, everyone scoffs.
Fine, be that way. Spend your days deleting 200 spam an hour, safe in the knowledge that your 101'st e-mail of your day is still free.
BTW, who the HECK are the idiots who buy ANYTHING from spam? Particularly "male enlargement" products?
With a rate of one in ten million, it seems possible that the buyers actually do suffer from some sort of mental retardation.
One hundred percent of my spam ends up in a SPAM folder before I open my email accounts. Gmail solves this issue.
Really simple to just take a moment to quick view the titles and senders to eliminate a false hit, which has not happened in the last three months.
Easy with one click to eliminate it all.
@ Albatross; Apologies, I know it's been done to death, but it still needs to be overcome before before your solution has a chance of being listened to. (For the record, I risk using gmail for my spam filtering needs. I used to post a lot to Usenet and I also have my email in plaintext on the network, so I know about receiving spam).
Your post advocates a
( ) technical ( ) legislative (X) market-based ( ) vigilante
approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)
( ) Spammers can easily use it to harvest email addresses
(X) Mailing lists and other legitimate email uses would be affected
( ) No one will be able to find the guy or collect the money
( ) It is defenseless against brute force attacks
( ) It will stop spam for two weeks and then we'll be stuck with it
(X) Users of email will not put up with it
( ) Microsoft will not put up with it
( ) The police will not put up with it
( ) Requires too much cooperation from spammers
(X) Requires immediate total cooperation from everybody at once
( ) Many email users cannot afford to lose business or alienate potential employers
( ) Spammers don't care about invalid addresses in their lists
( ) Anyone could anonymously destroy anyone else's career or business
Specifically, your plan fails to account for
( ) Laws expressly prohibiting it
(X) Lack of centrally controlling authority for email
( ) Open relays in foreign countries
( ) Ease of searching tiny alphanumeric address space of all email addresses
( ) Asshats
( ) Jurisdictional problems
(X) Unpopularity of weird new taxes
(X) Public reluctance to accept weird new forms of money
(X) Huge existing software investment in SMTP
(X) Susceptibility of protocols other than SMTP to attack
( ) Willingness of users to install OS patches received by email
(X) Armies of worm riddled broadband-connected Windows boxes
( ) Eternal arms race involved in all filtering approaches
( ) Extreme profitability of spam
( ) Joe jobs and/or identity theft
( ) Technically illiterate politicians
( ) Extreme stupidity on the part of people who do business with spammers
( ) Dishonesty on the part of spammers themselves
( ) Bandwidth costs that are unaffected by client filtering
( ) Outlook
and the following philosophical objections may also apply:
(X) Ideas similar to yours are easy to come up with, yet none have ever
been shown practical
( ) Any scheme based on opt-out is unacceptable
( ) SMTP headers should not be the subject of legislation
( ) Blacklists suck
( ) Whitelists suck
( ) We should be able to talk about Viagra without being censored
( ) Countermeasures should not involve wire fraud or credit card fraud
( ) Countermeasures should not involve sabotage of public networks
( ) Countermeasures must work if phased in gradually
(X) Sending email should be free
(X) Why should we have to trust you and your servers?
( ) Incompatiblity with open source or open source licenses
( ) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
( ) I don't want the government reading my email
( ) Killing them that way is not slow and painful enough
Furthermore, this is what I think about you:
(X) Sorry dude, but I don't think it would work.
( ) This is a stupid idea, and you're a stupid person for suggesting it.
( ) Nice try, assh0le! I'm going to find out where you live and burn your
The converse of this is that someone who purchases a spamvertised product or service implicitly underwrites the next 10 million spam messages sent--good advice to anyone who thinks they've "found a deal" through unsolicited email.
I'm a fan of an anti-spam solution that works on their own battleground--gain an army of bots and when a spam message is received from an address, generate enough traffic to that address to prevent any possibility of spam coming from it.
Will you still be so enthusiastic about your solution when somebody spoofs YOUR email address as the source and you end up on the receiving end of the "good" bots?
@Albatross - You left out one vital step: invent a time machine, so you can go back in time, and institute this fee before people felt entitled to free email. (How, is another question. You have not addressed *who* we pay this fee to, or *how*, and *what* will be done with it.) While you're at it, might as well get "them" to use a more secure protocol than SMTP, and no, something as complex and unfree as X.400 doesn't count....
@Sturat: methinks he means IP address. That's at least a bit harder to forge, at least convincingly.
How can you have Spam, spam, e-mail, spam, and spam WITHOUT the Spam?
There's nothing that anyone is ever going to be able to do to stop spam from being SENT.
The majority of spam is sent from zombie machines.
That means that the cost of each message AND machine to send it AND bandwidth is, essentially, ZERO. A single purchase is enough to justify sending 10,000,000 messages.
The best you can do is to block it from being received. And when you block it, make sure that the error message contains the phone number of a PERSON who can fix any problems with legitimate email being blocked.
The way they inserted themselves in the bot network is interesting. You could use such a parasitic attack to disable/poison them. Even just rewriting spam messages so the saps responding to them end up on non-functional sites would help, by decreasing the conversion rate. (And with this approach it might be difficult for spammers to find out what went wrong and where.)
I'm not so sure that spam can't be blocked on the sending side. ISPs can certainly throttle SMTP messages coming from machines they supply service to. If, as Albatross suggests, the ISP starts throttling after, say, a hundred emails a day, the volume of spam coming from a host CAN be meaninfully reduced. This would require a modest investment on the part of the ISP, and certainly they'd want to implement an exception list for their customers who are legitimate high-volume email senders. This wouldn't interfere with standards based SMTP communications, and the only burden it would impose on ISP customers is the one-time phone call to get their account added to the 'high volume email sender whitelist.'
This is certainly not a new idea. In fact, there were press stories a few years ago about ISP's doing exactly this to combat Windows zombies.
I have to believe that for an ISP, the investment in developing this solution would be more than offset by the lowered costs of bandwidth. I suspect the true costs, however, would be in tech support calls from users when suddenly their legitimate email won't go through because they've unknowningly sent a bajillion spam emails from their zombie'd box. The tech support call to help the customer get rid of malware would probably be much more expensive than the bandwidth savings.
@Calum - thanks for the /. intervention. :-)
Perhaps you can do the same for this idea - which I'm sure must also have been suggested many times.
The compromised botnet home PCs must be using open relays to forward on the email they are sending. If the owners of these relays (on a progressive basis) could be persuaded to auto-blacklist for 1 hour any peer email system that has a spam-to-non-spam ratio above, say, 90%. That would in effect freeze out substantial parts of the botnet emailers rendering them useless. The is-it-spam decision being made through filters provided by a site like Spamhaus.
Why wouldn't that work?
Another source of revenue is credit card fraud, after they get the card information.
I read that one of the TV marketers of Male Enhancement Pills would repeat bill under "fine print" authorization, slow to cancel what they considered a continuing order.
The Storm Worm guys would probably go beyond that.
Honestly, I've never seen a plausible technical solution to the spam problem. The best ones are band-aids, others make things worse.
The least-bad solution, in my opinion, is forget about the senders of spam, and focus upon those who use their services -- the pill vendors, porncasters, etc.. These people can be traced through the time-honored law-enforcement principle of "follow the money".
It should simply be illegal not merely to issue spam, but to advertise via spam. If law enforcement could carry out a few stings, and put away a few vendors, the resulting chill on the market could really put a dent in the demand for spammer services.
This approach wouldn't help with things like pump-n-dump stock spam (unless the SEC pulled its head out of the orifice where it is stored), or with "Hi, I might be your new bride from Russia" spam. But there has to be some profitability threshold below which industrial-grade spam is just not worthwhile. Perhaps putting a hurting on pill vendors could get us close to that threshold, in which case the spammers might just dry up and blow away.
Most (legit)web hosting companies offering emails charge after 250 mails a day.
I am sure almost all free email servers are now capped - the emails are being sent by rogue servers now.
On the other hand it's probably much easier to filter 100's of million of identical messages going to all your subscribers.
Only effective spam is that gets thru and I have seen almost all filters improve to a point that very little is.
"When sending junk mail is almost free, a one in ten million conversion rate is acceptable."
Except where there are modern anti-spam laws which could bring the spammer in jail for a while.
Cross-selling of customer information is what makes regular direct marketing profitable, so it should be no surprise if the same were true for spammers. Credit-card information, name and address, even the email address of someone stupid enough to respond should be valuable.
The tiny response rate, though, suggests another possibility for dealing with spam: stop the customers. If one could find the (by these estimates) few thousand or at most tens of thousands of people worldwide who still answer this stuff and convince them either by reason or by other incentives to stop doing so, that would put an end to the economic viability of non-phishing spam (and even much phishing spam) once and for all. Now there's a tragedy of the commons waiting to be set right.
What I find funny is the people who buy the "blue pill" on the internet, without knowing that 50% of Viagra is just a piece of blue coated sugar, and yet they manage to perform in the bedroom. A case for placebo and psychosomatics.
@ Carlo Graziani,
"The least-bad solution, in my opinion, is forget about the senders of spam, and focus upon those who use their services..."
I'm not even sure spam is realy worth bothering with these days.
As far as bandwidth hogs go try looking at the update files for virus and scum ware detection systems.
The cure is now almost worse than the illness...
Me thinks it's time to have a real rethink on how we deal with these issues.
if more of the spam receivers would click on the provided links it would increase the hosting costs for the spammers to a point where their business is not profitable anymore, or the servers are overloaded (e.g. flashmob/slashdot effects etc)
Why not fine all those who let their machines get inducted into botnets? Instead of having ISPs hunt for filesharers, they should be hunting for zombies.
I should add in terms of precedence, in some parts of the world, if criminal organizations use your property for illicit dealings you're still liable even if you were not aware of such activities. I see no reason why that shouldn't be extended to the net.
The concentration on the front end economics ignores the back end economics. In some ways, the modern spam operation is a man-in-the-middle attack. Even if a conversion *never* occurs, the bot-net owner still makes money for providing the spam service. Repeat business is definitely helpful, but all they really have to do to line their pockets is to maintain a level of spam that convinces certain recipients to try spamming to market their own products. In short, spamming is the product most advertised by spam.
Also, the spam problem has been solved. It's called doing edge labeling instead of node labeling. The only problem is that most users and ISPs seem unwilling to change their silly habits, yet are somehow still surprised when the generic email address they broadcast to the world gets so much spam.
On the flip side, maybe in a way it's a good thing we do get huge volumes of spam every day. This is just speculation but perhaps if we weren't as familiar with spam email, we'd be less equipped to recognize spam or phishing when it did occasionally slip through the cracks, and perhaps more trusting? Would the conversion rate rise?
Any good mathematicians out there, feel free to call me on it if I'm way off on this idea.
Explain yourself. Would this be similar to ye olde system of explicitly naming the routing path? How would this stop botnets from sending spam?
@paul : I agree, we need to find the list of those guillible people who are willing to buy anything over the internet. If you run across a list of those people, could you post it somewhere?
I hate to say this, but some people who seem outwardly intelligent WILL fall for spam solicitations. I have a friend who is by all accounts a reasonable, intelligent person, UNTIL you put him on the internet. He will fall for every chain letter and MMF scam that comes his way. If it weren't for me he'd be destitute by now. I've tried many times to school him on this sort of scamming, but he's remarkably resistant to the idea that he can't get rich quick on the internet. At his work, he wouldn't be suckered like this by a salesman, but the internet makes an idiot out of him.
Their extrapolation is flawed. How many internet users can there be? 350 million emails could cover a huge percentage of users. I don't think increasing the number of emails to 23 billion would result in much of an increase of revenue.
@David: the compromised PCs aren't using open relays. They're using the ISP's mail servers and sending as the owner of the PC. This is why things like charging for e-mail won't stop spam, since the spammer isn't the one who'll be getting the bill. Throttling of outgoing e-mail by the ISP would work, except that there's the false-positive problem and the infrastructure issue. My ISP forces all outgoing e-mail through their servers and attempts to filter spam, and they get a fairly constant stream of complaints about legitimate e-mails never getting to the recipient because either the filters misclassified the mail as spam and dropped it on the floor without notification or because the load on the filtering infrastructure was so high it caused the servers to malfunction and drop mail on the floor.
Calum suggests a number of problems with Albatross' proposed solution-by-economics. Still, the idea *is* pretty enticing, right? How about this variation:
Phase 1: Invent workable electronic cash.
Phase 2: Charge everyone who sends you an email 1c. That is, publically announce that you will have your email client automatically delete any email that does not contain 1c.
Phase 3: Profit. (Or at least, no spam.)
The idea here is that:
1. There's no central authority regulating the system
2. You send the 1c's back to your friends when you reply to them
3. If you still get spam, just up the charge to senders of email to the marginal cost to you of receiving spam
It may be that this approach doesn't work. It may fail at step (1) -- perhaps electronic cash is not inventable. Even if it is inventable, then the approach may fail at step (2) -- it maybe that you can't convince all your friends to sign up to the e-cash system. (I can't even get mine to sign up to email encryption.)
But I guess I'm claiming that, in principle, this would make the problem of spam just go away. That is, really really, it would solve the spam problem.
I presume this is either obvious or obviously wrong!
"The least-bad solution, in my opinion, is forget about the senders of spam, and focus upon those who use their services -- the pill vendors, porncasters, etc."
The big problem with this approach is that it leads to a whole new category of Joe Jobs (see http://en.wikipedia.org/wiki/Joe_job for a description). Hiring a spammer using a third party's name can be used to get that third party into legal trouble (and spammers aren't going to care about verifying who hired them so long as they get paid).
The 1c idea is novel, but here's why it won't work.
1. Spread large botnet.
2. Spam yourself on the order of millions of emails, each containing 1c from information gathered from each infected machine.
@a non i mouse
It's not necessary to explicitly name the entire path; just the "relationship" between the two nodes. There is no one way to do it, but there are various easy ways you can begin with and then scale it up to more secure implementations if you have the need. Many email providers, for example, allow you to construct a email@example.com style address. It'd be nice if there were a standard when it came to edge addressing so that email clients could better handle it, but it's still possible to work something out with current email practices.
I use a more advanced method that allows me to expire an address at the DNS level, so the spam never even leaves their server. I used to work on anti-spam software, which seemingly made me a target for spammers, and had my spamcop.net account suspended a few times because I was reporting over 3000 spam per day. So I gave up on all that and implemented a better way of dealing with email. Now, with zero filtering, I might get one spam a day via the public-facing business contact address, an edge which I can sever and replace at any time if the volume increases.
"Hiring a spammer using a third party's name can be used to get that third party into legal trouble"
I've seen this counter-argument, but I don't think it's a serious obstacle. The thing is, the collected evidence against a spam-advertised vendor has to stand up in court. Joe-jobbing at the "fool the kiddies on the net into hating someone" level is easy, but at the "burden of legal proof for conviction" level it isn't. I doubt very much that a Joe Job could trick the FBI or a State's Attorney into attempting to make a criminal case against someone.
Making spammers pay not with money but with CPU cycles seems an interesting idea. See MS' Penny Black project.
Making spammers pay not with money but with CPU cycles seems an interesting idea. See MS' Penny Black project.
I love the idea, but there was an interesting problem when I researched it. The access to computational power varrys GREATLY between people. An algorithm which is simple enough to not infringe on an old PDA's ability to send an email may be too simple to stop a spammer with access to a modern quad core desktop from sending 350 million emails.
To RH and Durable Alloy:
There's a number of very good results from cryptography/theoretical computer science community, which solve some of the problems you talk about. A good reference is: http://www.wisdom.weizmann.ac.il/~naor/PAPERS/...
The problem that I see with this approach is that the way the things are, the people who will pay (computationally) are the poor users that were taken by bot-nets, and not the real spammers.
And here's another tactic -- admittedly, it only works within "responsible" countries, but it combines well with ISP-level filtering.
Man, it really burns my biscuits that my completely unimplementable and incredibly oversimplified solution wasn't immediately embraced by everyone...
But it was worth it for Calum's checklist!
Of course my idea is based on simply extending the original SMTP metaphor of post offices and letters. The one thing they didn't implement was postage... As for how such a thing would be implemented, don't bother me with the details, I'm an idea man!
Although the computation power was provided by the victims of the botnet, but it's still somewhat effective because now an infected node can no longer send as much spams as before. Furthermore, an infected computer may become much less responsive so the victims can more easily notice that their computers are probably infected.
@Albatross: "...who the HECK are the idiots who buy ANYTHING from spam? Particularly "male enlargement" products?"
I'm with you 100% there; I chant that like a Gregorian Monk every time I hear a story on the news about spam.
What's really striking is how much cost is foisted onto the spam recipients, relative to the spammer's income.
Say it costs a recipient one-tenth of a cent to receive a spam email. I suspect this is conservative, given the costs of having people spend time to delete spam from their inboxes, buying and maintaining spam filters, and installing enough storage and bandwidth to handle the torrent of spam.
So 10 million spam emails impose a cost of $10,000 on the recipients, but yield only $100 in income to the spammer.
That is, for each dollar the spammer makes, he/she imposes $100 of costs on society.
"Calum suggests a number of problems with Albatross' proposed solution-by-economics. Still, the idea *is* pretty enticing, right? How about this variation:
Phase 1: Invent workable electronic cash."
(X) The police will not put up with it
If you have a spam problem, there is a very simple and immediate cure: change your email address. Then don't had it out to everyone who asks for it.
At last count I had about 75 active email addresses. I get spammed on exactly one of them, an old yahoo account, on which Yahoo! does an excellent job of filtering.
All the suggestions so far rely on one or more of
(a) "clever" technical solutions
(b) the State must punish someone
The technical solutions will all fail. The spammers are rational and smart.
The problems with giving the State even more coercive power over us should be obvious, but sadly are not to very many people.
However, there is a simple way to abolish spam. Each ISP occasionally sends fake spam to its own customers. Anyone who responds loses Internet access for, say, a week, for a first offense.
No clever technical trickery. No involvement by cops with guns. After a little while, no more spam.
The mystery is, why are ISPs not already doing this, if as they say, spam is a problem for them?
Spam is not about selling products, it is about selling the network.
Networks choked by spam (a 2/3 ratio with all other kind of packets for latest exteems) need more cables, need more servers, needs more storage.
Thanks to spam Telco companies have the demand boosted for network infrastuctures, and can charge higher fees to users.
While spammers gains few dirty K dollars/year with "sales", Telco companies have the multibillion dollar network infrastructure market inflated, and gains billions of dollars/year.
Spamming the spammers might work, but you probably need to spam the firms who buy their services.
Somewhere there is always a web form to buy from, or an email, or a fax, or a phone number.
Probably these could be overwhelmed with counter/educative-spam, driving the message home to stop using these techniques.
Spam markets only work with prices discrimination and reputation mechanisms. They cannot work with fungible payment because of the potential for fraud. Another problem with POW is that the total cost may be greater than the cost of spam depending on how you build the model, so it is not a settled question.
Here are the papers:
1)Ben Laurie and Richard Clayton, " 'Proof-of-Work' Proves Not to Work", Third Workshop on the Economics of Information Security, 2004, Minneapolis, MN, available online, at http://www.dtc.umn.edu/weis2004/clayton.pdf. comment: spam producers use zombie machines and thus have a different production frontier than legitimate email senders, therefore proof of work doesn't work
Debin Liu and L Jean Camp, "Proof of Work can Work", Fifth Workshop on the Economics of Information Security, 2006, Cambridge, UK, available online, at http://weis2006.econinfosec.org/docs/50.pdf. comment: the difference in the production frontier can be overcome by embedding proof of work into current anti-spam systems which include reputation systems, white lists, and black lists.
3) Marchall van Alstyne, An economic response to unsolicited communication,
Re: Going after advertisers who use spam:
The advertisers are not always directly in the know although they cant be clueless about it.
Due to several adword style referral schemes there is a market for click arbitrage.
Your click is sold to someone who pays x cents per referral , who sells it to someone (2x cents) .... etc..... who directs the clicker to the online pharmacy.
Plausible deniability is there. When you add jusrisdiction issues, it becomes a pretty tough sell.
I think that spam cannot be effectively blocked or filtered. The best countermeasure to fight spam will be to make it economically unviable. So what if my email program had within it the intelligence to parse emails I mark as spam, and visit the website in the background and throw everything returned into the bitbucket? The spam king will charge for the link hit, but the advertiser will be paying for worthless link hits which result in no revenue. The advertiser will quickly figure out that spam advertising is no longer economically viable and stop. This approach will work if email programs allow user to identify an email as spam, and the program visits the links in the email.
One of the biggest problems with spam is that spammers are not actually paying the full cost of supporting their businesses.
It is only profitable because they are cheating.
Schneier.com is a personal website. Opinions expressed are not necessarily those of Co3 Systems, Inc.