Schneier on Security
A blog covering security and security technology.
« ITConversations Interview |
| Identity-Theft Humor »
May 13, 2005
Spam is back in the news, and it has a new name. This time it's voice-over-IP spam, and it has the clever name of "spit" (spam over Internet telephony). Spit has the potential to completely ruin VoIP. No one is going to install the system if they're going to get dozens of calls a day from audio spammers. Or, at least, they're only going to accept phone calls from a white list of previously known callers.
VoIP spam joins the ranks of e-mail spam, Usenet newsgroup spam, instant message spam, cell phone text message spam, and blog comment spam. And, if you think broadly enough, these computer-network spam delivery mechanisms join the ranks of computer telemarketing (phone spam), junk mail (paper spam), billboards (visual space spam), and cars driving through town with megaphones (audio spam). It's all basically the same thing -- unsolicited marketing messages -- and only by understanding the problem at this level of generality can we discuss solutions.
In general, the goal of advertising is to influence people. Usually it's to influence people to purchase a product, but it could just as easily be to influence people to support a particular political candidate or position. Advertising does this by implanting a marketing message into the brain of the recipient. The mechanism of implantation is simply a tactic.
Tactics for unsolicited marketing messages rise and fall in popularity based on their cost and benefit. If the benefit is significant, people are willing to spend more. If the benefit is small, people will only do it if it is cheap. A 30-second prime-time television ad costs 1.8 cents per adult viewer, a full-page color magazine ad about 0.9 cents per reader. A highway billboard costs 0.21 cents per car. Direct mail is the most expensive, at over 50 cents per third-class letter mailed. (That's why targeted mailing lists are so valuable; they increase the per-piece benefit.)
Spam is such a common tactic not because it's particularly effective; the response rates for spam are very low. It's common because it's ridiculously cheap. Typically, spammers charge less than a hundredth of a cent per e-mail. (And that number is just what spamming houses charge their customers to deliver spam; if you're a clever hacker, you can build your own spam network for much less money.) If it is worth $10 for you to successfully influence one person -- to buy your product, vote for your guy, whatever -- then you only need a 1 in a 100,000 success rate. You can market really marginal products with spam.
So far, so good. But the cost/benefit calculation is missing a component: the "cost" of annoying people. Everyone who is not influenced by the marketing message is annoyed to some degree. The advertiser pays a partial cost for annoying people; they might boycott his product. But most of the time he does not, and the cost of the advertising is paid by the person: the beauty of the landscape is ruined by the billboard, dinner is disrupted by a telemarketer, spam costs money to ship around the Internet and time to wade through, etc. (Note that I am using "cost" very generally here, and not just monetarily. Time and happiness are both costs.)
This is why spam is so bad. For each e-mail, the spammer pays a cost and receives benefit. But there is an additional cost paid by the e-mail recipient. But because so much spam is unwanted, that additional cost is huge -- and it's a cost that the spammer never sees. If spammers could be made to bear the total cost of spam, then its level would be more along the lines of what society would find acceptable.
This economic analysis is important, because it's the only way to understand how effective different solutions will be. This is an economic problem, and the solutions need to change the fundamental economics. (The analysis is largely the same for VoIP spam, Usenet newsgroup spam, blog comment spam, and so on.)
The best solutions raise the cost of spam. Spam filters raise the cost by increasing the amount of spam that someone needs to send before someone will read it. If 99% of all spam is filtered into trash, then sending spam becomes 100 times more expensive. This is also the idea behind white lists -- lists of senders a user is willing to accept e-mail from -- and blacklists: lists of senders a user is not willing to accept e-mail from.
Filtering doesn't just have to be at the recipient's e-mail. It can be implemented within the network to clean up spam, or at the sender. Several ISPs are already filtering outgoing e-mail for spam, and the trend will increase.
Anti-spam laws raise the cost of spam to an intolerable level; no one wants to go to jail for spamming. We've already seen some convictions in the U.S. Unfortunately, this only works when the spammer is within the reach of the law, and is less effective against criminals who are using spam as a mechanism to commit fraud.
Other proposed solutions try to impose direct costs on e-mail senders. I have seen proposals for e-mail "postage," either for every e-mail sent or for every e-mail above a reasonable threshold. I have seen proposals where the sender of an e-mail posts a small bond, which the receiver can cash if the e-mail is spam. There are other proposals that involve "computational puzzles": time-consuming tasks the sender's computer must perform, unnoticeable to someone who is sending e-mail normally, but too much for someone sending e-mail in bulk. These solutions generally involve re-engineering the Internet, something that is not done lightly, and hence are in the discussion stages only.
All of these solutions work to a degree, and we end up with an arms race. Anti-spam products block a certain type of spam. Spammers invent a tactic that gets around those products. Then the products block that spam. Then the spammers invent yet another type of spam. And so on.
Blacklisting spammer sites forced the spammers to disguise the origin of spam e-mail. People recognizing e-mail from people they knew, and other anti-spam measures, forced spammers to hack into innocent machines and use them as launching pads. Scanning millions of e-mails looking for identical bulk spam forced spammers to individualize each spam message. Semantic spam detection forced spammers to design even more clever spam. And so on. Each defense is met with yet another attack, and each attack is met with yet another defense.
Remember that when you think about host identification, or postage, as an anti-spam measure. Spammers don't care about tactics; they want to send their e-mail. Techniques like this will simply force spammers to rely more on hacked innocent machines. As long as the underlying computers are insecure, we can't prevent spammers from sending.
This is the problem with another potential solution: re-engineering the Internet to prohibit the forging of e-mail headers. This would make it easier for spam detection software to detect spamming IP addresses, but spammers would just use hacked machines instead of their own computers.
Honestly, there's no end in sight for the spam arms race. Even so, spam is one of computer security's success stories. The current crop of anti-spam products work. I get almost no spam and very few legitimate e-mails end up in my spam trap. I wish they would work better -- Crypto-Gram is occasionally classified as spam by one service or another, for example -- but they're working pretty well. It'll be a long time before spam stops clogging up the Internet, but at least we don't have to look at it.
Posted on May 13, 2005 at 9:47 AM
• 44 Comments
To receive these entries once a month by e-mail, sign up for the Crypto-Gram Newsletter.
Spam by any other name is still spam. As mentioned it works because it is cheap. Enforcing laws make spamming more expensive to spammers that allow themselves to get caught. Spam will be out there as long as technology maintains the same protocols. Create a protocol where spam is impossible, adapt it and maybe then spam will be no more.
One of these days I'll beat Israel to the punch. Maybe.
Spam works because it's so cheap, and there are always enough morons to respond to it. And spam (like spyware) will never go away, because there is too much money being made. Big-name companies use it (and indirectly, spyware, too).
On the other side of the coin, anti-spam companies make a ton of money selling their products and services. Even the government accepts revenue to (pretend to) address the issue.
Do you really believe that any of these organizations really want to get rid of spam? Give me a break. People selling anti-spam products love spam as much as the spammers themselves do.
Get used to it; it's not going to stop.
As someone that runs an anti-spam/anti-virus service, I'm gonna have to disagree with the assertion that we don't want to get rid of spam. While certain anti-virus companies certainly have fallen to the darker side of inventing and hyping nonexistent threats to make a buck, not everyone is so unscrupulous.
Spam is a problem that threatens to infiltrate and thoroughly pollute every new and innovative method of electronic communication, and believe me, I'd rather see it gone.
Until we find a structural solution, though, I'm happy to provide a competent and effective service to our customers that don't want to see it.
As for proposals to introduce a cost to e-mail (and other forms of electronic communication) in order to make spamming unviable -- I think this is interesting, and possibly the best strategy, but in the back of my mind I worry about saddling the internet with such a burden (cost) when its relative lack of cost previously has been one of the biggest advantages in its phenomenal growth and success.
The only real break in spam distribution comes when marketing people realise that their "cheap" ways of delivering marketing are ineffective or too expensive.
Too expensive is already mentioned above.
Ineffective means that the cost of running business is higher than revenue generated through marketing.
There is an underlying cause for spam to be worthwhile: The low cost of setting up internet business. Most spam received in an electronic way is for such business - or used in the way of a portal to a goods delivery business.
As long as the cost of running a pure internet business is ridiculously low electronic spam is a viable marketing strategy.
I agree that spam will stay with us in the capitalistic world that we live in - it is part how business is done.
Anti spam products help to reduce the effeciency of spam - maybe over time marketing persons will realise that it makes sending electronic spam worthless for the business.
Then we will see a real reduction in spam. Where there is no demand, there will be no offer.
SPIT differs from email spam in that it requires more time to deliver, even if it is a recorded message. The economics of SPIT thus may be much worse than email SPAM. Can anything be done to slow SPIT down further and make it even less economical?
I expect that spit will be fought by many of the same tools that are used to fight telemarketers. The fundamental tool is CallerID and equivalents. These can be strengthened by legal requirements such as prohibiting falsification of ID information, etc. Then you get whitelists and dynamic user examination of CallerID (which can be much smarter than a simple computerized whitelist).
Since spit will be so much cheaper than telemarketing it is hard to predict the outcome. It is possible that you will find widespread rejection of VoIP if spit is not sufficiently discouraged.
Excellent post. However you know I lived in an apartment complex for a while that had a common place for getting mail. Everyday the apartment complex brought in a box for all the people getting their mail to throw all the junk mail in. That box was roughly 3ft X 3ft X 3ft and everyday it was filled. To me that is a lot of garbage, paper, and trees, not to mention everything that goes into ink and so forth. Why I think sometimes the numbers of how much sending regular mail are slightly scewed. When you can actually see it on such a massive scale daily for a complex of about 1000 apartments and how much gets thrown away without even looking at it. Why do we waste so many resources on it.
Thanks, excelent article.
"As long as the underlying computers are insecure, we can't prevent spammers from sending."
No, we can. We can raise responsibility level. For example from computer level to IP block level (expensive because cause needs of maintaining black/while lists of IP blocks), or from user to domain level (much easier, cheaper and very effective - working implementation is a SPF)
Very interesting. The question I have with Spit is, wouldn't it be possible to avoid it by putting one's VoIP number on the DoNotCall registry. Then there's the problem of enforcement, of course. And I don't know enough about the technology to understand whether tracking/blocking spitters is as hard as tracking/blocking spammers. Thoughts?
Wow, you really miss the mark on this post Bruce! I assume this is a response to my comments here:
The issue is that technology is not fixing the problem. The problem is getting worse, and technology is getting better at covering it up in one place (SMTP gateways), only to be outdone in other place(s). Um, I would think you'd be the first to tell the world that this is a losing proposition.
I love it when I hear a user say "hey, no spam today. We must be winning the spam war" then a few minutes later when another hundred variants have mutated beyond the anti-spam controls "hey, what the Q$%@#$ with all this spam?".
Really, Bruce, the status of your personal inbox makes for an amusing anecdote but it's hardly worth claiming as grounds for global victory.
Bruce, here's another reason you're off the mark:
"But the cost/benefit calculation is missing a component: the "cost" of annoying people. Everyone who is not influenced by the marketing message is annoyed to some degree."
That sounds valid, but what happens when users like you believe that security is winning the war on spam and you are no longer annoyed? In fact, you seem particularly hopeful to the point that the next spam message you receive might be a novelty, if not a successful trap that you fall into.
I think you're missing the point here, actually, at least as far as email is concerned - the problem is not that antispam solutions don't work, the problem is that people literally want to put *no* effort into making sure they don't get spam.
To illustrate what I mean... a good and well-trained auto-learning bayesian filter is able to filter out more or less all spam someone receives, while leaving virtually all non-spam alone. Bogofilter has done wonders for me in the past; I had a false negative rate of less than 0,05%, and a false positive rate of less than 0,01% in the end, which arguably is very good.
However, you can't get rates like that just by installing a program; even if a company would do the initial training of the filter for you, you still would have to teach the filter about false negatives and positives when they occur so it could adapt itself. This can be done easily, of course (in mutt, for example, you can just bind this functionality to some hotkeys), but the fact remains that you do need to do this - and the spam filter's quality *will* deteriorate if you don't do it.
So... there is a technical solution, and the fact that it adapts itself means that there's no real way for spammers to get around it; but it requires a relatively high level of user discipline, and in my experience, people who don't get much spam usually aren't willing to invest that.
Let me guess, you don't provide your email address because your mail filters aren't working so well...
"the problem is not that antispam solutions don't work, the problem is that people literally want to put *no* effort into making sure they don't get spam."
Precisely. The "effort" required doesn't solve the problem, it just masks a losing battle for a little while longer.
"the spam filter's quality *will* deteriorate if you don't do it"
Again, you're supporting my point that the technology can't solve the problem. Bruce's inbox is probably clean because of a combination of factors including luck, but I do not see how any of them are a proof of success in security unless we lower the meaure of success.
I don't think we should call mopping floors faster a great sucess, when we should be patching or replacing the leaky roof.
Very intersting article. I have to disagree on one point, though. Though the SPAM surely annoys a given recipient, I don't think the fear of a boycott is too likely.
In order to identify who sponsored the SPAM, the user would need to be opening and reading each one. If, like myself, they never so much as open it, they will never know who that was.
The few I've read seem to be "fly-by-night" mail order type things that could reorganize under another name, and write it off as a cost of doing business.
How can you say the spam problem is solved when you still have spam all over the place here? My point is, when spam products get better the spam will simply start looking more like an ordinary post (be it email, blog entries or whatever else).
"A 30-second prime-time television ad costs 1.8 cents per adult viewer, a full-page color magazine ad about 0.9 cents per reader. A highway billboard costs 0.21 cents per car. Direct mail is the most expensive, at over 50 cents per third-class letter mailed."
You might have a point here. Some might actually measure the success of a medium by the revenue generated, particularly with reagard to advertising that flows on/through it. So perhaps we are victims of our own culture -- SPAM is a naturally occuring phenomenon in our economic system that validates the huge success of email.
Well, here's a sign that Microsoft is resorting to truly primitive measures to fight spam. They have officially released a "tar pit" feature in Windows 2003 Service Pack 1. Note that that they explicitly state that they do not "require or recommend that all customers implement this feature":
The obvious question is are your good messages going into the tar pit along with the bad ones? That would seem, even by analogy, to be the opposite of any advance in filtering technology.
I think you're right, we're in an arms race against spammers.
Does anybody ever win an arms race? The only end in sight would happen when all players but one have spent nearly everything on the race, but maybe I'm just saying that because of history.
No, another way for an arms race to end is when one side evolves or develops a strategy the other side can't adapt to. For a long time, there was an arms race between castle-building techniques and castle-wall-breaching techniques, but cannons pretty much ended that.
If the technology advances to the point that spam essentially never works, or if the spammers' techniques advance to the point that spam filters essentially never work, then the arms race will end. (If spam is unlimited, it will kill whatever communications medium it infests, for pretty standard tragedy of the commons reasons.)
SPIT doesn't just have the potential to ruin VoIP - it has the potential to ruin telephones. In fact, as a VoIP user I have a much larger arsenal to combat SPIT (such as caller*id confirmation, automatic syncing between my address book and my whitelist, local exchanges that are shared only with other trusted individuals) than does a home telephone user.
SPIT has the potential to completely disable the traditional telephone network and its non-existent security (the trust-everyone-because-they've-clearly-signed-a-peering-argeement-to-be-talking-to-me model), and be sure that the major carriers aren't going to do anything about it until absolutely forced.
Blaine appears to be the only other person who's noticed that SPIT doesn't require the recipient to have VoIP, only the sender needs VoIP; and also points out that actually with VoIP on the recipient side, you're a lot more empowered to be able to deal with the SPIT. Also, I see also that at least Precision Blogger has made the mistake of believing that you can only send spit serially. It's trivial to open a whole crapload of SIP connection attempts in parallel, and once connected, you can certainly push the data in parallel too. Remember that spammers through their distributed zombie networks of compromised broadband hosts control more bandwidth than Yahoo, Microsoft and AOL combined. By an order of magnitude.
I have long been predicting the death of POTS voicemail services -- imagine the signal to noise ratio of your voicemailbox being, say, one real message per 5 SPITs -- and you have to listen to the first 5 seconds of a message to tell if it's real or not. Now imagine 1:50.
From one of the comments above: "Create a protocol where spam is impossible, adapt it and maybe then spam will be no more."
Wow, this truism is so nonsensical that it pretty much qualifies as spam - nonsense sent to fill the airwaves with junk!
Could it be that the reason why the good guys are a bit of ahead in the "arms race" right now is that spammers aren't generally smart and that most people who have the expertise to help either side are the good guys? And the fact that most people hate spam.. Also, there isn't much of any information available (that I know of) about attacking spam filters.
Here's one I found with google:
First, I'm slightly amused by the idea that when spam hits VoIP you'd only react to a whitelist. That has been my practice for the past five years already. I have only a mobile phone and, lately VoIP, and while I can see callers trying to get to me, I will only actually get a call signal from numbers/people I previously know.
So how do you get to talk to me? Send me an email. And an email that is informative and with a phone number (just the number, or a short, cryptic message will trap the mail in my spam filters).
Someone above said that spam filtering will not work because spam will just change to become more like real email messages. Well, if spam becomes exactly like real email - informative, relevant, personalized and to the point - I doubt many people would have anything against spam anymore. In fact, that would be the end of spam period.
"Bruce, here's another reason you're off the mark: 'But the cost/benefit calculation is missing a component: the "cost" of annoying people. Everyone who is not influenced by the marketing message is annoyed to some degree.' That sounds valid, but what happens when users like you believe that security is winning the war on spam and you are no longer annoyed? In fact, you seem particularly hopeful to the point that the next spam message you receive might be a novelty, if not a successful trap that you fall into."
If I don't see the spam, then there's no cost to me. There is still a cost to the infrastructure, but either 1) it doesn't matter to the infrastructure, or 2) those who pay for the infrastructure will implement their own controls to pay for it. If ignoring spam in the network is cheaper than fighting it, then that's the best solution. And if I don't see it even though the network ignores it, I'm happy with that solution as well.
I do also agree that our efforts against spam can in no terms be qualified as a success, until this success is such that everyone who has an email account will automatically and effortlessly benefit from it - that is, until people who don't see spam are the rule rather than the exception.
Until the folks who can avoid spam are only the most computer savvy guys like me, you and Bruce Schneier, our efforts against spam are no success, they're failure.
Additionally, until any legitimate email continues to be lost due to spam filters, our efforts are not a success, they're failure.
Serious time, business and opportunity are lost every time an innocent email is caught in the spam filter.
When you get to having 2000 spam emails daily, there's no way you can realistically monitor your spam for such misplaced emails.
I was going to point out the same thing. When Bruce says he's not getting any spam and that the filtering is happening automatically (without any efford from his part), he doesn't actually know how many valid (and possibly important) emails are getting discarded by the spam filter. I don't see any spam on my email either, but then again I'm not advertising my email address everywhere (and probably wouldn't get much spam anyway), but I know I've lost messages because of a spam filter my ISP uses.
While this was a good article and brings out some points that others have glossed over (such as looking deeper at both the real and intangible cost of spam), in the end, there is still no single solution proposed, nothing for people to rally around and focus on..
Look, spam costs virtually nothing to send. So there isn't any control to limit the amount of spam mail, unlike snail mail. Also, it is not always easy to distinguish spam from real mail. Furthermore, opening junk mail or watching an ad on TV never leads to getting a virus/trojan/spyware infection or exposing personal information. Spam email is just not the same as advertising or junk mail!
Fortunately, I use a free Bayesian email filter program which catches virtually 100% of the spam. Based on years of experience and the rating added to the spam header by the software, my Outlook rules automatically dispose of the garbage for me. But most people aren't sophisticated nor patient enough to train and use Bayesian filters.
As to the goal of stopping spam, it ain't going to happen UNLESS we have to pay postage to send email. There's just no other way. PERIOD. As you note, for every measure we take, spammers come up with a counter measure. It's like cops with radar guns vs. the radar detector manufacturer's.
I'm thinking that .005 (1/2 of 1 cent) per email might do the trick to end the spam deluge. If the typical user sends 200 emails per month, a charge of .005 cent per email would be a mere $1.00 extra per month - a pittance to be permanently rid of spam! THAT'S ONLY ONE DOLLAR PER MONTH PEOPLE! ISP's could even give users 200 free emails monthly to mitigate this point.
But the spammer would be on the hook for $5,000 for every 1,000,000 emails sent. That should be enough to take any profit out of that business. Even for legitimate marketers, a small charge like this adds up and would encourage them to ensure that everyone on their lists were opt-in customers. We could kill two birds with one stone.
Furthermore, for those who might respond that spammers will hijack user systems for spending spam email, then the user would be on the hook for paying for letting their system be vulnerable. It's called personal responsibility, something many of us don't want anything to do with these days. Forcing the user to ante up for their lack of knowledge or refusal to learn how to protect their computer would actually be an additional positive, as it would "encourage" users to learn safe computing practices. Nothing like draining $$ out someone's pocket to engender a bit of motivation on someone's part, I say [lol].
Of course, any mention of having to, god forbid, PAY for sending email is 100% GUARANTEED to raise many people's hackles. I find this humorous insight of how many people seem perfectly comfortable paying $3-5 for a cup of coffee, spending $50 weekly on lottery tickets, paying $50-100/month for a cell phone bill and/or paying up to $120/month for a maxed out cable service subscription.
We need to face reality. There is no way to legislate spam out of existence since it can originate from any country on this planet with a good telephonic system. Charging for sending email is the ONLY solution that spammers will not be able to find a technological way around. Half-measures just won't cut it. The sooner we acknowledge this, the sooner we can be rid of this problem and move on to deal with other important problems.
Here's a way to drive up the cost of spam - respond. Every day, pick one or two spam messages, call the phone number given and say, "I got your message, and I don't want to buy your product." If enough people do this, then the spammers will have to bear the high costs of dealing with the negative reponses.
Thanks Bruce, for pointing out that the root cause of spam is
economic. Following that logic, by following the money, leads to the
inevitable conclusion that those buying from spammers are the real
But what might influence someone considering a spamvertized product?
Well, assuming a list of paying customers is somehow obtained, and
publicized, the resulting public embarrassment could have a bigger
influence than Spamassassin and Spamcop together.
"When Bruce says he's not getting any spam and that the filtering is happening automatically (without any efford from his part), he doesn't actually know how many valid (and possibly important) emails are getting discarded by the spam filter."
Actually, I do. I check my spam filter regularly, and the false positive rate is very very low. Admittedly, I had to train it in the beginning. I knows the newsletters I subscribe to, and it has a white list with many of my regular correspondents.
What I don't know are the false positives from any of the spam filters in the network.
"Following that logic, by following the money, leads to the inevitable conclusion that those buying from spammers are the real problem."
I think that's true. Reminds me of the drug problem: you can spend all the money you want trying to disrupt the supply chain, but you're not going to make any real headway until you deal with the demand side.
Bruce you suggested a partial solution by showing in your analysis that the problem has an economic base. Make your negative reaction to spam known to the makers of the products, organize (actually some users are organized, in tech organizations or forums), and boycott companies. Technology cannot babysit your babies, toddlers, and teenagers; it cannot protect them adequately from pedophiles; it cannot give them values. Technology alone cannot solve the spam, spim, or spit problem; it requires humans, us. to get off our arses and do something about it. We are the solution with our pocketbooks.
You say "if you're a clever hacker, you can build your own spam network".
Please call these people by their real name - criminals. The creation and release of worms / backdoors / spam proxies is crime in almost any civilized country. These people are just another breed of criminals. Nothing more.
...you only need a 1 in a 100,000 success rate.
The blame for spam clearly lies with that one individual then. If we can stop this tiny tiny minority from getting the message, then we remove the economic basis of spam completely.
One way to do this is with encryption. The normal procedure is to encrypt a message so that only the intended recipient can read it. What we want to do is reverse it, in order to keep a message from its intended recipient. The process would work as followed:
1. The sender obtains his private key from his a public server.
2. The sender encrypts a message with his private key and send it to someone.
3. The recipient downloads the encrypt message.
4. The recipient obtains the sender's public key from the same public server, decrypts the message, and reads it.
What do we gain from the additional steps is time. Between the time when the message is sent and the time when it's read, we can destroy the message even after it has arrived by removing the sender's public key. In fact, we would destroy all unread messages sent by that person regardless of quantity.
How do we determine when access to someone's public key should be cut off? Simple, through democracy. If you send me a email, then I get to vote against you. Accompanying each message would be a combined hash of the recipient's email address, encrypted with the sender's private key. When enough of these unique tokens are submitted in spam complaints to the key server, it automatically cuts off public access to the key (or just change the key-pair).
Let say the threshold is 50 complaints in a day. If a spammer sends out 10000
spams and the success rate is 0.01%, the probability that the sole person who's interested would open the message ahead of 50 complainers (out of a potential 9999) is pretty slim. And increasing the number of spams sent doesn't improve the odds either.
Actually, I do. I check my spam filter regularly, and the false positive rate is very very low. Admittedly, I had to train it in the beginning. I know the newsletters I subscribe to, and it has a white list with many of my regular correspondents.
What I don't know are the false positives from any of the spam filters in the network.
Posted by: Bruce Schneier at May 15, 2005 03:45 PM
Yes, the spam filters in the network are a BIG problem. I've heard Verizon has a nasty one. Also SBC has a filter in place that bounces mail to the sender with a 544 denied message. They don't even tell the user they are bouncing the message! In fact, I cannot get your crypto-gram newsletter through SBC as it is marked as spam BEFORE it even reaches their email servers (they cleanse through an outside service first). So I now get your newsletter through a web account. But I bet their are a lot of people who don't know what is going on.
If you use SBC hosting or SBC/Yahoo, then you are DEFINITELY not getting some good emails because of this setup.
Anyway, looking over this discussion so far, I see the same attempts to come up with some sort of technology or user education solution. It ain't going to work people! As Bruce says, each step in prevention technology results in another counter-step from the other side. As to educating users, puh-leeze. You've got to know that's a non-sequitur!
The ONLY viable solution is to charge for sending email. Period.
@jojo>>As to the goal of stopping spam, it ain't going to happen UNLESS we have to pay postage to send email. There's just no other way. PERIOD
So who would physically collect payment for email? The ISP of the sender? Much (most?) spam uses hijacked or clandestine accounts, probably bypassing any such payment. How does the receiving ISP know payment was made? Legitimate users will be forced to pay, spammers will not.
Extra work counters the whole point. The biggest problem with spam is that it burdens the network (public and personal), adding huge amounts of processing to handle it makes the situation worse.
"Create a protocol where spam is impossible, adapt it and maybe then spam will be no more." - Israel Torres
My involvement with a VoIP startup only strengthened this belief. When I reviewed the SIP RFCs, I was astonished at how little attention was paid to security. Rather than address these concerns immediately, the protocol soon resembled early SMTP. The problems with scalability, authentication, and spam were already evident. SIP has simply been added to the number of protocols that are functional, but look elsewhere for a generic security solution.
From the article: "This is an economic problem, and the solutions need to change the fundamental economics."
Although true for the most part, this is not entirely an economic problem. As adults, we can be bothered to tidy our inboxes, but how do you offer these useful technologies to children when you know some barnyard porn or bukaki spam will inevitably target their accounts? If someone walked around your neighborhood and filled your mailbox with such flyers, you'd be looking for more than economic retribution. When everyone moves to VoIP (and they probably will) do you want your children to pick up the receiver and hear this kind of telemarketing on the other end?
Don't be surprised if biometric authentication eventually becomes mandatory for all network access, effectively making anonymity on the Internet impossible. There are simply too many commercial and social incentives that can't be satisfied with traditional access controls.
Although Chung Leong's reverse encryption solution is interesting, any wide-scale adoption of email encryption could interfere with the server-side content inspection that still plays an important role after IP blacklisting. And there's no reason to believe that spammers won't generate short-lived disposable private keys, since user names are even more plentiful than routinely discarded spamvertisement URLs. Nonetheless, it isn't the encryption that fights the spam, it's the fact that one must *register* something to send it. Essentially, this is a system to license the privilege to use email (not entirely a bad idea, but very contrary to our current mindset).
Bruce, I think we're losing the arms race. Or, more precisely, we're losing as long as the arms race exists. The reason is that this arms race has a key difference from most: as long as it's running, we're spending money but the spammers are making money.
And when the network providers, or anyone else, pays to implement spam filtering, this is a cost that you, along with the rest of society, shares. You pay more for your Internet connectivity because ISPs spend tens, perhaps hundreds of millions of dollars per year on spam control. You lose opportunities that you would otherwise have: that hacker out there working on a new spam filter is not working on another useful program. I'm writing less open source software than I used to because I'm spending time installing and configuring spam filters instead.
These guys are costing us, as a society, a shocking amount, both in cash and in lost opportunities. It's quite sad that we can't get it together enough to stop even the top few high-profile spammers that live in the United States.
You left out product spam, where gullible kids trying to be trendy pay obscene prices to wear clothes that advertise the company that they're paying to get the clothes. The clothing company gets its name broadcast on people's bodies, and they have not to pay but the cost of making the clothes, which is far less than their profit from the clothes, plus it makes their clothing line appeal to the "trendy." It adversely affects kids and their parents who could be buying much cheaper clothes with no advertising and spending the extra money on more important things. Just a thought.
It is difficult to listen to many of these 'solutions' which attack generic 'spam'.
Spam seems to be defined as 'all sorts of stuff I don't want in my inbox' - a definition that doesn't help describe the problem and potential solutions.
Spam used to be called UCE (unsolicited commercial e-mail) which fits a more narrow profile and often describes the cost effective marketing practices of reputable companies which might honor an 'unsubscribe' request. The folks are the least of our worries - I don't mind getting spam from them because I just might want to know about a great deal on some hot new IPS product. But then a lot of 'fly-by-night' outfits popped up trying to capitalize on making money by the impression or response rate - not caring about the damage left in their wake. These seem to be the primary focus of this post. Should these two subtypes of UCE be treated equally?
The things get worse when we see malware using spam as a transport in the form of mass mailers. This scourge is driven by a wholly different motivation (anarchy or notoriety?)and often requires a very different handling (more likely to be delivered via botnets and virus infected hosts).
A third general category of spam could be case of phishing - again a very different motivation (money yes - but driven by fraud rather than sloppy greed) and sharing some of the delivery techniques of the other two. In this case (as with the second) the economic model is very different than UCE and thus the solutions require this to be considered as well.
A majority of people probably see the spam 'problem' as being contained - so success may be a the right word - for now. (Once enough people get burned this perception may very well change.) If we look at our 'success' then what I see is the success of layered defenses. We have a whole variety of techniques used to reduce the damage, RBLs, gateway based filtering, SPF, Bayesian learning at the desktop, anti-virus tools and DHA and rate limiting.
The one thing we can not do is to condemn our computer users for wanting to use the systems we design and implement.
Schneier.com is a personal website. Opinions expressed are not necessarily those of BT.