Help with Mailing List Hosting

I could use some help with finding a host for my monthly newsletter, Crypto-Gram. My old setup just wasn’t reliable enough. I had a move planned, but that fell through when the new host’s bounce processing system turned out to be buggy and they admitted the problem might never be fixed.

Clearly I need something a lot more serious. My criteria include subscriber privacy, reasonable cost, and a proven track record of reliability with large mailing lists. (I would use MailChimp, but it has mandatory click tracking for new accounts.)

One complication is that SpamCop, a popular anti-spam service, tells me I have at least one of their “spamtrap” addresses on the list. Spamtraps are addresses that—in theory—have never been used, so they shouldn’t be on any legitimate list. I don’t know how they got on my list, since I make people confirm their subscriptions by replying to an e-mail or clicking on an e-mailed link. But I used to make rare exceptions for people who just asked to join, so maybe a bad address or two got on that way. Spamtraps don’t work if you tell people what they are, so I can’t just find and remove them. And this has caused no end of problems for subscribers who use SpamCop’s blacklist.

At a minimum, I need to be sure that a new host won’t kick me out for couple of spamtraps. And if the solution to this problem involves making all 100,000 people on the list reconfirm their subscriptions, then that has to be as simple and user-friendly a process as possible.

If you can recommend a host that would work, I’m interested. Even better would be talking to an expert with lots of experience running large mailing lists who can guide me. If you know a person like that, or if you are one, please leave a comment or e-mail me at the address on my Contact page.

Tags: ,

Posted on August 3, 2015 at 5:58 AM • 48 Comments

Comments

Marnix Petrarca • August 3, 2015 6:10 AM

Heya Bruce, why don’t you ask Gordon Lyon (Fyodor) of Insecure.org? He certainly will know something about it with his hackers lists, and good sense of privacy?

Regards,

Marnix

NL

mark • August 3, 2015 6:15 AM

Note, that reconfirmations usually come at a severe cost in terms of subscriber count. For example everyone who runs newsletters into a folder and reads them irregularly (but does read them). Also everyone else who changes his mind when prompted.

I think you should treat this mailing list as a business asset and avoid reconfirmation at all costs.

Bernard • August 3, 2015 6:20 AM

Try Adam Engst ace@tidbits.com

ISTR TidBITS handling similar problems before, and their mailing list is pretty big. Adam will certainly point you in the right direction.

PT • August 3, 2015 6:49 AM

I think the first question is; do you really need to maintain a “monthly newsletter” at all? It might have been a cool feature about 20 years ago, but today … I mean no offense …

Sorin • August 3, 2015 7:11 AM

Hi Bruce,

I am having the exact same problem described here: http://www.sorinmustaca.com/2015/07/23/itsecuritynews-info-says-farewell-to-mailchimp/

Basically, Mailchimp kicked me out because of the spam (which same as you, I have no idea how it got into my list) and because of “keywords” related to malware, spam, hacking. Imagine that I run a list with “IT Security News” where this is the main content. 🙂 .

Here is the solution I found and it works perfectly so far: Google’s Feedburner Email Subscriptions.
It is free, it doesn’t have any restrictions if you run a clean list (and we do).
The problem which i need to solve is to inform the existing subscribers to resubscribe. 🙁
There is no way to import the list into Feedburner. And to send an email to over 2400 subscribers is a bit too much for me. You mention 100K subscribers, so this is an even bigger issue.

I am curious how you solve it, I will watch this thread.

Good luck!
Sorin

Gervase Markham • August 3, 2015 8:09 AM

Ironically, you could help solve your spam problem using click tracking. If you sent out a message with an image with a personal URL in it, then you wouldn’t need to confirm the accounts of everyone whose image loaded. That would mean you’d only have to do reconfirms on a subset of the subscribers (those whose image didn’t load), which would reduce your dropout rate.

Click tracking doesn’t only have evil uses…

So maybe MailChimp is the way? Go with it, with an apology to subscribers and a promise you’ll turn off clicktracking as soon as you can. In the mean time, use it to identify active accounts and do reconfirms on the others.

Frank • August 3, 2015 8:09 AM

MailChimp can also be used without need for subscribers to confirm subscription again, by facilities to import email addresses and new subscribers can be added manually, again without additional confirmation. Plus MailChimp could be contacted directly to request an exception to the click tracking, due to the nature of schneier.com focus on security and privacy…. Plus http://www.yourmailinglistprovider.com/ might be worth a look.

Barack Obama • August 3, 2015 8:11 AM

I’m all for getting rid of spam, but I’m not sure that this approach of using fake emails for spam detection is useful. I know that when I’m confronted with some web form that just asks for an email address for “validation” or the like – say, some random blog – if reply confirmation isn’t required I will just use a random email that I don’t and can’t control, that is 98% likely to pass automated validation. That’s the only possible defense against a particular problem: the operator who is honest now, but might face a serious incentive to sell your name on later, and be tempted by it then. Nowadays I mostly use addresses at one of the dedicated throwaway email providers without bothering to register them first, but that’s not the only way to do it. If even a few other people work this way – and they surely do – there will be lots and lots of “unused” email addresses floating around in legitimate records out there. I know it’s a gray area, but the fact is that I’m not willing to assume anything about people’s future honesty if I’m not given carte blanche to ensure it myself.

Jeff Rivett • August 3, 2015 8:43 AM

As someone mentioned elsewhere in the comments, you can probably negotiate with Mailchimp to disable click tracking, if it’s that important to you. Frankly, I don’t see the problem with it. Anyway, I’ve been using Mailchimp for a couple of clients that have a few thousand subscribers and find it generally excellent. Far better than anything else I’ve tried.

Darren • August 3, 2015 8:46 AM

Brian Krebs seems to like HostWinds, since they were willing to lose money by getting rid of ill-behaved mailers in order to protect their integrity; but they don’t seem to offer a listserv-like service (just hosting), so that would mean running your own mailer software.

I have heard very good things about SimpleLists; their privacy policy and ToS seem to fit what you’re asking for and they have a good reputation among my circle of acquaintances.

George • August 3, 2015 9:06 AM

The idea that “I only use this email address for you, and I got spam. Therefore you’re breached” is wrong. I’ve been involved with investigations that involve email interception without breaches. Postcards written in pencil, people.

Mike the goat • August 3, 2015 9:12 AM

Hi everyone (yes, it’s been a long hiatus). I’d love to help but we are all too scared here of our upstreams to host anything that puts out even what one would consider to be low volume amounts of mail. Spamcop/spamhaus et al – although I do respect what they are trying to do have been quite overzealous and in turn have scared a lot of the Internet into erring too far on the side of caution, resulting in a cottage industry of companies who’ll “mail it for you” – who often rotate through netspace quickly leaving vast netblocks listed in these dnsrbls. Not exactly an ideal situation. One would think that we’d have some kind of multicast email solution these days, but then again we haven’t even got widespread adoption of multicast TCP. A shout out to Nick P, Clive, namewithheld and figureitout!

Mike the goat • August 3, 2015 10:01 AM

Dirk: at the risk of hijacking this thread, indeed I am. Have been bogged down with mudane life and a medical condition – am much better and am slowly re-engaging with the sec community! I guess I should move this into the Friday column – but yes, I am indeed still alive and doing okay! 🙂 Great to see you again.

Mark Newsome • August 3, 2015 10:04 AM

Reconfirming your emails probably won’t work. It’s likely that those spam trap addresses were added to your list intentionally by an attacker who will simply resubmit them.

Ignacio Arriaga • August 3, 2015 10:14 AM

Hello Bruce,

If you want, I use to follow your site and I am the CTO of a site similar to mailchimp. we can provide service to you without problem with the circunstances that you have.

The goal of the comment it is only to offer you our tool, not to be spammy, so if you want you can write to me on my email and leave the comment unpublished.

Thanks,
Regards.

Daniel • August 3, 2015 11:16 AM

While I understand the business side of it I’m with PT. Take this roadblock to think deeply about whether you need a mailing list at all. Frankly, I haven’t subscribed to a mailing list in at least a decade and it is very rare when I find there is something I want to read that is exclusively on a mailing list. But that’s just me. I might not represent the norm.

od74m5 • August 3, 2015 11:29 AM

Bruce, the EFF has a pretty big mailing list and I’m sure their technologists and activists have dealt with these types of issues before. You could ask them how they manage it and what systems they use.

od74m5 • August 3, 2015 12:01 PM

Bruce, another suggestion is the Riseup collective. They’ve been around since 1999, they care a lot about privacy and security, and they provide mailing lists to activist groups by donation. I’m sure they’d be thrilled to host your list. You can contact them here.

Also, since other commenters are saying you shouldn’t even have a mailing list in 2015, I’d just like to say I do value mailing lists. If I don’t use Facebook, Twitter, or Kindle because they’re all in the business of surveilling users, that leaves only RSS and email for subscribing to this blog. Plus, email is the only way to get a consolidated set of the blog posts for the last month.

MyFirstNameIsPaul • August 3, 2015 12:18 PM

I’m not sure what you consider a reasonable price, but I have found the Mailgun service by Rackspace to be very easy to use, but note that my needs are pathetic.

They have a simple pricing calculator you can use to see what the service would cost you. (Note that if you need some small resource to manage other aspects of your list, you could get a $15/month 512 MB Rackspace Public Cloud Server and get an additional 40,000 free Mailgun emails per month ~$20 worth.) Given the level of customer support I have received for my tiny little account, it is hard for me to believe they would boot any legitimate customer, and I have found numerous very large companies using their service to send me transactional email.

For reputation issues, I see that schneier.com currently has no SPF record and your current provider didn’t use DKIM on the most recent newsletter.

All the reading I have done states that having an SPF record, DKIM record and DMARC record will help decrease spam rating, plus the aggregated spam reports that large providers send for DMARC can have useful information. Another important factor is reputation of the IP address or addresses that send the mail. (Mailgun has an option for $59/month to have a dedicated IP address.)

Although you could balance all of that against just hiring someone to set up a server with dedicated IP address, since it sounds like you are primarily just doing a monthly blast, so you could space that out over some time (24 hours?) such that even the most anemic server could handle the load.

I’m just an enthusiast, so I don’t personally know anyone, but these two people have written stuff I learned a lot from:

Mike Hillyer How To Send One Billion Email Marketing Messages Per Month.

Reason (yes, this person is publicly a handle) has a blog at exratione.com with contact info and articles on various easy-to-implement solutions to email management and other admin issues. Reason runs his/her own newsletter that is plagued with deliverability issues due to the nature of the content, so has even gone so far as to write a Drupal module for managing bounces more effectively.

Muffin • August 3, 2015 12:37 PM

I feel there’s a few choice words that could be said about SpamCop (and blacklists in general) here. They may be a good idea in theory, but in practice, if even Bruce Schneier, whose diligence, technical expertise and trustworthiness I’m sure we can all agree are beyond any doubt, cannot meet their standards, how can mere mortals hope to?

Remember, the point of blacklists is to solve problems for people, not create them. When it comes to spam, false positives are just as much a problem as (in fact arguably a bigger problem than) false negatives are.

Nick P • August 3, 2015 12:41 PM

@ Mike the Goat

Good to see you’re making it. Look forward to next convo on Squid thread.

Bystander • August 3, 2015 1:45 PM

Just wanted to add freelists.org. They are normally not used to such volumes, but asking won’t hurt.

I like their general attitude and also their attitude towards privacy. Archive would be included.

@ Mike the Goat
I am just a bystander but it is great to see you back here.

Kai Howells • August 3, 2015 5:40 PM

If you’re seriously considering MailChimp and the only reason you’re not using them is because of their click-tracking, then why not do a soft-launch with them – import your list, or a subset of your list, and send out a few test mailings before doing the real thing.

Alternatively, reach out to them – I’m sure there’s at least one person on their staff who are aware who you are, and it’s possible that they may be willing to bend the rules as you have a very well established history of sending out email newsletters.

If you want to go the DIY route, then this is relatively easy to do, however that would necessitate sending email from just one or a few IP addresses, which would be easy to get blacklisted if there are spam trap email addresses already in your list.

Running a mailing list could be done on pretty much all but the smallest VPS plans from most providers, but you’d really need to get your list in shape before moving it all over.

Clive Robinson • August 3, 2015 6:01 PM

@ Mike the Goat,

Nice to hear you are on both the mend and the up.

Just don’t make the mistake of trying to do to much, which unfortunatly I did this weekend and only just avoided getting draged off to hospital yet again. Definitely not the way to enjoy what’s left of summer, especialy with so much fruit early this year needing to be picked and preped and frozen for a preserve and chutney making marathon this autumn or dried, candied or pickled for biscuit, cake and bread making or to add to game and stews through the winter and spring, wild boar and pickled plum being a much liked celebration table piece 😛

Kurt Seifried • August 3, 2015 7:53 PM

So I looked around a lot, and we ended up hosting it ourselves, basic setup is a cluster of small machines acting as inbound servers (running spam filtering and anti virus and whatnot to provide basic sanity), then a list server running mailman, and then a cluster of small machines to act as outgoing servers (so mailman can basically send mail at several thousand messages per minute since it doesn’t have to talk to remote mail servers that may be slow/etc.). Total cost to run on AWS is like 200$ a month.

I was unable to find any spam/virus/etc filtering company that doesn’t charge by the user as opposed to simple volume (e.g. by the gigabyte). You can outsource deliver to something like Amazon Simple Email Service (SES) but I honestly wouldn’t bother.

With mailman you could trivially invite everybody to subscribe, they’d have to click a URL hit “reply” to the email, so you shouldn’t lose to many people. This may/may not catch the existing spam traps depending on how smart they are.

David • August 3, 2015 11:21 PM

Maybe check out Exact-Target?

The seem to be a Tier-1 email service
provider. I find them one of the
least objectionable from an email
recipient/admin perspective: they
support configuring proper reverse-DNS
for IPs dedicated to customers
(so I can easily whitelist desirable
senders) and know how to keep sending
IPs clean and off DNSBLs.

My neighborhood association uses
Constant Contact and they seem
ok as well. Can’t be too expensive
if the NA can afford them.

name.withheld.for.obvious.reasons • August 3, 2015 11:22 PM

@ Mike the Goat
Good to hear from you Mike. Fortunately in your absence little has transpired that would cause one to pick up a brick. Will follow-up on a squid…

Figureitout • August 4, 2015 12:54 AM

mike the goat
–Good to hear from ya finally, you can stop chewing on my shoelace now :p

Haggishunter • August 4, 2015 8:20 AM

Dear Mr Schneier: I wrote it elsewhere but here it is more appropriate. May I suggest you try something similar to the website http://www.takimag.com . That seems to work quite well. Best Regards.

Josh Kirschner • August 4, 2015 12:06 PM

HI Bruce,

I second Kai’s comment above about “warming up” Mailchimp with a few innocuous emails until they drop the click-thru requirement or simply disclose in your upcoming emails that click-tracking is turned on and you will turn it off as soon as MC allows – people can make the choice whether to click or not. Personally, I could care less whether you have click tracking on, and people can always visit your site directly if they see an article they want to read but don’t want their click tracked.

Beyond that one hurdle, Mailchimp is a very flexible, highly-reliable and easy to use platform. We’ve used them for our sizeable list for years and found it to be a far better platform than Constant Contact or Aweber.

One more note, Mailchimp will remove any known (by Mailchimp) bad addresses when you import your list. That may help with your SpamCop issue. Though, SpamCop and others generally block on an IP basis, not a list basis (see: https://www.spamcop.net/bl.shtml), so not clear why you would be getting blocked for a bad address on your list unless you’re sending under a dedicated IP (i.e., not a shared IP that most ESPs, like Mailchimp, use.) or your current ESP is really the root of the problem.

FWIW, old unused email addresses sometimes get repurposed for spam traps (the logic is that since there is no legitimate reason to be sending to that address, it must be spam). If your list hasn’t been properly handling bounces, that could be part of your issue. I would suggest you clean your list of those who haven’t engaged but since you are not tracking clicks, you can’t do that. 🙂

Frank • August 4, 2015 3:06 PM

I’m with PT and Daniel. Yours is the only newsletter that I’ve kept and, to be honest, it’s only out of principle because of my job.

Maybe it’s time to reconsider your options. I’ll stay subscribed no matter what you do, but every month I’m wondering how long making it takes you and who really reads it instead of simply coming here.

James Woulfe • August 4, 2015 10:05 PM

Mr. Schneier. Check out the web site hosted here in NYC, Panix.com. Been around a long time, I think they could help you out. Jim.

Mike the goat • August 5, 2015 4:56 AM

@Nick,nwfor,et Al: thanks everyone for the welcome back! It’s great to be back on such a wonderful forum.

Re mass mailing platforms: I second the recommendation of mailchimp. That said most ISPs make distinction between a listserv (majordomo etc) sending mail and bulk mail. A little bit of communication with your upstream can go a long way when sending large volumes of legitimate mail, kinda preempt any abuse complaints. These dnsrbls are a real problem with false negatives.

Mail is really one of the last remaining vestiges of an internet that was fundamentally based on trust esp with its non existent resistance to sender forging. Yes, SPF/dkim/etc can prevent it but only when there is mass adoption. In the mean time widespread MUA support so at least the user agent can tell the viewer that the sender’s supposed domain publishes records and appends keys to all mail so the mail is definitively proven to be a forgery is a good step forward but I can think of many reasons why this behavior wouldn’t be ideal, e.g. people who send all their mail including those for other domains using their ISPs outgoing mail server.

My cellphone provider blocks outgoing SMTP except via their relay and they block all the usual ports. I have an IPSEC tunnel to my home network always up on my phone just so I can send mail through my usual relay. Another interesting habit is many cell providers intercept and transparently cache http, which makes me uncomfortable.

@Clive: Thanks. Am taking it nice and slow, which is not my uh, default setting.

Rhialto • August 5, 2015 5:31 AM

The extremely annoying thing about DKIM is that it breaks mailing lists. I have seen suggestions to rewrite the From: headers from the real author to that of the mailing list to “fix” mails sent via a listserv. As far as I’m concerned, that amounts to fraud…

me • August 5, 2015 8:04 AM

@Bruce, you should take your email “in house” — literally! Worked for Clinton. Business class line and some static ips. Ask about changing the reverse DNS records for the ips when ordering. Its usually just an email to support once its setup. Then lock down the SPF records as someone said. Email can handle the odd storm or two… DJB is your friend! He knows a thing about email.

Brian R • August 5, 2015 9:20 AM

I used to run multiple mailings per month, 25-50k subscribers, using Campaign Monitor, which worked very well, would handle the re-subscribing if necessary, makes it easy to take care of DKIM and SPF, doesn’t force tracking links.

If you have a list of confirmed subscribers separate from your unconfirmed ones, you could import them separately and force the unconfirmed ones to confirm, rather than the whole list.

Because of their pricing plans, it was very economical when we did multiple mailings per month, but less so with only one monthly mailing, so the cost might be prohibitive.

Solid solution, though.

Ankur • August 7, 2015 6:15 AM

i am surprised only on recommendations of ESP where you can get your list cleaned and spam trap removed with list cleaning service. i was researching on web on list cleaning and spam trap removals and stubmled on it. just search for good list cleaning service and get spam trap removed and then host the list on any provider suits your cost and delievery. mailchimp is not the only ESP. you can find whole bunch and if needed email me i can connect you to two providers i work with and they offer great service.

but first thing if you get good list cleaning service do let me know as i am looking for same.

Tony H. • August 10, 2015 5:13 PM

I second the suggestion to use Lsoft’s LISTSERV®. (Yes, LISTSERV is a registered trademark, at least in the US.) They’ve been around for almost 30 years (and their roots go back beyond that), and so have seen the “old” Internet of the 1980s and 90s, and managed the transition into the “new” one. And they’ve coped with the major breakage that Google, Yahoo, and AOL have inflicted on all mailing lists over the last year or two.

There are roughly three ways to use it: License a copy and run it on your own system(s). Works fine on a small Windows or UNIX box/image. Or use one of their hosted services. Neither of these is free. Or perhaps the best bet: get one of the several universities who run technical lists on their own copies of LISTSERV to host it for you. I’d guess at least one would be happy to have such a prominent list hosted among their others.

Some university based LISTSERV servers that host technical lists:
U. of Alabama
Marist College
U. of Arkansas

Lsoft maintains a global public list of lists at Catalist

Sorry this sounds like marketing. I have no connection with them; I’ve just been subscribing to LISTSERV lists since 1991.

Leave a comment

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.