ISPs Blocking TLS Encryption
It’s not happening often, but it seems that some ISPs are blocking STARTTLS messages and causing web encryption to fail. EFF has the story.
Comments
Larry Seltzer • November 13, 2014 7:43 AM
John Levine, who knows these things, says it’s an exaggeration. They’re blocking STARTTLS on port 25, but port 587 (where you should be doing STARTTLS anyway) is still available.
Chris • November 13, 2014 8:14 AM
I don’t understand what this has to do with web encryption. As far as I know no web protocol supports anything other than tunnel-mode.
Ben • November 13, 2014 8:14 AM
This behaviour can be caused by the default “ESMTP Inspection” rule on a Cisco PIX and ASA5505 router (and probably all their range). The rule is enabled by default on the PIX and ASA.
It is supposed to be a defence against attacks on (I think) older versions of Sendmail etc. by enforcing the SMTP protocol and cutting the connection if it is violated.
So it may be that:
1) people are behind a cisco PIX or ASA without realising it, and blaming the ISP
2) ISP have enabled the rule on their own Cisco routers at some point on their network without understanding what it does.
Looks like cock-up to me but then again there have been an awful lot of cockups, so I am starting the be more suspicious of them.
Clive Robinson • November 13, 2014 8:17 AM
As more and more web sites are now seeing the light and making https the default or only supported protocol, perhaps email server admins should bounce any emails without security options on.
Provided a suitable “bounce” message is sent back to the originators inbox, this will cause customer services of offending service providers an increased work load, thus hopefully encoraging management to “do the right thing” by their customers…
Simon • November 13, 2014 8:20 AM
Talking only about mail submission misses half of the story. MTA to MTA communication uses port 25–correct, it’s for Mail Transfer not Mail Submission, but there’s no good reason not to encrypt mail transfer too.
Thoth • November 13, 2014 8:22 AM
We have actual began discussion of the STARTTLS stripping in the previous post here: https://www.schneier.com/blog/archives/2014/11/narrowly_constr.html#c6682687
Secure communications over network is just part of the whole secure environment. End-to-end security, secure systems and setups, OPSEC, after the fact handling of emergenices and all that are part of the bigger picture.
PGP/GPG/SMIME would have protected the contents but not the routes it takes unless it is tunneled through some kind of pseudo anonymous routing tunnel which we did discuss in that post and Clive Robinson did mention about secure multicast communication which attempts to spread load of suspicion to nearby nodes and then nodes sending out relayed messages at different timing to attempt to confuse observers (https://www.schneier.com/blog/archives/2014/11/friday_squid_bl_449.html#c6682721).
Clive Robinson • November 13, 2014 8:35 AM
@ Bruce,
These MITM failures to observe security settings or deliberately change them, are as bad as the fall back failures of security.
What would be your views on sending an RFC proposal to the IETF that there should be an official “sunset clause” put in place for all protocols involving security. Such that there is a “grace” period of say one year before protocols that have been superseded by a more secure replacment should be nolonger approved for use and should not be routed on public networks?
I know people will moan that it is “draconian”, “not possible” or “to costly”, but these arguments are easy to debunk.
Importantly it will encorage embedded hardware manufactures to spend a few cents more to ensure their products are upgradable conveniently or risk losing market share. I’m aware of the “knock on effects” of this but the downsides are minimal at best and have quite a few positives, including “open source hardware” solutions becoming much more prevalent.
65535 • November 13, 2014 8:37 AM
“2) ISP have enabled the rule on their own Cisco routers at some point on their network without understanding what it does.” – Ben
Maybe and maybe not.
Somewhat like “I gave Congress the least untruthful answer.”
Possibility number 3
3) ISP(s) have enabled the rule on their own Cisco routers at some point on their network understanding what it does – but hides that fact from the customer.
Possibility number 4
4) ISP have enabled the rule on their own Cisco routers at some point on their network understanding what it does – but hides that fact from the customer because of a NSL or CALEA order.
paul • November 13, 2014 9:03 AM
I know I shouldn’t be surprised any more, but seeing yet another set of companies do the kind of stuff that used to get people 5AM wakeup calls from guys with badges and automatic weapons does still give me pause.
I think there’s also an incredible business opportunity here for people who want to set up free wireless hotspots that do deep packet inspection and modification for anyone willing to pay. As long as it’s construable somewhere from the terms of service, it seems it’s OK.
stvs • November 13, 2014 9:35 AM
“This type of STARTTLS stripping attack has mostly gone unnoticed because it tends to be applied to residential networks, where it is uncommon to run an email server”
This is really a story about a misconfigured email server. The larger issue is it’s now standard industry practice to block smtp port 25 outbound AND outbound for residential customers.
First, a secure email client must be configured for secure smtp port 587, not port 25.
Second, the smtp server should be configured not to accept insecure connections on port 587.
Third, the workaround for incoming mail is to pay for a service like DNSMAdeEasy’s Mail Server Forwarding, which will resend port 25 incoming packets on some other nonstandard port that isn’t blocked by the ISP, then use port forwarding to remap those packets to port 25 on the LAN.
As a matter of practice, residential ISP customers must regard port 25 as blocked and route around that fact.
Michael35673 • November 13, 2014 9:41 AM
Looks like a misconfiguration on client side. STARTTLS tries but does not enforce TLS, and should not be used if the e-mail server is known to support TLS. Since it falls back to non-encryption on failure. TLS should be used instead.
DMP • November 13, 2014 10:51 AM
The irony being we wouldn’t have this problem if we didn’t deprecate “wrapper-mode” SMTPS (port 465) in favour of SMTP+STARTTLS on port 587.
Bob S. • November 13, 2014 11:54 AM
Luckily, I guess, there is no need to strip SSL/TLS from Time Warner email services.
Everything goes plain text, except the webpage login, which of course is plain text at the server level and easy pickings.
This innovation saves time and money for the corporation/nsa so they can work harder to merge with another great company called Comwaste/nsa or something like that.
People would be protesting in the streets if they really knew how bad things are in the world of bits and bytes….but mostly it’s quite invisible, secret and unaccountable. Plus they lie. A lot.
Anura • November 13, 2014 11:55 AM
@DMP
Yeah, it was an incredibly bad idea to move to STARTTLS over the old SMTPS protocol; it pretty much removes all protection against man in the middle attacks unless you configure the client specifically to force an SSL connection. More good reasons why SMTP should be deprecated.
Honestly, all the internet protocols need to be migrated to only support encrypted connections. If you don’t have a trusted certificate, just use ephemeral keys and indicate the connection is not trusted (or don’t indicate that the connection is trusted); it really is as simple as that.
Daniel • November 13, 2014 12:43 PM
My view is that this is another example of the internet being too complicated to trust. Maybe it always was but the lack of users meant that there was not a great deal of interest from nefarious parties, so it is only become apparent now. But if I were a “bad guy” I’d steer as clear away from technology as I possibly could. It’s simply an unnecessary attack surface.
John Hardin • November 13, 2014 1:15 PM
The irony being we wouldn’t have this problem if we didn’t deprecate “wrapper-mode” SMTPS (port 465) in favour of SMTP+STARTTLS on port 587.
Unfortunately SMTP+STARTTLS on port 587 has the extra contextual baggage of “authenticated MUA“, which you may not want for MTA-to-MTA transfers for reasons like bypassing spam scanning.
thevoid • November 13, 2014 3:08 PM
i went through all this fairly recently, because i was looking for a mail
account with full encryption (imaps/smtps). smtps(465) is still used, even
if technically depreciated. i use a program called ‘msmtp’ as an MTA
(technically a plugin for mutt), which supports smtps, and also found
openmailbox. org (a free site, also has other encryption methods, signing,
etc).
as @stvs mentioned above, port 25 is blocked inbound and outbound. but i
have tried using port 587 on a number of servers with no luck. my isp says
it doesn’t block it, in fact recommends using it, but i’m not so sure.
of course, from system to system this data is all floating around cleartext
unless MTA-MTA transfers are encrypted, and i don’t see any real technical
reason they can’t be.
highly_trained_hypocrite_sociopath • November 13, 2014 5:51 PM
One thing I always get a kick out of is the engineers and admins behind these entities who always go to cons and have liberal statements for the public..
It’s like those people who watch porn but tell people who ask that watching porn is for losers..
DOMESTIC SPYING IS BAD!! .o0(YEAH WE STORE DATA STREAMS.. I KIND OF WORK FOR THE NSA AND HAVE An AGENTS ONLY JACKET.. I PLAY GOLD WITH THEIR PR GUY)0o.
Jonathan Wilson • November 13, 2014 6:38 PM
I have seen arguments suggesting that ISPs are turning this on because a number of spam bots are starting to use STARTTLS to hide the spam from sniffing by ISP-level spam bot detectors.
Daniel • November 13, 2014 9:28 PM
@Jonathan Wilson
But of course. That’s the issue Bruce harps upon. Encryption protects the good guy and the bad guy just like plain text exposes the good guy and the bad guy. So at the end of the day one has to make a choice. (1) you let some bad guys through in order to enable the good guys or (2) you expose everyone.
Some people will insist that there is option three. That you can hide the “good guys” traffic and expose the bad guys traffic. LOL. All the evidence indicates that this way lies tyranny.
65535 • November 14, 2014 2:28 AM
I did a quick check of a handful of customers who use a full email servers their residence And use Verizon DSL [somewhat difficult task – note I said DSL not mobile internet services].
The two customers I found both used Micros@ft Servers with Exch@nge and a dynamic IP with a “No-IP” type of DNS service. Their email servers do work – but both are using SSL/TLS only over standard ports.
To maintain confidentiality for the customers I can only say one client uses a deprecated SBS 2003 with exchange over 443 and RWW ports [remote web workplace]. And, doesn’t use a mail forwarding service [Active Directory, DNS with A records – I did not ask about reverse look-up records].
The individual’s email recipients must a Pro version of Micros@ft client OS and accept the self-signed certificate, join his “domain” or be in his AD database [and accept the related security template for AD]. I assume the recipients reach the server by port 80 then go to port 443.
The other client was more reserved in commenting. This individual uses Server 2008 and Exchange 2007 [with AD or directory services, DNS and so on]. This individual does use an email forwarder and “No-IP” style of dynamic DNS update via router [I forgot to ask about the certificate – probably has small group email base].
I would guess very few people have access and an old SBS 2003 [or newer Server 2008 server with Exchange] plus the skills to make it work.
I have located a an old Cisc0 Pix/ASA and may try to run transparent firewall with default settings to see if the Cisc0 Pix settings Do overwrite the StartTLS flag [to duplicate Verizon’s actions – if the customers will allow].
Chuck • November 14, 2014 6:18 AM
I run my own mail server through a residential line. I work hard to ensure that STARTTLS encryption is offered when receiving mail from the Internet. A server is not required to accept or encrypt any communication according to the SMTP protocol which runs on port 25. But I would be very upset to find that my ISP has decided to interfere with encryption or communication that comes to or from my ip address.
Angel • November 15, 2014 12:21 PM
@highly_trained_hypocrite_sociopath
One thing I always get a kick out of is the engineers and admins behind these entities who always go to cons and have liberal statements for the public..
It’s like those people who watch porn but tell people who ask that watching porn is for losers..
DOMESTIC SPYING IS BAD!! .o0(YEAH WE STORE DATA STREAMS.. I KIND OF WORK FOR THE NSA AND HAVE An AGENTS ONLY JACKET.. I PLAY GOLD WITH THEIR PR GUY)0o.
I think that sort of conclusion comes from:
1) Imagining posters here really are espousing liberal sentiments against surveillance, but secretly work for the NSA
2) And/Or, noticing that some posters dance around the subject, and at the same time seem to admire the US Government… Fan Boys. Like you find anywhere.
They are critics, but also pundits.
They do not really think out what their amore means. Would they get a job looking at other people’s junk? Or maybe giving orders? Maybe they would become masters of disguise and have multiple forms of government created ID? Or to simply be on a secret mission for their boyfriend, the government, and his big hunka USDA meat.
It is like why people sleep together or have crushes, but never actually think out the future.
Angel • November 15, 2014 12:37 PM
As the article its’s self points out: It IS common… for email not to have encryption.
It may be not so common for ISPs to strip the encryption. Which is doubtful, considering the size of the ISPs doing this and the difficulty of catching them doing this.
This is because governments around the world have been reading your emails all along and using that data for profit.
There are several weak points in the STARTTLS protocol, however. The first weakness is that the flag indicating that a server supports STARTTLS is not itself encrypted, and is therefore subject to tampering, which can prevent that server from establishing an encrypted connection.
To a person not involved in MITM security issues, or who reads about them but has never actually tried to experiment with these issues… this may seem excusable. Who would expect, for instance, to suddenly discover that trusted doctors usage of leeches actually is a bad idea, or that sticking a metal instrument through the eye canal to disable a part of the brain is not bleeding edge science.
Or who would expect that it actually is not best for people with a different color skin to have separate water fountains, bathrooms, and have less rights?
The public should not have security for their communication. Since the telephone was encrypted it has been possible to listen to everyone. That is very valuable communication and has helped build American industry.
Even worse, if the American public starts to do it, and then the economy can not longer be controlled because their information is not able to be used at will, the other nations will maybe be smart enough to follow suit. How can you steal and control their businesses and governments if you don’t know what they are saying and thinking??
If someone is making a big trade, and you plan to profit from that — how can you, if you no longer know when they are doing this?
This will destroy the global economy and make it chaotic.
Economies must be controlled.
Devil’s advocate sarcasm, obviously.
ASN • November 17, 2014 6:08 PM
Angel said:
“Even worse, if the American public starts to do it, and then the economy can not longer be controlled because their information is not able to be used at will, the other nations will maybe be smart enough to follow suit. How can you steal and control their businesses and governments if you don’t know what they are saying and thinking??”
Wont ever happen, which is why I sort of find these conversations humorous. Even if everyone encrypted everything, it wouldn’t make a difference to the NSA. What they can’t do today, they can throw billions at and do tomorrow. Money is no object. I remember hearing a CIA official say “What NSA wants, it gets.”
There is no technical solution to this issue. NSA will always be ahead of academia due to their privileged nature, budget, and special frame of reference on the Internet (i.e. having the backbone tapped domestically, under the ocean, and abroad).
Bruce is right: it has to be done politically. And considering Obama (the most leftist president in U.S. history) doesn’t seem inclined to make much of a change, I find a political solution highly unlikely. Perhaps a libertarian would make changes, but I find it unlikely anyone from the two major parties would do much to rock the cradle. And I don’t think it’s because of political ideology. Politicians, no matter the party, have one thing in common — they are control freaks. Knowing everything about everyone is power. And once you have a power like that, it’s hard to give it up. That’s why they hacked Angela Merkle (because they could).
Nick P • November 17, 2014 7:11 PM
As Thoth pointed out, there is a technical lesson to learn from this and it’s that you should use tech that doesn’t trust the middleman (much or at all). There’s quite a bit of it with varying usability and price. Even if subverted, those all still provide a higher baseline by ensuring the crypto is actually used to block any scumbags that aren’t in on their scheme. And if they’re not subverted, they’re at least honestly trying to deliver the service, blocking you from maybe more scumbags.
Protocols like PGP, OTR, etc. Need stronger endpoints & probably purpose built devices for using those protocols. Tinfoil Chat is a great start on the security end. User adoption of such limited solutions is unlikely to happen massively. If anything, people keep making the tradeoff to greatly increase their personal risk for certain features. And black hats have plenty of risk to exploit.
Ali • December 1, 2014 5:54 AM
I’m not entirely 100% sure but I believe ISPs in Oman (where I live) are doing this to block access to certain sites that are circumventable through https, notably VPN service providers, for example trying to visit https://www.strongvpn.com/ results in a “connection interrupted” on firefox or any other browser.
Subscribe to comments on this entry
Leave a comment
Sidebar photo of Bruce Schneier by Joe MacInnis.
CallMeLateForSupper • November 13, 2014 7:41 AM
“It doesn’t happen often”. So it doesn’t happen to every transmission an ISP handles. When it does happen though, are the same clients always affected? That is, does this “feechur” target individual persons? Either way, it adds a non-zero amount of fuel to discussions of neutrality.