Shortened URLs, produced by services like bit.ly and goo.gl, can be brute-forced. And searching random shortened URLs yields all sorts of secret documents. Plus, many of them can be edited, and can be infected with malware.
Academic paper. Blog post with lots of detail.
Posted on April 18, 2016 at 6:00 AM • 28 Comments
Convenience has its Price • April 18, 2016 6:41 AM
Don’t use cloud services, don’t click on shortened links or personalized tracking links (with unique IDs) like in email.
Don't use a 'smart' phone and block advertising.
They are all just too dangerous (besides addicting)!
r0b0 • April 18, 2016 7:01 AM
TIL: Some people thought that hiding a public document behind a URL shortener somehow makes the document private.
RonK • April 18, 2016 7:18 AM
@ r0b0
I think you oversimplify; for many purposes, a sharing link for a web document service which has an embedded key with, for example, 256 bits can be considered effectively private.
What I learned is: most people have no understanding about how important entropy is for various forms of security. Again, not surprising.
Gabe • April 18, 2016 7:41 AM
The difference between the Google response and the Microsoft response is amazing.
paldubee • April 18, 2016 7:53 AM
Were shortened URL's ever meant to "hide" or "secure" content? If they were, then the mistake is in the user, not the technology.
David • April 18, 2016 8:03 AM
I'm not sure this is really a security risk with short urls, more an education issue when somebody needs to share sensitive/private data (i.e. don't use short urls!).
I think in this case there is a problem with the end service. The paper mentions Microsoft's OneDrive and Google Maps because urls could be modified to access or modify other content. The conclusion sums this up.
Clive Robinson • April 18, 2016 8:14 AM
I guess at some point somebody must have thought they could hide behined shorten URLs and found in certain limited cases they can...
Such limited cases are for the likes of hiding link spam and malware from users.
Because browser designers did not add a feature to expand tiny URLs so that the user could see where the links were realy going prior to actually downloading a page. Likewise a blog etc moderator who might use black lists find that spamers etc just use Tiny URLs instead of the real site URL which has been blocked. This has been made worse by the likes of Twitter, making Tiny URLs a necessity, and pre-conditioning users to think Tiny URLs are "safe" to click on...
j4g9w6t • April 18, 2016 8:21 AM
Let's not forget that shortened URLs are also used for surveillance. Most shortener services allow the creator of the URL to see analytics about the traffic that passes through.
I wish there was a browser addon that could un-shorten links. It could follow all the redirects similar to "curl -ILs http://short.url/xxxxx", then replace shortened URLs with un-shortened ones in the links' href attributes. That way users would know the links' real destinations by hovering the pointer over them like normal.
Daniel Martin • April 18, 2016 8:50 AM
@paldubee You ask:
Were shortened URL's ever meant to "hide" or "secure" content? If they were, then the mistake is in the user, not the technology.
No, but long URLs (containing 256 bits of entropy, and sometimes more) were meant to secure content, and then short URLs were presented to users as a way to easily share those long URLs. Somewhere, the idea that making a short URL would undo the security in the longer URLs was lost.
(also, shame on Microsoft for making that traversal vulnerability - the same accessToken in the URL shouldn't give access to the whole account)
I don't think file names contain any entropy, you just enumerate them... As for "shortening" an already low entropy (ascii/utf8?) source??
Now, maybe it helps randomize what would otherwise be sequential access to a specific user...
Here, let's frame this the way you guys frame stuff for me: it's /security through obscurity/. Nothing to see here expect the respective responses from said corporate overlords. If it was even rudimentarily secure they would be .htaccess symlinked folders right?
The only security is randomly serializing direct access to your data, ONE still makes your 'address' "shared" even if the intent is not to go "public" you still have to drive by every house on that Street.
Phil Beesley • April 18, 2016 10:06 AM
Yonks ago, we talked about the Semantic URL. I might exist at http://mydomain.isp.com/home.htm, or html. My anti-whoever rant would exist at http://mydomain.isp.com/nasty-hate-speech.htm. Every reader would expect an argument by reading the URL. Having learned a lesson, perhaps I wrapped my nasty hate speech in other words.
I'd post a shortened URL to hate sites hosted by a subtle mate. If you can't read the URL, you don't know what you are expected to read.
Shortened URLs are bad for all of us.
Slime Mold with Mustard • April 18, 2016 10:09 AM
When requesting directions online, I always start with a point several miles from my actual starting point. If I have the vaguest idea of the destination, I indicate a nearby intersection. I have a large collection of quite detailed paper maps and atlases. I do not mark these with pencils or pens.
Mike Gerwitz • April 18, 2016 10:15 AM
My first reaction to this was "no shit". That said, this study is important to demonstrate exactly what the problems are, and what further issues can be exploited after the fact; I'm not downplaying that.
With that said: the first thing I tried when discovering URL shorteners long ago was to modify the URL and see what I got, and I'm sure that's been the case for many thousands of others. No brute forcing, but I can't imagine others haven't tried before this.
The more fundamental issue with one of these in particular---mostly-random URLs to remote resources---is that these URLs are _far_ from a substitute for authentication. GET requests are logged on remote servers, might be leaked through referrer headers, and are stored in the client (browser) history; there's numerous means of getting at any of those. If you paste the URL to someone else, who knows where it will go. And what of services that automatically send your uploads to these services---if you upload data to a "cloud" bug tracker, or chatroom, which automatically generates these URLs?
And to top it all off---you can't audit any of it. Who accessed your sensitive data on AWS? Who discovered your browser history? Who's looking at server logs? What are others you sent the URL to doing with it? Who got the message about that resource you uploaded, or who has access to that link?
You should reject "cloud" services for many other reasons, but I've always found it incomprehensible why this is considered an acceptable practice to begin with.
Twitter is partially responsible for the proliferation of shortened URLs. Their 140 character limit (which was initially based on being able update a status using text messages) essentially forces people to use shortened URLs. Since Twitter is such a widely-used social network, a change in their site policy to require direct links could increase security.
Gerard van Vooren • April 18, 2016 11:33 AM
@ Clive Robinson,
"This has been made worse by the likes of Twitter, making Tiny URLs a necessity, and pre-conditioning users to think Tiny URLs are "safe" to click on..."
Sometimes I think that it's just stupidity^x. It reminds me of Microsoft with their ingenious Windows XP that allows the "I love you" virus because of three stupid mistakes (of which only one was actually dealt with, the other two still exist today, probably because they didn't want to deal with the installed base). It's quite funny that Ted Unangst just wrote a blog post about this featuritis. The conclusion is that being too clever could be very dumb. But the problem indeed gets worse when people are also an masse being conditioned.
ianf • April 18, 2016 12:38 PM
@ Clive Robinson, Gerard, 💤
What's with sudden Twitter-shorturl-bashing?
You seem to have overlooked that internally, and during transport between server and clients, all its URLs, whether original-long, or shortened elsewhere, are encoded as 10-character long ordinal t.co strings – which, given the volume and throughput of the message firehose, is the only viable solution (i.e. all Twitter URLs sans the scheme are 15 characters long… I tried to calculate the capacity of that methodology, but kept arriving at astronomical figures, or the calc indicated overflow error). It's only when a tweet is delivered outside the realm of the service, that the internal URL is turned back into the originally submitted one (hence subject to the maximum msg length).
Eric • April 18, 2016 2:49 PM
I concur with @paldubee: URL shorteners were designed for convenience, not security.
In the world of commercial law I see plenty of documents shared with similar services particularly when the communication is between partners/attorneys of other law firms or sent to a private clients. (Documents shared internally are normally accessible via shared drives/folders).
HOWEVER even though the documents exist in the cloud you can only access them with a password; plus the 'shortened' links are nothing of the sort, e.g.
http://cloudstorage.com/zlaw/f8TYUrg6oX6HqIudCeRYMPUuC
I've seen documents limited to X number of downloads and/or expire after Y number of days. On occasion I've also seen Rights Management applied (to deter printing, copying etc.)
The technology to keep documents secure is out there which is why I think blaming URL shorteners is pointless. One such service which seems to handle 'encrypted links' well is Tresorit - I'd like to see Microsoft and Google implement the same (although neither are zero-knowledge... yet)
https://tresorit.com/blog/how-to-share-files-with-anyone-securely-a-video-tutorial/
https://tresorit.com/files/encrypted-link-whitepaper.pdf
Certainly corporate Data Loss Prevention Policies should kick in to stop sensitive documents from being shared on insecure cloud services.
When used sensibly, and in conjunction with a secure cloud, I don't see the problem. User education is the issue in my opinion.
Example of how secure 'shortened' links could work:
Our encrypted links offer a zero-knowledge alternative which makes sharing single files possible without requiring a Tresorit client.
Encrypted links can be used similarly to public links: you post or send a link to someone, which has an authentication token inside. The link allows the recipient to download the file.
However, the decryption key is stored in the link and is never sent to Tresorit servers. Thus, the content of the file remains encrypted on the server, and is never accessible by Tresorit.
To achieve this, encrypted links use a fragmentation identifier in the URL (preceded by a # sign) to store the key. Once the user opens up the link, a JavaScript is loaded into the browser, which decrypts the data stream received from our web-server.
The JavaScript based AES256 decryption and HMAC-SHA512 checking code has been heavily optimized to provide rapid access almost identical to an unsecure public link, but without the risk.
One crucial difference between sharing a tresor and an encrypted link: public key cryptography is used to establish a secure key exchange when sharing a tresor. Encrypted links contain the secret key without additional protection. If the encrypted link is accessed by unauthorized parties while being transferred to the recipient via email, for example, your data can be accessed.
Bob • April 18, 2016 3:27 PM
"Making things easier to get to makes them easier to get to."
This non-story has already received too much attention.
Jim • April 18, 2016 5:53 PM
Coming from the early world of 'drive by'/'watering hole' attacks, I hated them when they came out, and still hate them.
Too easy to disguise the final destination, is simply 'why'.
They came out at just the time when finally [many] JS [implementations] was "fixed" and ensuring links were very difficult to hide the final destination.
Dirk Praet • April 18, 2016 6:12 PM
Interesting observation by @dakotasmith : "Twitter's URL shortener is a Colombian domain. Facebook's domain is in Montenegro. Bit.ly is Lybian. NSA only targets international traffic."
Vesselin Bontchev • April 19, 2016 2:20 AM
@j4g9w6t There's an add-on for that.
John • April 19, 2016 4:59 PM
The character limit on Twitter,
Prevents limericks — if you're a quitter.
But with one little cheat,
You can quickly defeat
(see line 1)
Peter • April 20, 2016 1:52 PM
"NSA only targets international traffic"
Riiiight ... and dogfood is made from dogmeat .
Sorry, but that was just SO funny I couldn't help myself ..
Bumble Bee • April 22, 2016 12:59 PM
Security risks of stupid IT policies
I'm posting from Baltimore County Public Library in Maryland and the local login page for the wifi network is using an URL that reads like
—>http://1.1.1.1/fs/customwebauth/login.html?switch_url=http://1.1.1.1/login.html&ap_mac=xx:xx:xx:xx:xx:xx&wlan=BCPL-PUBLIC-WIFI&redirect=http://www.myhomepage.example.org/
which clobbers an *ahem* public IP address that happens to be registered and apparently used in some capacity or another in the Research district of Melbourne, Australia.
Err, umm, are we qualified to work in IT when we can't read RFC 1918 and use specifically allocated private address space for such purposes?
10.0.0.0 - 10.255.255.255 (10/8 prefix)
172.16.0.0 - 172.31.255.255 (172.16/12 prefix)
192.168.0.0 - 192.168.255.255 (192.168/16 prefix)
Does anybody know how to contact the IT department at BCPL? I don't work in IT anywhere because I just can't deal with this kind of interminable stupidity.
[bbee@localhost ~]$ traceroute 1.1.1.1
traceroute to 1.1.1.1 (1.1.1.1), 30 hops max, 60 byte packets
1 gateway (192.168.42.129) 0.622 ms 0.613 ms 0.685 ms
2 1.1.1.1 (1.1.1.1) 6.522 ms 6.702 ms 6.650 ms
[bbee@localhost ~]$ whois 1.1.1.1
[Querying whois.apnic.net]
[whois.apnic.net]
% [whois.apnic.net]
% Whois data copyright terms http://www.apnic.net/db/dbcopyright.html
% Information related to '1.1.1.0 - 1.1.1.255'
inetnum: 1.1.1.0 - 1.1.1.255
netname: APNIC-LABS
descr: Research prefix for APNIC Labs
descr: APNIC
country: AU
admin-c: AR302-AP
tech-c: AR302-AP
mnt-by: APNIC-HM
mnt-routes: MAINT-AU-APNIC-GM85-AP
mnt-irt: IRT-APNICRANDNET-AU
status: ASSIGNED PORTABLE
changed: hm-changed@apnic.net 20140507
changed: hm-changed@apnic.net 20140512
source: APNIC
irt: IRT-APNICRANDNET-AU
address: PO Box 3646
address: South Brisbane, QLD 4101
address: Australia
e-mail: abuse@apnic.net
abuse-mailbox: abuse@apnic.net
admin-c: AR302-AP
tech-c: AR302-AP
auth: # Filtered
mnt-by: MAINT-AU-APNIC-GM85-AP
changed: hm-changed@apnic.net 20110922
source: APNIC
role: APNIC RESEARCH
address: PO Box 3646
address: South Brisbane, QLD 4101
address: Australia
country: AU
phone: +61-7-3858-3188
fax-no: +61-7-3858-3199
e-mail: research@apnic.net
remarks: ++++++++++++++++++
remarks: + Address blocks listed with this contact
remarks: + are withheld from general use and are
remarks: + only routed briefly for passive testing.
remarks: +
remarks: + If you are receiving unwanted traffic
remarks: + it is almost certainly spoofed source
remarks: + or hijacked address usage.
remarks: +
remarks: + http://en.wikipedia.org/wiki/IP_address_spoofing
remarks: + http://en.wikipedia.org/wiki/Regional_internet_registry
remarks: +
remarks: ++++++++++++++++++
nic-hdl: AR302-AP
tech-c: AH256-AP
admin-c: AH256-AP
mnt-by: MAINT-APNIC-AP
changed: hm-changed@apnic.net 20110822
source: APNIC
% This query was served by the APNIC Whois Service version 1.69.1-APNICv1r0 (UNDEFINED)
Nobody Special • April 22, 2016 11:09 PM
@Bumble Bee...
I think the following explanation will shed some light on your predicament.
http://www.revolutionwifi.net/revolutionwifi/2011/03/explaining-dhcp-server-1111.html
In short:
"Almost all public documentation by Airespace and Cisco show examples assigning the virtual interface an IP address of 1.1.1.1, which has become the de-facto standard for most deployments. The virtual interface is not attached to any physical ports, is not routable across the network, and is only used by the WLC internally."
Bumble Bee • April 24, 2016 1:45 PM
@Nobody Special
Oh, great, now I need a Facepalm — I mean Facebook — profile — err, I mean, never mind.
Bumble Bee spends most of her time online researching flowers on Google and downloading pollen via Bittorent. She navigates by sunshine and dead reckoning.
Now let's get back to the interminable stupidity: companies like your aforementioned Airespace and Cisco forcing shitty proprietary "de-facto" standards and riding roughshod over RFC's. Now get off my lawn or you'll be stung hard and it will really hurt.
In short: the internet is an engineered-obsolescent engineered-insecure pile of manure and it is terminally broken. I just can't compete with horseflies.
Bumble Bee • April 25, 2016 9:14 AM
Oh, excuse the language. The word "shitty" in the above post isn't mine. It was inserted maliciously by someone else. If you can't secure your own blog, or if you insist on altering my posts by inserting additional foul language, I will definitely go post somewhere else.
...or maybe it's the stupidware on my computer, or all that Cisco garbage (forging SSL certs and such) that all my online communication has to go through.
@John
Haha, brilliant. The exact length ... and pleasingly authentically Learean, thanks to the repeat.
Subscribe to comments on this entry
Photo of Bruce Schneier by Per Ervland.
Schneier on Security is a personal website. Opinions expressed are not necessarily those of Resilient, an IBM Company.
Leave a comment