Schneier on Security
A blog covering security and security technology.
« New GAO Cybersecurity Report |
| EU Counterterrorism Strategy »
July 20, 2010
Economic Considerations of Website Password Policies
Two interesting research papers on website password policies.
"Where Do Security Policies Come From?":
Abstract: We examine the password policies of 75 different websites. Our goal is understand the enormous diversity of requirements: some will accept simple six-character passwords, while others impose rules of great complexity on their users. We compare different features of the sites to find which characteristics are correlated with stronger policies. Our results are surprising: greater security demands do not appear to be a factor. The size of the site, the number of users, the value of the assets protected and the frequency of attacks show no correlation with strength. In fact we find the reverse: some of the largest, most attacked sites with greatest assets allow relatively weak passwords. Instead, we find that those sites that accept advertising, purchase sponsored links and where the user has a choice show strong inverse correlation with strength.
We conclude that the sites with the most restrictive password policies do not have greater security concerns, they are simply better insulated from the consequences of poor usability. Online retailers and sites that sell advertising must compete vigorously for users and traffic. In contrast to government and university sites, poor usability is a luxury they cannot afford. This in turn suggests that much of the extra strength demanded by the more restrictive policies is superfluous: it causes considerable inconvenience for negligible security improvement.
The Password Thicket: Technical and Market Failures in Human Authentication on the Web:
Abstract: We report the results of the first large-scale empirical analysis of password implementations deployed on the Internet. Our study included 150 websites which offer free user accounts for a variety of purposes, including the most popular destinations on the web and a random sample of e-commerce, news, and communication websites. Although all sites evaluated relied on user-chosen textual passwords for authentication, we found many subtle but important technical variations in implementation with important security implications. Many poor practices were commonplace, such as a lack of encryption to protect transmitted passwords, storage of cleartext passwords in server databases, and little protection of passwords from brute force attacks. While a spectrum of implementation quality exists with a general correlation between implementation choices within more-secure and less-secure websites, we find a surprising number of inconsistent choices within individual sites, suggesting that the lack of a standards is harming security. We observe numerous ways in which the technical failures of lower-security sites can compromise higher-security sites due to the well-established tendency of users to re-use passwords. Our data confirms that the worst security practices are indeed found at sites with few security incentives, such as newspaper websites, while sites storing more sensitive information such as payment details or user communication implement more password security. From an economic viewpoint, password insecurity is a negative externality that the market has been unable to correct, undermining the viability of password-based authentication. We also speculate that some sites deploying passwords do so primarily for psychological reasons, both as a justification for collecting marketing data and as a way to build trusted relationships with customers. This theory suggests that efforts to replace passwords with more secure protocols or federated identity systems may fail because they don't recreate the entrenched ritual of password authentication.
EDITED TO ADD (8/7): Four blog posts by the authors of the second paper.
Posted on July 20, 2010 at 1:52 PM
• 40 Comments
To receive these entries once a month by e-mail, sign up for the Crypto-Gram Newsletter.
@"We also speculate that some sites deploying passwords do so primarily for psychological reasons, both as a justification for collecting marketing data and as a way to build trusted relationships with customers. This theory suggests that efforts to replace passwords with more secure protocols or federated identity systems may fail because they don't recreate the entrenched ritual of password authentication."
I wrote about passwords and eCommerce in my master's thesis. I think this excerpt is half right. Part of it is psychology, but I don't know if it is primary. A large part of it is ecomonics and marketing (as they acknowdge). When it is employees you are authenticating, you can make them do as you want.
However, when it is customers, you have to balance between security and usuability, as they can take their business elsewhere. if books.by.adam.com requires a secure authentication mechanism they don't have or don't understand, they may very well log onto books.by.eve.com set up a password and get their darn book.
Sometimes, one has to sacrifice some security for usability.
At other times, they have to sacrifice some risky customers in order to implement more secure practices.
Both extremes present risk of incidents (at the side of too easy) and dissatisfaction (at the side of failure). Being the most secure site in the world is useless if no one uses you, and being the easiest to use doesn't help if no one trusts you. It's all about balance.
Given the govt's aim as reported in the TOS post earlier, the Bonneau, et. al., paper's methods could well be considered criminal.
These two seem to reach diametrically opposed conclusions:
"We conclude that the sites with the most restrictive password policies do not have greater security concerns."
"Our data confirms that the worst security practices are indeed found at sites with few security incentives, such as newspaper websites, while sites storing more sensitive information such as payment details or user communication implement more password security."
Whom to trust, whom to trust? Microsoft or Cambridge? :-P
In the blurb, I don't see an acknowledgment of differing security assessments.
If I have staff to detect a compromised account quickly, I might allow weaker passwords and have a lower ticket volume from simple 'forgotten password' issues.
Password Policy cost is a complex issue.
Higher value target have to deal with things like phishing quickly and I bet which may lead to a desire to shift money away from password policies and towards monitoring and intervention.
Web security is really a nightmare. I can't say someone who made this kind of trade off is wrong. Its got to be complex to figure this out.
I do not understand why developers store passwords in the clear. Jesus, it's a matter of adding some random salt (takes 2 seconds to implement), and using Password("passwordhere") or md5("passwordhere"). Bloody hell!
"This in turn suggests that much of the extra strength demanded by the more restrictive policies is superfluous: it causes considerable inconvenience for negligible security improvement."
This hardly seems a justifiable conclusion to an otherwise accurate synopsis. Stronger password policies do increase security, but at the cost of user acceptance. One can hardly derive the opposite by saying that because large sites don't use a strong password policy, strong password policies don't increase security by any large factor.
Look at the recent iTunes hack, where many accounts were compromised, likely through a weak password policy. Thats a large site with a fairly poor password policy. They get a lot of users, so the few minor attacks against those are easily justified by the extra profits they will get from having a larger user base.
I think you likely have to look at what the site is being used for, as the second paper does. If you are protecting something secret (like credit card info, or company secrets), you cannot afford to lose those, as that could be losing your entire profit margins via losing customers or showing your new technology before its ready so competitors get an advantage. If you are protecting someones profile page from being defaced, then you can afford to have those hacked, especially if the site is free or erroneous charges can be refunded easily (especially with digital goods, like music, videos, etc).
@Brian: from the intro of the first paper:
"Some of the largest, highest value and most attacked sites on the internet such as Paypal, Amazon and Fidelity Investments allow relatively weak passwords."
So I guess the question is who has higher security requirements than those sites? If they make it work why can't others?
I notice only ten entities are listed as checked in both articles.
That's 10/75 for one study and 10/150 for the other, not enough to detect clear trends across samples.
The studies are welcome, yet more needs to be surveyed before other confounding or driving factors can have their influences be apportioned out
I wonder how defense and state department policies follow these factors in varying usage situations, from PR portals, to anony-reporting, to other security protocols, in varying countries. Lithuania vs Myanmar vs UK vs France vs Israel vs Iran, for example.
Also, how do they manage cloud mediated versus point-to-point internet traffic?
Any one care to give their three letters worth?
Websites that don't enforce strong passwords I don't mind so much, but what really bugs me is websites that prohibit strong passwords. I've been told by sites on more than one occcasion that passwords can't contain interpunctuation (why not?) or that they exceed the limit of 8 characters (why that?). Pretty stupid, and pretty annoying.
@rich, because they don't know better. Every year a new batch of PHP newbs are let loose on the world...
I'm always afraid to register with any site that a) looks like an OSCommerce site, or b) requires you have no more than n characters in your password -- where n is like 8. This implies (possibly erroneously) that they have a password varchar(8) field in their DB. I can't fathom another reason to restrict the password max length if you are encrypting it.
@Kevin Amen. American Express has a limit of 8 characters for their passwords. They also don't allow certain punctuation. Ironically, they do require that you have both at least one letter and one number in your password.
I can't think of any (good) reason for a website to structure their password policy like that, especially a financial institution.
>sites with the most restrictive password policies do not have greater security concerns.
Amen - software developer sites are the worst.
The programmers know the importance of good security - so insist you have an 8 character, upper/lower/symbol/digit passwd.
All to protect some php-BB question and answer site for an SDK only 6 people use.
"Security without inconvenience isn't very secure." -- me
Seriously, Rich. The various flavours of Unixes got this right thirty years ago.
That various applications are still screwing this up is an atrocious reflection upon computing technology, standards, and on passing off the costs to someone else.
Those who forget their history are just plain doomed.
Okay, now I've let off a bit of steam, I think the point is that lousy password policies allow all sorts of shenanigans, but stringent password policies don't help much.
eg., the password 'Jon' is readily guessable, and so should not be allowed. The password 'Jon45651' is still only eight characters, but will never be guessed.
So the takeaway for those implementing password policies is that you should, as an administrator, run a quick dictionary attack against any given password. If that attack fails, the password is good enough.
All further just pisses your users off.
Forget some horrible policy document, just test them. If they pass, they're fine. If not, tell the user why not, and make them try again.
@Kevin: "I'm always afraid to register with any site that a) looks like an OSCommerce site, or b) requires you have no more than n characters in your password -- where n is like 8. This implies (possibly erroneously) that they have a password varchar(8) field in their DB. I can't fathom another reason to restrict the password max length if you are encrypting it."
Yeah, a max size on a password field - any size - is a sure-fire red flag. The site is not protecting your password, and if they aren't protecting yours they probably aren't protecting an admin's either, and anything you put in their trust is likely to be in the hacker domain soon enough.
A hash of your password doesn't care how long the input password is, and that's all the site should be saving.
Personally, I love most the entropy-reducing rules, like it may not repeat any characters or numbers. Why sacrifice so much entropy when what you are really trying to stop is a password like "aaaaaa1"?
To the point of the articles, I think that "required complexity" in passwords is also somewhat a reflection on community risk.
For instance, a site which is doing nothing more than protect your contact email address: if you aren't willing to put that behind a secure password then it's just your own fault. On the other hand, a site where you have administrator rights over a forum, or where your account authentication is being used as a spam reduction mechanism, if your account is compromised many other users are affected. If your account being compromised is likely to affect the corporation or agency setting the policies, then those policies are likely to be much more strict than otherwise.
''Jon45651' is still only eight characters, but will never be guessed.'
I'd guess it immediately following Jon45650. But why guess it when a keylogger can just take it?
[I know I know... just having a lil fun...]
@dmc "These two seem to reach diametrically opposed conclusions:"
The Cambridge paper seems to distinguish between content sites and commerce sites, while the MS one looks at commerce vs gov an edu.
What they say isn't really in opposition. Cambridge finds that content sites have super lightweight policies (e.g. 1 character), while commerce have middle of the road (e.g. 6 character medium strength). MS finds that gov and edu tend to have super strong (e.g. 8 chars, upper, lower, digits and special).
Cambridge concludes that blogs and content sites have lightweight policies becuase they're not protecting much. MS concludes that gov and edus have super strong policies, because they don't give rat's ass about their users.
A weak password policy (or no policy at all) is pretty reasonable if the information protected by it has low value - why bother? If a user feels his information is somewhat important for him he may choose a slightly stronger password. So it is perfectly ok for many classes of web sites, especially ones offering free services, like social networking, file/photo sharing, news portals etc. to have weak password policies. To the contrary, having a strong policy would annoy most users and they will go elsewhere.
Having said that, let's look into paid-for services and e-commerce. Here not annoying users is even more important as it has a direct influence on revenue. Therefore many e-commerce sites have weak or non-existing password policies. But if all an attacker guessing a password could do is to browse user's preferences, contact details or past orders and order goods in the victim's name (but having to pay himself) it's more or less acceptable.
The problem is that many online sellers store client's CC data "for convenience" while still having weak password policies, again "for convenience" of users. From a merchant's point of view it is pretty reasonable as the upside (larger client base) benefits him and the downside is an externality - it's mostly client's problem when someone defrauds him.
The solution has been discussed here many times: authenticate the transaction, not the user. But this requires cooperation from the banking world.
Now a small excursion into my personal online shopping experience on both sides of the Great Pool. I'll compare US-based online stores with Polish ones. (Don't know much about other European countries' practices as virtually everything I am unable to buy locally is cheaper - including shipping, duties and taxes - when ordered from USA than from Germany or UK.) The sample size is pretty small, but shows a tendency in business practices.
When shopping in US-based online stores the only viable payment option is a credit card, with all its downsides. (International trade issues aside, there aren't any other options even for US-based customers). I try not to have the CC number stored permanently, but some sites insist on it. Few stores allow to enter the CC details for one-time charge without (explicitly or not) storing it. Some allow to remove CC details form your accounts profile later. Not that I worry much as I use a 'virtual' CC for this so I am pretty safe.
On the other hand, shopping in Polish online stores I very rarely have to use a credit card. Most banks here have developed their own ways of payment collection (incompatible with each other - there's no standard for it) which payment processing companies have integrated into their systems and provided generic APIs for merchants. It seems complicated, but user experience is easy and quite convenient. If you have a credit card for start, you probably have an account with some bank already. Most banks offer online access, so you probably already have enabled it and are using it (unless you are the paranoid type). Most of those banks that offer online access also offer associated payment processing service.
The user experience while shopping is like this: when you click 'pay' at shop's checkout page you get redirected to the secure payment processing company's website with amount of payment and all details already filled in. You then get to choose payment method - either by a credit/debit card or you just click your bank's logo presented on the page to choose a direct payment. In the latter case you get redirected to your bank's website, where you accept the transaction. All banks' systems I have seen use two-factor authentication for that. After a successful autentication you are redirected back to the payment processing company's web page, and then to the store's web page. Only the payment status is propagated back to the merchant, so even if you use a credit card its details are never seen by the merchant, only by payment processing company. For the user the process is no more complicated (and often less) than typing in CC number, expiry and printed name, but more secure as it does not reveal sensitive data to anybody but your bank - which you have choosen to trust already.
I agree this is not a perfect solution (as opposed to some common standard) but it works pretty well here. Most small to medium online stores use this approach as this is cost-effective for them and moves payment security responsibilities onto someone else. Some really big ones have their own systems in place (sometimes allowing to store CC details optionally) but integrate the banks' direct payments anyway, as these become very popular and even have dwarfed the card payments volume as not all people here have credit/debit cards and even if they have thay may be reluctant to use them for online payments because they feel it's insecure (and it is - to some extent).
I shake my head when sites have complexity requirements, then ask for a "password hint." "Complexity" and "hints" are a contradiction.
I find it strange that the first report comes from Microsoft.
Windows system passwords' hashes (NTLM) are neither salted nor employ any key strengthening mechanism (e.g PBKDF2) to thwart dictionary/bruteforce attacks.
aikimark - "Security without inconvenience isn't very secure."
Surely you meant "without convenience"? If security measures are inconvenient, people will usually try to bypass them instead of enforcing them.
"If they make it work why can't others?"
Do they really? Paypal offers cheap footballs, and has reported that fraud rates with those footballs has fallen.
Of course, online authentication is much easier to secure then offline, so password policies don't really matter that much when you can lock out an account after several failed login attempts.
Sorry, to follow up the last post, the first article:
"Some of the largest, highest value and most attacked sites on the Internet such as Paypal, Amazon and Fidelity Investments allow relatively weak passwords."
Their definition of "high value" is a site which has value *to the individual setting the password*. For those sites, the individual already has every reason to set a secure password, and doesn't need an artificial impetus to force him to do so.
Their definition of "low value" sites included .edu and .gov sites, which, granted, do not have much valuable information from the perspective of the individual user, but which *do* have critical information for the institution at large. Since the user has little intrinsic benefit from the pain of remembering a more complex password, these sites have to impose the need for complex passwords.
I think my theory makes more sense than just accepting that "low value" sites don't care about their users.
can God devise a password so fiendishly complex that even he cannot memorize it?
or, in the case of us mortals - if you require me to have a capital letter, three numbers, a mark of punctuation used only in Finland, and a Masonic symbol, the only thing that'll get me to do is forget my password or write it down somewhere. Same with changing my password every 90 days.
Bank of America just made me change my password after seven years - of course I promptly forgot it and had to call up their social engineering guinea pig farm. I got in by identifying only things that someone would know if they stole my last statement out of my mailbox.
@Jon: Depends, are those the last 5 digits of your SSN? ;-)
@Jake: "...the only thing that'll get me to do is forget my password or write it down somewhere."
Depending on your threat model, there might be nothing wrong with writing down a password.
Generally speaking, we do quite well in securing small valuable pieces of paper.
Doesn't any restriction make things less secure by reducing the potential password space?
@Paeniteo - you are quite correct on that count. I should write down the first halves of all my passwords, leaving the second halves a common prefix or suffix that I can leave committed to memory.
I found this quote astounding:
"... storage of cleartext passwords in server databases, and little protection of passwords from brute force attacks."
How does anybody not know to prevent those two things. I run an online game. It's a free game, and is very small. There's almost nothing of any real value inside (though a few players donate to keep the servers running and get some nice items in exchange). It took maybe a week and maybe a couple thousand accounts for the first jerk to set up a dictionary attack against the players and start breaking into accounts.
If something as meaningless as my little bit of entertainment draws in a dictionary attack that quickly, just about anything else that requires a password is going to be far more appealing as a target. It's simply unacceptable to leave that avenue of attack open.
As for password hashing, even the simplest of tutorials (and I learned most of my PHP from online tutorials) will almost always tell you to hash your passwords, even if it doesn't tell you anything else about security.
I did of course quickly patch the game to prevent dictionary attacks, and knew to salt and hash passwords from the beginning, and that aspect hasn't been a problem since. The real security issue for me has been faked form submissions. The number of things people can think of to post and break intended behavior is astounding.
One thing that i do have to say before i even read these two studies just looking over the comments. The person who's talking about using md5 seriously needs to think about what he's saying. I hope he's not a developer in charge of any type of security.
Also i cannot get over how many sites out there don't even hash the users passwords once before sending it upstream. Just by doing that alone besides having a better security policy for the passwords would help you since the amount of time for the person to crack the password is thus increased. Also on php i am then encrypting the data with blowfish(since mcrypt doesn't support threefish yet and i use the full 448 bits for the key). The bcrypt is set to do ~150 passwords per second at the current time that is because I don't want the logins to be screeched to a halt until i get a login server.
Do simple things on your database end, and you'll increase your security immensely. Also there is NO excuse for not using SSL now a days due to people like startssl offering a free ssl certificate. Anyone dealing with users logging in to their server should use ssl along with some sort of hashing on the clients end and then also on the server end.
As others have said, any site that says passwords with only n characters and also says you cannot use certain characters in your password are a big no no. For example blizzard's battle.net service only allows for you to use $! and . for the special characters. Also i doubt that they're hashing it since if they were there'd be no need to filter out characters.
They're a huge company and they're doing that kind of crap. So it doesn't surprise me that others are using such weak policies.
If your language of choice for the authorization of the server doesn't support bcrypt(for some odd reason or you cannot get support for it from a library that is safe) use sha512 at the very least, if you don't i pity you and your users.
Also to continue what i was saying since i forgot to say something...
You should also limit the amount of logins that one can do whilst failing to some value like 10(at the most) per hour and of course follow basic common sense guidlines but i guess common sense isn't truly so due to what we keep seeing in the world.
First, I agree with you -- people who refuse to put certain security measures in place when it comes to software are taking the lazy man's way out. Claiming time/cost etc for some of the issues is just plan wrong.
That said, if you recommend using SSL (whcih I agree with, so don't misunderstand that), then what is the point of hashing it on the client side since all you are doing is:
2) Trusthing the client to do the hash (HTTP Post manipulation will bypass that and you'll end up with a non-hashed version of the password anyway).
3) won't stop a key logger.
It seems like you are wasting cycles if you really de enforce the SSL Transmission.
8 characters being a 'secure' password?
Surely you're kidding... I attempt to get the highest I can when it comes to websites. KeePass and TrueCrypt do wonders ;]
I think all these 8 character limits are probably because the website is just a frontend to some ancient big iron financial backend.
"I find it strange that the first report comes from Microsoft."
It comes from Microsoft Research, whose results only occasionally make it into Microsoft products.
Anyway, back to the primary point of hashing before it goes upstream. This is a security precaution in case the person gets caught by someone using an SSL stripper as far as i know there's no real way to prevent everyone from knowing that the site isn't supposed to be encrypted(if we're talking about a new user here) so thus their password is just out in the wild. The more watchful user though, should notice that the site is not encrypted and doesn't look normal.
The browser itself doesn't just say "oh look this site's not encrypted but normally is." it'll just serve up the site as it as if nothing's wrong. One of the other things is to prevent the widespread damage in the event of a full system wide database breach/server breach. I've taken all possible steps to prevent such a thing thus far but it is still possible that it could happen. And if i do just do bcrypt then that leaves me them with being able to do how many ever hashes per second that they can with the time value set. Currently i have it set at 150 so this means they can go in apply the nonce/salt to it all and get going at it at 150/s by hashing it before it's sent they have to hash it then hash it again. Since it's all about slowing the person down.
Also on bcrypt normally in php it just prepends the salt to the hashed string in the database which is good i guess since you can use any random salt you want for them each time. But it's also bad since if there is only a database breach they now have the salt. If you keep said salt out of there, and on the code files it's now in a completely different place.
Everything that i have done is all about trying to slow down the potential attackers as much as possible when they come to my site but also not making it take noticeably longer for the good user. I have the IDS, escape all of the sql queries, also use sprint f with literal string/integer for values. Use a bit of .htacces to block 90% of the proxies only allow proper/known http headers to access it, limit the amount of times that someone can try to get with an account per hour. Also on ssh i have the port set to a nonstandard one, limit it to 3 tries per hour, and also have made it only accept connections from my ip-address with one connection at a time.
I know that there is such things as ip spoofing but it should help mediate the attacks.
I also live with the fact that no matter what i do to help keep my databases safe, sometime somewhere if someone wants to get into them, they will. All i do is try to reduce the probability of someone getting in, make it as hard as possible for them with layers of security, and if they do get in, i try to mitigate the damage before hand.
Totally amazing if you really do all that stuff. All warranted but it's a lot more than 99.9% would do.
Though not sure I agree with this.
>I know that there is such things as ip spoofing but it should help mediate the attacks.
sure you can set any source ip you want but nothing is going to come back to you.. unless you own all the infrastructure in the middle as well.
Am I missing something big... because you know this world is full of exploits that once apon a time i would have said were impossible.
password() and md5() being SQL functions, this would mean including the cleartext password in the SQL statement. If your server logs statements, you are logging passwords in cleartext. Better to encrypt the password in the application before storing.
Schneier.com is a personal website. Opinions expressed are not necessarily those of Co3 Systems, Inc.