Good Essay on the History of Bad Password Policies

Stuart Schechter makes some good points on the history of bad password policies:

Morris and Thompson’s work brought much-needed data to highlight a problem that lots of people suspected was bad, but that had not been studied scientifically. Their work was a big step forward, if not for two mistakes that would impede future progress in improving passwords for decades.

First, was Morris and Thompson’s confidence that their solution, a password policy, would fix the underlying problem of weak passwords. They incorrectly assumed that if they prevented the specific categories of weakness that they had noted, that the result would be something strong. After implementing a requirement that password have multiple characters sets or more total characters, they wrote:

These improvements make it exceedingly difficult to find any individual password. The user is warned of the risks and if he cooperates, he is very safe indeed.

As should be obvious now, a user who chooses “p@ssword” to comply with policies such as those proposed by Morris and Thompson is not very safe indeed. Morris and Thompson assumed their intervention would be effective without testing its efficacy, considering its unintended consequences, or even defining a metric of success to test against. Not only did their hunch turn out to be wrong, but their second mistake prevented anyone from proving them wrong.

That second mistake was convincing sysadmins to hash passwords, so there was no way to evaluate how secure anyone’s password actually was. And it wasn’t until hackers started stealing and publishing large troves of actual passwords that we got the data: people are terrible at generating secure passwords, even with rules.

Tags: hashes, history of security, passwords

Posted on November 15, 2024 at 7:05 AM • 22 Comments

Comments

Anonymous • November 15, 2024 7:34 AM

Well, I agree in part. Policies are a problem with people being people. But they can be improved with what we know now.

And the hashed databases can indeed be tested. The criminals do just that by hashing common variations.

The proposal to use public key crypto is likely to have many unintended consequences as well (making the protected file with the passwords a target, untrusted servers or employees on the hunt for password reuse…).

Anonymous • November 15, 2024 7:35 AM

Well, I agree in part. Policies are a problem with people being people. But they can be improved with what we know now.

And the hashed databases can indeed be tested. The criminals do just that by hashing common variations.

Clive Robinson • November 15, 2024 9:17 AM

@ Bruce “Only”

The authors name sounded familiar so I “followed a trail” to,

https://dicekeys.com

Where under “leadership” I found two names well known to me, Joseph Bonneau and you.

I know this is only a “personal blog” not a Journal or other Academic Publication, but over the past decade with increasing “federalism” of “Social Media” and similar people are getting not just “Snarky” but are looking for “authors” to “Declare Interests” in things other than “Papers”.

So you might want to consider “staying ahead of the mob” and putting in a footer to mention close, business, etc associations when they arise.

Clive Robinson • November 15, 2024 9:22 AM

@ Bruce “Only”

The authors name sounded familiar so I “followed a trail” to,

https://dicekeys.com

Where under “leadership” I found two names well known to me, Joseph Bonneau and you.

So you might want to consider “staying ahead of the mob” and putting in a footer to mention close, business, etc associations when they arise.

Not Anonymouse • November 15, 2024 10:43 AM

But you can evaluate how secure everyone’s password is, even when it’s hashed… you just have to do it within the window of time before it’s stored at rest hashed. It is clear text at some point in memory, use that point.

In modern times more and more sites do it all the time: whenever a person sets or resets their password, we check it against Troy Hunt’s “Have I Been Pwned” service to see if it’s a bad password, if it’s in that database, we deny the user the ability to set their password to that bad password, and make them pick a better one first. Once this has been implemented in your code, we can nag and then finally force a one-time reset to bring everyone under this kind of policy, which is immensely better than requiring character classes. For those who never login after years and years eventually a user cleanup is in order.

Before anyone takes this to “horrors, checking an external service”… well, yes, you can download his whole database and do everything locally, if you have enough bandwidth and space and inkling to do so and keep it up to date. Or you can use his “k-anonymity model” and do it live as a service. Or you can roll your own by perusing “dark web” releases of troves of passwords yourself, duplicating his work. It’s your choice, depending on your threat model.

So I disagree that hashing is a mistake at all, it just makes you do your evaluation in a different way. But perhaps I am making your main point that the best success of implementing such evaluation (and fixing it too!) that we’ve seen so far actually depends on hacker releases of troves of data…

Joe D • November 15, 2024 10:58 AM

Couple this with a requirement to periodically change your password to really screw things up.

At a former employer, we had to change the password every 90 days. I wound up sticking a counter at the end and incrementing. password_01, password_02, etc.

Tony • November 15, 2024 1:35 PM

Hashing passwords was not a mistake.

Wasn’t Facebook recently in the news for keeping passwords in plain text, and accessible to every employee?

Hashing passwords keeps strong passwords (as chosen by password managers) safe from being leaked when some other part of security is compromised.

Cybershow • November 15, 2024 2:31 PM

To add a little to Clives remark, I feel deja-vu reading so many times of
“brilliant people, at the very top of their field, whose well-intentioned
recommendations led to decades of ignorance.” Invariably the complaint is not
actually against their ideas but the unthinking mass that followed them off the
cliff, a comic-book trope that we fall on our own swords via overconfident
statements like “make it exceedingly difficult”.

Philosopher Herbert Marcuse wrote “One Dimensional Man” as a critique not just
of technological rationality and political control of advanced industrial
civilisation, but of the psychological forces behind these sorts of mistakes.
We keep allowing narrow sets of interests to set dangerous agendas due a
tendency to invest too much in singular systems of thought. People look for
thought leaders and brilliant new paradigms rather than finding more mundane
“better ways of being”. And if you doubt that I heartily recomend Monty
Python’s “Life of Brian”.

Of course Thompson and Morris are not outdated. Thompson “On Trusting Trust” asks
some of the hardest questions that we still refuse to confront today. Alas,
Schechter falls into a similar shaped hole with his strident assertion that
Rivest, and Shamir’s asymmetric methods meant “it didn’t have to be that way”.
As if we could re-run history and find no weaknesses in public key methods that
lead to their own pitfalls.

NIST’s latest venture into common-sense is laudable, and something I wrote
about in https://cybershow.uk/blog/posts/secrets to help generate debate
amongst students around all approaches to access control and authorisation. The
questions of what is “authentic”, what is “identity” and what is “trust” remain
elusive.

Hopefully we will also gain the humility to acknowledge that not just password
composition but other supposed panaceas like “biometrics” and “zero trust” will
themselves be “not just annoying, but counterproductive.” Not because they are
especially faulted as ideas but because of the simplistic thinking that there
is a single right_way to do security. Those bandwagons are the problem and
that’s what I think Stuart Schechter is trying to articulate.

TexasDex • November 15, 2024 3:32 PM

I think the real lesson is ‘people are good at making easy to remember (and easy-to-guess) passwords despite all the rules’.

Mike D. • November 15, 2024 4:26 PM

So… why is hashing passwords a bad thing? Our Mikrotik router which keeps all the VPN’s PPTP secrets as plaintext in a handy web interface is doing the right thing?

(And yes, I know PPTP isn’t recommended; it’s just inertia at this point and I also have L2TP enabled for anyone who wants that instead.)

Clive Robinson • November 15, 2024 8:52 PM

@ Not Anonymouse, ALL,

Re : Consider the whole system security not part of the systems complexity.

With regards,

“But you can evaluate how secure everyone’s password is, even when it’s hashed… you just have to do it within the window of time before it’s stored at rest hashed. It is clear text at some point in memory, use that point.”

Unfortunately increasingly “no you can not”. Because you are considering only one part of the system (the server).

With the increasing use of password managers, passwords can be just a four digit pin or less that is “hashed” on the users device. They then use the hash or checksum of it as their “over the wire” password.

As such the hashed pin has high complexity but next to no entropy. So is actually quite insecure. Further hashing the hashed pin at the server will not in any way increase the underlying entropy thus security.

If the “attacker” has sufficient knowledge of you and the system you use for managing your passwords then they can attack the user system or use knowledge of it to find the over the wire password.

It’s one reason people need to fully understand the implications of Auguste Kerckhoffs’s Principle restated by Claude Shannon as,

“The enemy knows the system”

In the case of passwords and passphrases the week link is almost always the user and their fallible memory.

As an attacker “who knows the system” you find and exploit the point where “entropy is at a minimum”. Because this makes the resulting search space you have to work through small in comparison to that of a high complexity point.

As a system designer you would know that the human memory weakness is what you need to mitigate effectively. One way to do this is by “adding entropy”.

In past systems this is what the “salt” at the server end was for, but this is now because it is “static” considered insufficient.

More modern systems use a dynamic process. One such –which is now seen as ‘getting old’– was to use a “shared secret” in a token to encrypt a changing value like a counter or timer. This would be appended to the user entered pin and the result then hashed or encrypted and this result would get sent as the over the wire password.

Due to the predictability of a timer or counter, this is seen as insufficient. Thus in some systems the server generates a true random value that is encrypted via a user specific key and sent to the users device. The random value then gets transformed in a user specific way to form part or all of a way to modify the password / passphrase to convert it to the over the wire value.

But it’s very easy to get these systems wrong and leave an opening to an attacker in either the protocol or source of true randomness etc. Or worse not hide shared secrets securely.

John Beattie (jkb) • November 16, 2024 8:05 AM

What about password cracking tools used by the sysadmin? The salt is usually held with the hash, so that is available to the cracking tool.

Re public key cryptography, his approach to securing the private key looks a lot like one of the old escrow systems. The simplest way to break any password after that would be to get the private key. Rubber-hose decryption, blackmail, bribery. Hacking the server where the key is held.

All are single points of failure and all make my careful use of a RNG to generate a password completely pointless.

Rac • November 16, 2024 2:38 PM

This is the first I’ve heard of criticism of password hashing in general. Have I missed prior literature from Bruce where he criticized the practice?

Clive Robinson • November 17, 2024 6:46 AM

@ Rac, All,

With regards,

“This is the first I’ve heard of criticism of password hashing in general.”

The form of hashing being used is supposadly a “One Way Function”(OWF) andvis a form of encryption. The use of the OWF in theory means that once hashed the only way to find the original “plaintext” is to either intercept it before it’s hashed or by running a form of “Brut Force” or “Dictionary” search to make “Rainbow Tables” and similar,

‘https://en.wikipedia.org/wiki/Rainbow_table

Like all technology this is “agnostic to use”, that is it’s the “observers view point” that decides “good or bad” and for the “directing mind” what they are trying to achieve.

The use of passwords is an “adversarial game” with more than two players, but can be reduced for most arguments to,

1, Owner of system
2, User of system
3, Attacker of System

The aim of the Owner is to protect against both Attackers and Users.

The thing is that the two aims are in many respects “mutually exclusive”.

That is Users find passwords nothing but an inhibitory intrusion into their work flow, something our host @Bruce has talked about in a more general way in the past.

So many users would if they could make their password as simple as possible if they were allowed to do so[1].

However short passwords of low complexity are the first things an Attacker would try[2].

So the issue for the Owner is how to ensure both,

1, Users have long complex passwords.
2, Attackers can not find the password.

Thus the Owner want’s to see the User password in “plaintext” to check it is not short or of low complexity, but also stop the Attacker seeing the “plaintext” even in part[3].

This means that for the Owner they want the Plaintext to get to the system, this makes the password vulnerable in all sorts of ways even with modern encrypted communications[4].

The flip side is that “Owners need” to “Police the User” is in effect a crack an attacker can use to their advantage.

One obvious solution these days of PC’s and Smart Devices is to move the hashing of the plaintext password to before the vulnerable communications. But unless carefully thought out this means the Owner can not “Police Passwords”.

Which is why “premature hashing” is seen by some as a real issue, as they know Users will quickly fall back to the weakest of passwords.

Yes there are ways to mitigate this, but it falls to the issue nobody has really solved,

“How to exchange root of trust secrets securely without doing it in person?”

Which is where the fun really starts.

[1] Many users would like to “just press enter” if they could, or something else that is both very short and easily memorable “to them”. Thus if allowed most user selected passwords would be of at best “minimal value” from the security aspect.

[2] The problem these days is Attackers can fairly easily try passwords of a length and complexity most Users can not remember…

[3] The problem with humans is in general they don’t do Random, or Complex, and they have terrible memories. Thus there is almost always some kind of structure to User selected passwords that an Attacker will exploit. Thus knowing as little as the password length aids the Attacker.

[4] Back when terminal were connected to a system over short lengths of “serial cable” the only comms security available to most was the physical protection of the cable and the signals on it. With the advent of affordable modems and thus “remote working” this was very insecure at best. Even when encryption of the line became possible, users expected “Full Duplex” behaviour which ment each plaintext character got sent individually. Thus no matter how complex the encryption algorithm it was in effect reduced to a “simple substitution cipher”. Thus the encryption algorithm had to be used in some form of wrapper we now call a “mode” that mixed things up. Often but not always this was by “chaining” which is the equivalent of a “poormans stream cipher”. Then there is the problem of “Man In The Middle”(MITM) attacks on the encryption setup that still work very effectively today. So even with the use of much stronger encryption like those used in HTTPS TLS etc these issues still exist.

Winter • November 17, 2024 5:58 PM

The conclusion of 5 decades of experience with passwords can be summarized as:

Any passwords the majority of users can memorize and type blindfolded can be cracked on modern equipment by brute force
The number of different passwords needed by the average users forces the user to record them in writing or in a password manager
Lost and forgotten passwords are inevitable and require special password reset procedures that subvert password security
High value passwords are stolen by keylogger malware

The obvious conclusion is that passwords have failed as a secure authentication solution and should be retired.

TFA, MFA, biometrics, and Passkeys are already pushed to replace passwords for this very reason.

I can see a 3FA system to replace passwords:

Something you know (simple password, can be spoken)
Something you own (passkey, mobile)
Proof of identity (fingerprint, face, iris)

Just as a Gedanken Experiment, such a system could be implemented in a single phone that collects all three securely. Optionaly, they could be entered on three different devices. It could also be implemented as two out of three with single use emergency reset codes for a lost factor.

But passwords themselves have outlived their usefulness.

Jelo 117 • November 17, 2024 11:08 PM

I tried 111 … 111 (eighteen 1s) and the password strength checker showed solid green and said very strong password. So I use that, but I added an additional six 1s, for future-proofing.

Vikarti Anatra • November 18, 2024 4:32 AM

Possible solution to this mess is:
– allow user not to invent passwords at all, use some kind of device user arleady have – hardware token or any smartphone (without stupid ‘security’ checks). No additional passwords,just working SSO. If system can’t support this – allow ‘app passwords’ generated via ‘regular’ system.
– make it possible for user to use their password just once per session (example in work: connect hardware token, unlock it(via password), enter 2nd password to connect to ‘secure’ VM to access internal e-mail and some CRM tools, enter 3rd password to acces regular wiki. Also use 3rd password to access private maven repository (access is via VPN but passsword is still necessary because ‘security’, devops-recommend way – just store this 3rd password in plaintext on local machine).
so 3 different passwords. VM is also timeouts and user needs to re-enter password. Passwords also needs to be changed every 3 months.
Guess how really complex was passwords are while all rules are satisfied?

Clive Robinson • November 18, 2024 1:55 PM

@ Winter, ALL,

With regards,

“The obvious conclusion is that passwords have failed as a secure authentication solution and should be retired.”

From a security and human user management issue, this has been true for this century so far at least. It’s also why I keep saying they should be replaced and have done so for more than the past couple of decades.

The question is,

“Why have we not replaced them yet?”

The answer is multifaceted and changes with time.

Firstly is “storage cost” when passwords first came about hard drive storage was up in the $1/character or greater range. Whilst the per character storage cost has dropped to the point it is very small, there are other costs like access speed and availability and the fact we now store “salts and extended hashes” that take us up to 1KByte per user or even higher with other types of non-flat-file DB etc system.

Whilst perhaps not to much of an issue with SoHo or MeSE organisations, for modern cloud and social media sites that have upwards of tens of millions of users… Worse for them they also tend to store profile information per user and that can be hundreds of thousands of bytes per user. So the cost of Security authentication (AuthN) and authorization (AuthZ) is still quite high for some even now.

Secondly text based pass words/phrases are or should still be “Simple Plain Text”, which is a “lowest common denominator” that should work every where… However as some have found trying to use extended character sets is problematic, especially when “internationalisation” has to be accounted for. In effect it’s one of those “serialisation edge cases” with overtones of “gallons into pint pots” if you try to “get clever” with modern languages and libraries when you’ve not yet learned to “drain the swamp” (we get the same problem with nearly all things involving encryption algorithms).

Thirdly, but if we don’t use “plain ASCII text” pass word/phrases what else are we going to use. Put simply,

“We’ve been seeking but not finding.”

Just about every alternative has issues with issues like standards and protocols giving rise to implementation issues.

Fourthly is the “route of trust” issue. To get a sufficient increase in end to end security to prevent attackers being successful needs a two part system of a “high entropy” “shared secret” and of necessity a “low entropy” “something the user knows” to gate it’s use.

Fifthly is that the shared secret needs to be protected in some way. A software vault is defeated by an attacker getting access to the stored “shared secret” in any one of hundreds of different ways. One of which can be as simple as “key logging” the user input.

Sixthly the use of a hardware token is often security wise less secure than a software vault. They are also expensive, not just for the hardware but all the “user issues” arising.

Which brings us to what has for decades been the killer,

“The cost of user management.”

If you want real security then the cost of this is in many cases quite extraordinary.

In part the drive to “Single Sign On” systems was to cut this cost. But in reality it was not very successful,

“Because users are human and statistically bad news.”

There are other issues I could list, but untill fairly recently passwords were the “most cost effective” at low security.

But some have realised there is a catch in “legislative changes”. The way some “child protection” legislation is written, the service provider carries liability for children getting access to what the legislation says the should not. As with a lot of ICT legislation the scope is not just overly broad but close to impossibly broad for service providers, especially with other “Data Protection Legislation”.

In effect passwords and passphrases are now being legislated against by child protection and other legislation.

Robin • November 20, 2024 8:09 AM

@Winter, all

“I can see a 3FA system to replace passwords:

Something you know (simple password, can be spoken)
Something you own (passkey, mobile)
Proof of identity (fingerprint, face, iris)”

I appreciate that this is a very small gripe in the grand scheme, but there are occasions when one user, quite legitimately, and with the approval of the owner needs to access data/websites/accounts on behalf of others. This happened when I was exercising Power of Attorney for my mother (Alzheimers) and it happens now when my partner is away in places where there is no, or very poor quality, phone and internet connections or when she is, as sometimes happens, unavailable due to hospitalisation.

As the security measures have become more rigorous it has become more and more difficult to “help out”. I foresee a time when it will be impossible, thanks to rules being devised and implemented by people (I suppose much younger and healthier than me) who don’t have the same problems. Be nice to have some kind of authorised second user on some sites.

OTOH I also see how that kind of access could be a gift for scoundrels.

Winter • November 21, 2024 2:43 AM

@Robin

I appreciate that this is a very small gripe in the grand scheme, but there are occasions when one user, quite legitimately, and with the approval of the owner needs to access data/websites/accounts on behalf of others.

Banks have had this problem for as long as bank accounts existed. Recently I saw it with the introduction of TFA. Their solution is to grant access to additional individuals who each identify themselves individually.

It should be a technologically relatively simple addition to any type of account. The authorization is not simple, I assume.

Bogdanovic • December 15, 2024 3:03 PM

So the first part of the article makes total sense to me…

But when they come to hashing, asymetric keys and stuff.

First of all: There have been instances of passwords being encrypted instead of hashed. I remember Adobe. But instead of making anything more secure they basically just leaked all the passwords. So we have for sure datapoints, that it is a bad idea. I don’t find many data points to other ends.

The Idea of “storing the secret key safely somewhere” reminds me a lot of those “key escrow” or “government backdoor” ideas…

And who decides who’s getting access to those keys? who makes sure they are not leaked? Nah… it’s just a really bad idea!

Clive Robinson • December 15, 2024 7:44 PM

@ Bogdanovic, All,

“The Idea of “storing the secret key safely somewhere” reminds me a lot of those “key escrow” or “government backdoor” ideas…”

The issue of encryption “keys” is one that makes life interesting and falls into the philosophical group of things where,

“The only three numbers that make sense are Zero, One, and Infinity.”

That is you can have crypto with,

1, Zero Keys (Hash Algorithms).
2, One Key (Symmetric Algorithms).
3, Multiple Keys (Asymmetric, N of M, etc Algorithms).

In some respects “Zero Key” or Hash systems are not just harder to design, they are actually weaker on a “gate count” basis.

Or software for them is more complex, fragile, and less efficient or non optimal.

Thus unless the Hash comes for free –and it never does–, designers have wanted to go with symmetric encryption first or asymmetric encryption second.

The Morris Unix Password system for instance was based around using a modified symmetric encryption algorithm in a “mode” where you did not need a “secret” key.

You can do this because you are not looking for a Hash with all it’s properties but in effect just a limited “One Way Function”(OWF) any crypto function can be bent into with a little bit of hammering.

In fact you can hammer any keyed algorithm out of shape into another algorithm using lesser numbers of keys when you only want a limited OWF functionality for certain uses like the storing of partial or full passwords.

Schneier on Security

Good Essay on the History of Bad Password Policies

Comments

Leave a comment Cancel reply