Why Are We So Stupid About Allowing Overused Passwords?Organizations Should Blacklist Commonly Used Passwords - But Not All of Them
Password security guidance: Do block users from picking commonly used passwords. But when creating password blacklists, don't block users from picking any password that's ever been seen in a data breach.
See Also: The Future of Digital Identity
That's the guidance offered by security experts for how organizations can use password blacklists to increase security, without making password-picking a nightmare for users.
"A block list of 320-something million passwords would IMHO be madness in almost any situation."
The question of how to build and employ a better password blacklist is one I've been hearing in recent weeks, ever since Have I Been Pwned creator Troy Hunt released a collection of about 320 million unique passwords in the form of SHA1 hashes as part of his new Pwned Password service. All of the published passwords have been seen in prior data breaches.
The impetus for the password release is to help battle credential stuffing, referring to hackers cycling through lists of known usernames and passwords to try and find combinations that unlock someone's account, Hunt says, adding that blacklisting known passwords squares with updated security guidance from the U.S. National Institute of Standards and Technology.
Beyond blocking users from picking passwords that contain dictionary words, sequences of numbers or digits, or context-specific words - such as the name of the service they're using - NIST now recommends also taking into account passwords that have been seen in previous data breaches.
Deciding which of these passwords to block outright, however, "depends on a heap of factors, including the demographics of your audience, the value of the asset being protected and the other controls the site has in place," Hunt tells me. "Strictly speaking though, if someone is creating a password that's in that 320M list, it's a bad one. Someone else - possibly them - has used it before and the simple mechanics of creating a good password - random and unique - means that should be impossible."
Don't, however, try to block outright every one of millions of passwords that have been published by Hunt, advises Norway-based password security expert Per Thorsheim, founder of the annual PasswordsCon conference.
"A block list of 320-something million passwords would IMHO be madness in almost any situation," Thorsheim tells me.
Crack Me If You Can
Hunt's Pwned Password list has also been reviewed by the password-cracking wunderkinds at CynoSure Prime - previous winners of the annual "Crack Me If You Can" competition at the Def Con conference in Las Vegas - together with security researchers @m33x and Alaska-based Royce Williams, aka @tychotithonus.
On a fun note, the researchers decided to crack the password hashes released by Hunt. "The actual cracking took us several days, using a variety of tools," CynoSure Prime tells me, noting that their password-cracking arsenal included such tools as MDXfind and Hashcat.
CynoSure Prime previously cracked hashed passwords dumped by whoever attacked pro-infidelity online dating service Ashley Madison in 2015, using MDXfind, together with a custom tool they wrote called bcval (see Researchers Crack 11 Million Ashley Madison Passwords).
Lowercase Passwords Abound
Turning their sights to the Pwned Passwords list, CynoSure Prime and friends found some interesting trends after they cracked and analyzed all of the passwords. For starters, nearly one in five passwords in Pwned Passwords were less than eight characters. In general, researchers recommend that passwords be at least eight characters in length and note that anything longer won't necessarily be tougher to guess.
The researchers also found that 26 percent of the Pwned Passwords used only lowercase characters, while only 2 percent of the passwords in the dump mixed uppercase and lowercase numbers, as well as numbers and symbols.
Goal: Tough to Guess Passwords
Mixing things up can help create stronger passwords, but there's no hard and fast rule for the best way to build a password that's resistant to guessing, according to "An Administrator's Guide to Internet Password Research," published in 2014 by Cormac Herley and Dinei Florencio from Microsoft Research, together with Paul C. van Oorschot at Carleton University.
For example, a 26-character password composed entirely of lowercase characters that contains no dictionary words or signatures will be much harder to guess than an eight-character password that is "12345678." But if a user reuses the same 26-character password across sites for accounts tied to the same username - such as their email address - and one of those sites gets breached, then their security will have been compromised.
That's why security experts recommend never reusing a password. Royce Williams, for example, says that for any password that does not need to be memorized, "use a password manager to generate random passwords" as well as store them (see Why Are We So Stupid About Passwords? Yahoo Edition).
How Big a Blacklist?
Per the NIST guidelines, organizations should prevent users from picking commonly used passwords. But how many passwords should an organization include on its block list or blacklist?
"The best reference in existence regarding the use of blacklists," Thorsheim tells me, is "Pushing on string: the 'don't care' region of password strength" - a 2016 study published by password security researchers Dinei Florencio, Cormac Herley and Paul C. van Oorschot.
"[The] paper talks about online password guessing, and findings suggest a blacklist of the top 1,000 to 10,000. More than that won't add much extra security, but may severely impact usability and annoy people with repeated messages of 'password not allowed,'" Thorsheim says.
But he notes that these are only rough guidelines and will not apply to every situation. "Let me be clear: I find it very hard to set a 'standard' for this, as it must be evaluated case by case, service by service, country by country, etc.," he tells me.
For example, sites that cater to different spoken languages may need blacklists of different lengths - and compositions - than other sites.
"The top 10,000 passwords in the world might be heavily influenced by English names and words," he says. For example, "what works nice for any English service mainly targeted at English-speaking users will probably have much less of an effect in a Norwegian-only environment."
In other words, while password blacklists can help sites ensure that users pick more secure passwords, one size does not fit all. In addition, Thorsheim says a site will ideally pick an approach and then continue to test and refine it, to optimize security and usability.
Ideal: Middle Ground
For example, rather than blocking outright all 320 million passwords contained in the Pwned Passwords list, Hunt notes that "there's also middle ground available here: you could always block the top X passwords, then warn on anything else that's appeared before and give the user the option to change it."
Such an approach balances security by blocking a known-bad set of commonly selected options, which will raise the security baseline, while also empowering users who can be bothered to pick even better passwords.
"I like that approach," Hunt says.
August 30: Post updated with comment from CynoSure Prime and to clarify the tools they used to crack the Pwned Passwords hashes released by Troy Hunt, as well as the Ashley Madison dump.
August 31: Post updated to detail the practice known as credential stuffing.