There are a lot of systems that meet those requirements.

> - sshd compiled with musl

So alpine as well as a bunch of other embedded Linuxes. Huge install base.

> - not using PAM

What you get if you try to have a minimal setup. Default sshd config if you install from source.

> - password logins enabled

The default. Super common.

> - at least one account with '*' as the password "hash"

Almost every Linux system.

> - at least one OTHER account with a valid password

Almost every Linux system.

So sure, there are caveats, but they cover a massive install base.

This is a super bad bug

Alpine Linux (which uses musl) images are popularly deployed by those who use docker, and apparently the issue is there on the default install.

Not sure whether the PHP vulnerability actually had been exploited, but I'm pretty sure there are literally thousands of machines out there affected by this ssh issue right now as we speak.

This isn't going to end well.

---

Addendum (which may or may not be related to the above) --

Everyone has heard of the tale of the boy who cried wolf. The main moral of the story is not to troll your fellow townsfolks so they don't distrust you. But everyone with a sane mind gets that.

The (IMHO) under-appreciated lesson is actually: don't be those villagers. Believe stories you are given, even if you might look like a fool for believing in them. Don't assume what happens is the same as what happened in the past. Look at the facts yourself. Trust and verify.

I think the conclusion that can be drawn from this is that the behavior of crypt() is howling-at-the-moon insane, and it should no longer be considered for any system built in this century, at least not without extensive wrappers that correct and normalize all these edge cases.

This is a vastly clearer explanation than the linked page. Thank you!

PSA: you can typically declare different types of string so that your type system objects to mixing data of different types even though deep down we know they're both strings.

Even using "no_password" and "error_while_hashing" would have prevented it. The problem was picking a generic, meaningless signal string.

This is a side-effect of Unix-y "it's all plain text" design that leans extremely heavily upon in-band signalling (among other design decisions).

If /etc/shadow was a JSON (or XML, even, although that would make me feel gross), this bug would have been much easier to see, because a field that reads '"hash": ""' sticks out a lot* more than a custom microformat that you don't understand.

With data structures, there's never "plain text" - it's all structured. The question is whether you want to use existing formats and tools, or create a new one for every tool and literally make footguns like this.

  NO_PROXY=1 curl -qs --data @./my\ message.txt -- \
  "https://user:password@example.com/user/${userid}/my%20message.txt"

I think everyone is over fixated on AI finding complex bugs beyond a typical person's understanding either to locate or exploit.

I think for any one of those bugs there might simply be half a dozen footguns or bugs that once it's spelled out you wonder how anyone let it slide.

I think that's the case where LLMs might indentify most.

It's challenging though as the current AI audits produce so many false positives it's hard to know when it's being pedantic or when it's genuinely caught a human being a bone head

In many ways each individual decision makes some sense; wanting to stop loads of programs being affected by a (remotely triggerable?) crash wanting to avoid time based attacks wanting to be able to set accounts to not have a password

Each one relies on a perfectly valid argument - that * can't match because we all know it'll be rejected by the other thing.

    struct Input<'a>(&'a [u8]);
    

    impl<'a> Input<'a> {
      fn new(input: &'a [u8]) -> Result<Self, Error> {
        if input.len() >= 256 {
           return Err(Errro...);
        }
        
        Ok(Self(input))
      }
    }

You must be new here. While I applauded the idea of musl when it first came out (coincidentally, glibc wasn't having the best time in its development then), the developer's arguments why <some behavior drastically different from every libc under the sun> was akchyually correct, which was why you had to fix a program that hadn't been broken for 30 years and now suddenly was, made me put him under the "clowns" category and never look at musl seriously again.

When later it turned out that musl's procrustian stack size made Python not really runnable under it, it cemented my resolve to not go near musl.

And now we have this.

A little?!? I’m sorry, but musl is a joke, this is yet another proof of that fact, and I have no idea why anyone uses it in production, they have only themselves to blame after the many warning signs.

For security it's better, but in many cases if you're not writing security-oriented software it can be worse from a user perspective. I remember the first job I had over 20 years ago was working on VoD streaming and one of the "tricks" to improve stability was mmap'ing the 0 page so that NULL dereferences wouldn't immediately segfault. Ugly as hell but when you were working with all sorts of more or less well coded media codecs and security wasn't a concern at all (worst case scenario the user would stream movies without paying) it made some amount of sense.

I suspect that Microsoft has the same history: the entire Windows platform was built with single-user PCs in mind, mostly only dealing with trusted input. The users were also expected to be tech-illiterate for the most part and unable to troubleshoot deep technical issues.

More recently I'm still working on video transmission, but on the production side this time. I also originally adopted the mentality that I should crash early if some unexpected situation was encountered, but I ended up walking that back a lot, because having a critical piece of live media production reboot mid FIFA worldcup final is generally Not Great. Degraded functionality is usually preferable. I still crash early in dev builds, but the production code just logs an error and continues if at all possible.

That said, I definitely agree with you for something like password handling. In this case, I do think it's warranted to crash the program without survivors if something is amiss.

> Also how many times do we have to learn that letting badly programmed software die when unexpected things happen is the better option because it means people actually notice that there's a problem.

100%

> Microsoft has historically been one of the leaders on the wrong side of this one

Objective-C ahem nil messaging...

> Also how many times do we have to learn that letting badly programmed software die when unexpected things happen is the better option

Sadly, we will never learn this because failing fast runs counter to intuition, and a class of person will never override it.

[flagged]

> Just don't use `*`

Well but, alas, a non-zero number of user probably do use it, though and "upgrade X" is a clearer and better instruction to communicate than "change this setting where maybe you don't even know what it means and it might break your workflow, but saves you an upgrade".

It's because alpine has updated all openssh-server packages already. Even from EOL alpine versions.

Nope, wasn’t able to reproduced it in my firecracker VM, which isn’t publicly accessible anyway.

As far as I understand, it wont work on default setups. You need musl-c, no pam and openssh.

Did you try one incorrect standard password fail first?

EDIT: works after downgrading OpenSSH, turns out even EOL Alpine version got updated packages!

Also didn't have much luck on Alpine 3.20.10 with OpenSSH_9.7p1 which I've freshly installed from an official ISO.

Last two entries in /etc/shadow:

    test:*:20762:0:99999:7:::
    q3k:$6$snip.GgDQNTrqqX.A0:20762:0:99999:7:::

SSHing into test@ with one 'wrong' password then one 'long' password doesn't work. I have UsePAM no and PasswordAuthentication yes in my sshd_config.

I can't reproduce it either. I used alpine:3.24.1 and openssh is openssh-server-10.3_p1-r2

I tried adding PermitRootLogin yes and PasswordAuthentication yes. Tried adding another user with a valid password.

Verified shadow entry has a *. Can't login as root or a test user. Tried another incorrect password first as well.

edit: can reproduce after `git revert 384e25e6bd1f2e7b30095900e48b570d46953510` in aports and rebuilding openssh without the fix. It also accepts the password on the 2nd+ attempt, NOT the first.

No, didn't work for me open OpenWRT x64 running in a container but that is probably 'cause I'm using dropbear instead of openssh-server. That's what OpenWRT uses by default so the impact there will be minimal assuming that most installations don't change to openssh-server - it isn't installed by default.

[dead]

[dead]

This looks entirely vibe-coded to me. No one is going to put so much effort for a vuln.

Yes, correct for OpenWrt. OpenWrt defaults to dropbear for both server and client. However, more advanced users may optionally choose to replace it with OpenSSH, or install OpenSSH side-by-side, to gain additional features of OpenSSH.

  > root@OpenWrt:~# cat /etc/openwrt_release | grep DISTRIB_RELEASE
  > DISTRIB_RELEASE='25.12.6'

  > root@OpenWrt:~# which ssh
  > /usr/bin/ssh
  > root@OpenWrt:~# ssh -V
  > Dropbear v2025.89
  > root@OpenWrt:~# ldd /usr/bin/ssh
  >  /lib/ld-musl-aarch64.so.1 (0x7f8e120000)
  >  libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x7f8e0b8000)
  >  libc.so => /lib/ld-musl-aarch64.so.1 (0x7f8e120000)

  > root@OpenWrt:~# which dropbear
  > /usr/sbin/dropbear
  > root@OpenWrt:~# dropbear -V
  > Dropbear v2025.89
  > root@OpenWrt:~# ldd /usr/sbin/dropbear
  >  /lib/ld-musl-aarch64.so.1 (0x7fbb900000)
  >  libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x7fbb898000)
  >  libc.so => /lib/ld-musl-aarch64.so.1 (0x7fbb900000)

  > root@OpenWrt:~# apk search openssh
  > openssh-client-10.6_p1-r2
  > openssh-client-utils-10.6_p1-r2
  > openssh-keygen-10.6_p1-r2
  > openssh-moduli-10.6_p1-r2
  > openssh-server-10.6_p1-r2
  > openssh-server-pam-10.6_p1-r2
  > openssh-sftp-avahi-service-10.6_p1-r2
  > openssh-sftp-client-10.6_p1-r2
  > openssh-sftp-server-10.6_p1-r2
  > openssh-sk-helper-10.6_p1-r2

Dropbear added the 100 character password limit in 2019.77, so earlier versions might be susceptible.

[delayed]

Maybe that's on you though. I enjoyed it, though the discrepancy between the `*sshole` name and the `sshhole.com` domain made me twitch. How is anyone[0] obsessive enough to find this and write it up and yet not obsessive enough to make the amount of `h` consistent?

[0]: Be they human or LLM. Or an entity with any proportion of the two. I do not judge nor discriminate.

I thought it was a great write up. It did feel a bit AI-coded, but whether it actually was or not, it's still a good bit of work imo.

LLM output fatigue?

I do actually. My guess is that this was not just one-prompted, but someone took the time to refine it based on their tastes and expectations until it matched something they would actually like.

However, if someone already developed a strong AI-slop-tingle it will still be triggered by some telltale signs and let their subconsciousness know that they can't possible enjoy this.

What is dead can never die. All blueteamers are ironborn these days.

Never attribute to malice that which yada yada yada

And like who even needed to plant back doors? If you're the NSA you already had an agent swarm in 1993, but it was made out of dudes from Berkeley and New Haven instead of racks in Prince William County and 印西 (and they were literal secret agents I guess, pretty metal). You can get back doors just by foraging, no need for agriculture.

You have considerably underestimated both how fallible humans are, and how high their tolerance is for risks which aren't immediately visible.

This isn't even just the software industry, humans are like this about everything. Imagine explaining to something rational that (a) humans didn't realise that they won't be able to competently operate a motor vehicle while watching TokTok dances, and (b) when we finally made that illegal because it's such an obviously bad idea they would keep doing it anyway.

This is why those "What if it builds SkyNet?" conspiracies were so stupid. Why on Earth would you build SkyNet when you can just invent Weeaboo and the idiot humans will die playing ?

actually this looks very underhanded indeed, what are the odds they pick the same sentinel character?!

Be careful, saying this gets you downvoted like https://news.ycombinator.com/item?id=50364646

Panagram is just guessing.

Such solutions always felt like swapping one risk for another. With good separation (running the VPN on a separate host from the SSH login, otherwise a bug in the VPN solution still might result in a shell on the box), at best you'd need two vulnerabilities before someone gets in, but it's not fundamentally different unless you find a way simpler protocol where you can be more certain that it's secure than ssh is (for all its scrutiny and track record of no pre-auth bugs for decades)

Putting services behind ssh forwarding was the way to secure something you're worried about rather than the other way around, but Wireguard actually makes a good argument nowadays. Iirc they did proofs for the cryptography and it has good design in terms of attack surface, and it's old/mature enough now that I'll say it's probably better than exposing SSH. On the other hand, when this thing running in your kernel gets compromised, you're slightly worse off than with someone logging into a passwordless service account as in the current bug

What is out there that uses musl without PAM and hosts an OpenSSH SSHd with PasswordAuthentication?

Possibly public SFTP servers. I have allowed null passwords in the past for anonymous public SFTP and I run Alpine Linux exclusively for my servers both VM's and physical. The shadow package (PAM) did not get installed by default. I installed it but I had to specify it in the installation so there are likely people without it. There is probably a way to find them on shodan.io.

I will activate one for people to play around with for today:

    sftp hn@ping.ohftp.org

There's no firewall and no password. Hack away HN.

    PasswordAuthentication yes
    PermitEmptyPasswords yes

cd /pub

get so*

A large number of Docker containers?

Edit: Sure, a small minority presumably run sshd under these conditions. But a small minority times the number of Alpine containers running in the world gets to a large absolute number.

Edit 2: You really don't need to look very hard to find public container examples at million-download scale.

Agent sandboxes today probably

OpenSSH considers this to be a vulnerability in musl and not OpenSSH, but shipped a fix anyways since that helps mitigate impact... Whether you agree with that or not, it's up to you :)

Probably because mitre take days to weeks to give out CVE numbers (if ever). Just get the fix out there.

> Its not asking for trouble, it is there will be trouble, not if but when.

Did any of those bots brute force a password? Because a 20 character random password is comparable in brute force complexity to an SSH key. The experiment you ran was just as valid for SSH keys on that port, and had no bearing on password vs key.

How is password auth weaker than key auth? Bugs in auth code can happen in both pathways. And apart from stupidly short passwords both should be equally impossible to brute-force.

In other words, conceptually, at the end if the day, isn’t a key just another name for a password, really?

I run SSH with password authentication on but the port binds to a ULA address. It’s fine. No need to be so absolutist.

This particular bug works even with password authentication turned off.

I mean there are 65k possible ports so it's the equivalent of guessing an 16 bit password, i.e. two (extended) ascii characters. It's not very surprising that 'multiple bots' found it: a single `nmap` command will give you the open ports in a few seconds.

100% and with local tool calls many AI systems will do ssh "for you" and end run around any security theatre that you may have going on with LLMs.

I think we should get used to eternal "break the world Thursday", just like our predecessors had to come to terms with Eternal September [0].

[0] https://en.wikipedia.org/wiki/Eternal_September

but alpine famously does not use systemd but openrc?

I use alpine-based containers whenever I can just to keep the world safer from SystemD rotting Linux.

Never enabled sshd there... except on my PostmarketOS PineBook Pro... oh shit!

Thanks for mentioning that before I tried to scroll down. Forewarned, I was able to click on the "2026" link once the left-handed "scrollbar" (not the right word but I can't figure out what the right word should be) appeared, which let me jump past most of the verbiage and get to the "are you affected?" section just a couple pages later.

It's people like you I fire with glee

A* name

> So, nobody fuzz tested sshd before?!?!

OpenBSD probably did (which doesn't use musl), RedHat probably did (or some other distro using glibc), but they all don't use musl.

Yeah, I am mystified as to how this could have happened.

"unfortunately, a large portion of programs which call crypt are not prepared for its failure and do not check that the return value is non-null before using it. thus, always "succeeding" but giving an unmatchable hash is reportedly a better behavior than failing on error." [1]

[1] https://git.musl-libc.org/cgit/musl/commit/?id=b3c4cc121f70f...

> did musl really feel the need to accommodate bad implementations like this

I don't know about this specific case but in general? Yes, totally. No one is going to fix all existing software that depends on libs behaving exactly as glibc, you have to accept some trade-offs. Either your new implementation stops being perfectly simple and clean, or no one uses it.

There's a decent-size "any AI usage is theft" contingent that reacts dramatically to any usage of AI whatsoever.

You've got a great point. But let's also acknowledge that there's a pretty stark contrast between these:

- I found a way to get AI to help me while I made this

- I asked AI to make this

The people you're objecting to are complaining about the latter, this is an example of the former.

Not sure why but a lot of people seem to be missing that you don't have to look at AI as a binary "you're either with us or you're with the terrorists."

Just do HITL (human in the loop). Use an LLM to help you author, but steer the AI in whatever direction you want. It's like having a slightly delusional copy of yourself to pair program with. If you take the time to provide an LLM with some additional skills files and the like, with you in the pilot seat, you can get solid results (it doesn't look as cool as a hands free agent setup but the quality far surpasses anything they can do solo).

> Can we please now normalize AI usage in development?

1. It is already normalized outside of a tiny, but very vocal bubble that lives mostly on Bluesky (sometimes also on Reddit/HN/Mastodon).

2. I don't think that even the most staunch AI hater would object to using AI for security research as long as all the artifacts (code, docs) that were produced based on the research are human-written. "Yes, there is a bug in my program, but I refuse to fix it, because I know that you found it using AI" would be a pretty weird position. Not sure, how would you defend it.

It's shocking that it took so long with the amount of investment and hype you'd expect this kind of bugs to be already fixed, but it still need a human to tell the agents what to do, which could also work with any fuzzing framework

*sshole, the * is the star of the show

You can use \ to escape formatting.

No, the Alpine containers don't come with sshd preinstalled. You'd have to manually install via dockerfile and leave it unconfigured.

No one runs sshd in containers, at least I hope so. What would even be the point when you have docker/kubectl commands available?

I can't understand it. All this scrolling and pictures are distracting me, can't concentrate really.

> Also, what's with this damn trend of typing in lowercase only?

Anticapitalism.

sigh, it's not a new trend. it's just yet another pop-cultural appropriation from the gay community, now mainstream thanks to ai. as trump's gay dementia progresses, pretty soon even he will be truthing about other men's asses in lowercase, just to get a rise out of them!!! however, the *sshole project's shibboleths of lowercasing and anal asterisk punctuation strongly suggest its authors may be bona fide members of the lgbtq+ community.

why do gay people love typing in lowercase?

https://thetastefultwink.substack.com/p/the-queer-lowercase

> Let’s sit and twink about the data. 24.6% of my 19,000+ respondents abandoned capitalization entirely. If you control for age, queer people were 1.5x more likely than straight people to type in lowercase. Lesbians out-lowercased every other sexuality at 28.3%, followed by bisexuals at 24.9%. Honorable mention to the lowercase enbies, at 29.2%.

New and Changing Social Evaluations of All-lowercase and Exclamation Points

https://repository.upenn.edu/server/api/core/bitstreams/edbc...

> both exclamation points and lowercase usage are associated with queer identity.