Rendered at 18:13:09 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
mpyne 1 days ago [-]
OK, I think I see what’s going on with this, in this configuration.
The goal from an attacker’s perspective is to get this to be true: `$ret = crypt($pw, $hash); $ret == $hash` when called by OpenSSH’s sshd. This is how sshd validates that the user logging in knew the account password.
The `$hash` is the “settings” to the crypt call and will normally look like “$6$--gibberish” for users with passwords set, but could look like “*” or “!” for users not intended to be able to login.
The bug is from the intersection of all of these things:
1. musl can return “*” for errors, rather than leave the return value NULL. Other libc have adopted something like this too, but seem to use a different result “*0”.
2. musl can also return “+” for errors, but only does so if $hash starts with “*”. This is intended to eliminate the possibility of an error causing `$ret == $hash` to be true, however it assumes that applications calling `crypt()` will set $pw and $hash to values corresponding to the `$ret == $hash` comparison to be made immediately afterwards.
3. OpenSSH can be convinced to pass the hash of a different user if the hash is “*” or “!” to avoid timing attacks that can enumerate the user list, so it will set $pw and $hash to correspond to different users. $hash will be a real “$6$--gibberish” style hash in this case, and OpenSSH expects `crypt()` to either return NULL or return a hash string. But now there’s two hashes OpenSSH uses. It does: `$ret = crypt($pw, $hash2); $ret == $hash1`;
4. This should normally not be a problem because OpenSSH will pass in valid $pw and $hash2 that should not lead to errors in hashing, so the $ret that musl generates should be a real hash that cannot match the “*” or “!” $hash1 for the login-disabled user. However as a DDoS mitigation, musl will return error if the password is too long when using SHA-2-based hashes (the default, it seems).
5. This error will result in a non-NULL result of “*” or “+”. Since OpenSSH passed in a real hash ($hash2), musl will return “*”, which compares equal to the actual user hash $hash1 for the disabled-login user.
The OpenSSL adjustment ensures a minimum hash length for the user before logging them in, even if crypt() succeeds somehow.
chuckadams 1 days ago [-]
I think the conclusion that can be drawn from this is that the behavior of crypt() is howling-at-the-moon insane, and it should no longer be considered for any system built in this century, at least not without extensive wrappers that correct and normalize all these edge cases.
markasoftware 1 days ago [-]
Conclusion is type safety is important
GuB-42 22 hours ago [-]
In this particular case, it is type safe. NULL, by the C standard does not refer to an object. It means that every pointer in C is effectively an option type, which mean that a function that returns a pointer can return NULL for "no result", which, in the case of crypt() is perfectly adequate.
It is the opposite that is unsafe, that is, when the returned value can't be NULL, but it stills returns a pointer. C++ has references in an attempt to fix this problem.
Because C tends to prioritize performance over safety, a NULL pointer dereference is considered undefined behavior and the compiler can invoke nasal demons if it makes the code go faster, but with the right options, you can have it checked and invoke what in other languages would call a panic handler, but on UNIX is the SIGSEGV signal. You can then attach a debugger or read the core file and see what went wrong.
Note: I am not talking about "char *" for strings, that one is unsafe, but NULL is not. It is just a special case of an option type, or a result type, or an error union. Modern languages just extended this idea to objects other than pointers, and with more than one type of error.
annex-winged-cr 7 hours ago [-]
The implicitness is exactly the problem. The workaround that caused this vulnerability was done because people assumed that crypt() would never return null.
You don't even need a fancy result type. A simple Option<String> would've forced downstream users to consider the failure case, avoiding this situation in the first place.
f33d5173 51 minutes ago [-]
.unwrap()
Now what?
shakow 29 minutes ago [-]
First you shouldn't have unwraps in prod code, and you can verify that in your CI/build process/whatever. Second, now you have a crash of the SSH server, which is arguably much better than some random logging in as root.
alerighi 7 hours ago [-]
Exactly, whoever program in C knows that every function that returns a pointer may return NULL (even if not documented) and thus every pointer must be checked.
However a better API (to me) that I would have written was:
That way btw the function doesn't need to allocate memory inside, which is (being a mostly embedded programmer) something I don't like at all. Plus returning an error enum makes it clear instead of having to look at errno.
But POSIX API is not something well designed, it dates back to the '80 so... probably today would be written completely differently
miki123211 7 hours ago [-]
In theory, having newtons and pounds force be represented as `float` is type safe. In practice, it destroyed a multi-million-dollar NASA spacecraft because of human error.
If NASA were to use the equivalent of `pub struct Newtons(f32); pub struct PoundsForce(f32);`, where the types Newton and PoundsForce were not interchangeable, this would never have happened[1].
[1] I'm aware that that story is a bit more complicated than that, that CSV files were involved (which are, again, not type safe), and that, in that particular scenario, more would have to be done to preserve type safety across formats, vendors and codebases, but my point still stands.
> The newtype is opaque, so the only way to observe it is to convert it back to a [f32].
itishappy 15 hours ago [-]
> It is just a special case of an option type, or a result type, or an error union.
These are not the same, and your next sentence suggests why. Option has but a single error type, but result types and error unions allow arbitrary types of error.
SkiFire13 11 hours ago [-]
> In this particular case, it is type safe
That depends on how you interpret the pointer type, but in any case there are multiple interpretation and just the fact that these exist will create confusion and type unsafety.
LordDragonfang 19 hours ago [-]
> the compiler can invoke nasal demons if it makes the code go faster
I had never heard that phrase, and was delighted at the answer when I googled it its origin, thank you:
> “When the compiler encounters [a given undefined construct] it is legal for it to make demons fly out of your nose” (the implication is that the compiler may choose any arbitrarily bizarre way to interpret the code without violating the ANSI C standard)
cowlevel 13 hours ago [-]
However this is limited by the compiler's actual abilities and code. The C standard doesn't forbid nasal demons, but the compiler isn't programmed to invoke nasal demons and isn't running on hardware with that capability, so it won't happen.
I've heard nasal demons invoked as a literal possibility. No they are not a literal possibility.
Xirdus 12 hours ago [-]
Maybe your compiler isn't, but GCC is designed to summon every nasal demon it can. It especially loves to elide null pointer checks. There was a PR that almost made it to Linux mainline that did some weird stuff in kernel space where memory mapped to address 0 is valid and contains important stuff. It got a pointer, read data from it, then checked if pointer is null before giving root permissions for some operation. Reading the code, everything seems alright. Even stepping through it with debugger shows that it works. But when compiled with -O2 or higher, GCC removed the null check and now everybody got root permissions unconditionally - because reading from null pointer is UB regardless of circumstances, and GCC treats UB as dead code even if it is reachable.
cowlevel 11 hours ago [-]
> GCC is designed to summon every nasal demon it can.
Which is zero, to be clear. There's not even a standard USB device class for a nasal demon releaser
A null check elision is not a demon up my nose - it's a null check elision.
Xirdus 4 hours ago [-]
Sure, and char[] is no string because you can't tie anything with it. And don't get me started on the insects that allegedly live inside digital source files - that's an absurd concept, no self respecting computer specialist should be talking like that.
neomantra 8 hours ago [-]
Having made devices for other human holes, you can just use the standardized HID class. It can be adapted for nasal demons incoming and outgoing.
miki123211 7 hours ago [-]
Wait, so if you're running on embedded, you're not allowed to store anything at address 0, requiring you to waste that little bit of RAM?
That's... a bizarre choice, considering how religious C is about not making any assumptions about what a machine can and cannot do, even going to the trouble of working with non-IEEE floats and int types that are not a multiple of 8 bits.
Xirdus 4 hours ago [-]
Unfortunately yes. ISO/IEC 9899:1999 footnote 83: "Among the invalid values for dereferencing a pointer by the unary * operator are a null pointer".
poly2it 21 hours ago [-]
You might consider it type safe according to C's rules, but it is not type safe according to contemporary opinion because the success type and error type are the same.
eru 16 hours ago [-]
> [...] because the success type and error type are the same.
That's a bit weird? In Rust, if you have an thing that returns Result<String, String> that's still type safe, even though success and error types are the same.
I think I understand what you want to express, maybe. You might say it's in-band vs out-of-band error signalling?
itishappy 15 hours ago [-]
> In Rust, if you have an thing that returns Result<String, String> that's still type safe, even though success and error types are the same.
These types are not the same:
Ok<String>
Err<String>
markasoftware 12 hours ago [-]
Those are not types at all. Ok and Err are type constructors for the Result<A, B> type that take arguments of type A and B respectively.
SkiFire13 11 hours ago [-]
> type constructors
Nit: the type constructor is `Result`: given two types `A` and `B` (not instances of those types, but actual types) it gives you a new type `Result<A, B>`.
`Ok` and `Err` are called "variants" and can be used to construct a new _instance_ of `Result<A, B>`.
quotemstr 21 hours ago [-]
Contemporary opinion among who? Words mean things, and "type safe" simply does not mean "agrees with my preferred style of error reporting"
poly2it 20 hours ago [-]
Papers on type soundness from the last 30 years usually explore the domain further than C semantics.
pron 19 hours ago [-]
Soundness means that the program behaves in accordance with what its types express. You're right that C may be unsound due to undefined behaviour, but I don't think this is about soundness. What you're saying is that papers on types show how you can use types to express more constraints.
ranger_danger 16 hours ago [-]
There can be other valid perspectives and definitions of that word than yours.
I think almost all (tech) arguments stem from simple disagreements and/or different assumptions made on the definition of a (subjective) term.
pron 15 hours ago [-]
You may be right in general, but soundness comes from formal logic, where they're pretty fastidious with their definitions. Soundness certainly doesn't mean the expressivity of the types (i.e. what constraints they can represent); it's about the relationship between the runtime behaviour of a program and the types in the code.
ranger_danger 10 hours ago [-]
Again I think you're trying to push a specific definition as more correct than OP's somehow. Why not just ask them what they meant to understand their intent/context instead of saying they are wrong?
Unless the point of your argument isn't to understand OP but simply to "win."
pron 6 hours ago [-]
I think I understand their point and even said, "What you're saying is that papers on types show how you can use types to express more constraints," and that's correct. I merely pointed out that that's not soundness, which is something I think they'd like to know. There may be some leeway in the definition of soundness and how it's similar to or different from type safety, but either way, type expressivity is something different altogether (and C certainly has issues with both expressivity and type safety or soundness, but they're different issues). So the point isn't to win - the argument was right and I'm not claiming that C's type system is expressive - but to explain the different terms and their meaning in programming languages and formal semantics.
ciefa 20 hours ago [-]
Can you link some?
quotemstr 20 hours ago [-]
That's not the same as asserting that soundness requires adherence to the current fashion, sum types over results and errors. TIMTOWTDI.
You don't have to think C is sound or good to distinguish a disfavored API style from mathematical unsoundness. There are infinitely many type safe APIs and languages that are, despite their soundness, ugly and hard to use.
Let's not project every dimension along which a language can be good or bad onto the axis of safety.
majorchord 17 hours ago [-]
Opinion among people who know better than to try to authoritatively explain subjective words with rigid dogma, because that just makes them look silly.
15 hours ago [-]
rjbwork 24 hours ago [-]
My thoughts exactly.
"Make invalid states unrepresentable" is such a useful adage when designing even basic systems, much less the gates to control your server.
citrin_ru 1 days ago [-]
My conclusion is that an old tradition of not returning NULL in crypt as a workaround for buggy code which doesn't handle errors was a bad idea.
shess 13 hours ago [-]
The idea of trying to provide a workaround for buggy code calling crypt() is terrifying. For sure, trying to grease the skids for buggy code calling printf(), go for it. But it is hard to imagine cases where slightly-incorrect usage of crypt() is a good idea.
jeremyjh 1 days ago [-]
This is still a consequence of a poor type system.
creata 23 hours ago [-]
I don't think so. Even in a language with a good type system like Rust, somebody could have made this API:
/// Hash the phrase with the given setting.
fn crypt(phrase: &[u8], setting: &[u8]) -> [u8; CRYPT_OUTPUT_SIZE];
Then, in a later version, `crypt` needs to become fallible, so they need to signal errors somehow. But because they really want to preserve backward compatibility, they can't change the output type to `Result<..., ...>`. And so they end up doing the same thing that was done in libcrypt - they output some error value and hackily make sure it's not equal to the `setting` string, under the mistaken assumption that the output of `crypt` would only ever be compared to the `setting` string. And then someone creates the exact same bug.
Maybe I'm wrong, but it looks to me like the exact same incentives and means to screw things up exist in C and in Rust for this particular bug.
markasoftware 23 hours ago [-]
IMO type safety requires both language features and a mindset where you take efforts to keep things type safe. Your example has only the former.
For example, in a Rust web app, you can choose to use String for encoded URLs, unencoded URL fragments, escaped html, unescaped HTML, usernames, etc. Or you can choose to use newtypes and carefully design all APIs to only take and return the correct newtypes.
gf000 23 hours ago [-]
You are looking at it from the wrong direction.
The problem is not that you can write bad stuff in rust, the problem is that this particular usage pattern can't be sanely coded in C. This is a public API that should inform users and enforce certain properties. A Results type does exactly that, you would have to go way out of your way to force invalid stuff through that type alone.
creata 23 hours ago [-]
I don't think so. In both Rust and C, the solution is the same - make the function explicitly fallible, instead of trying to preserve backward compatibility. In C, the function would probably just look like this, returning zero on success and negative on failure:
int crypt2(const char *phrase, const char *setting, char *out);
You can call such a signature "insane" if you want, and that's fine, but it would have averted the bug that is the topic of discussion.
markasoftware 22 hours ago [-]
what you are suggesting is a form of type safety. There are still degrees of type safety even in C, and this bug resulted from deciding that backwards compatibility is more important than type safety, which I think we agree on.
lokar 21 hours ago [-]
And C has effectively deprecated bad APIs over the years. Just not this one.
quotemstr 20 hours ago [-]
It's a perfectly sane signature, especially coupled with an API surface that uses the return channel for errors generally.
19 hours ago [-]
cowlevel 13 hours ago [-]
This isn't good either. Crypt is a pure computation that can't fail for arbitrary reasons. The former implementation could fail if the salt was invalid, which is something you could guarantee. Now it also fails if the password is invalid. That's new behaviour and no matter which way you wrap it up it has the potential to be incompatible. They should have added the maximum length recommendation to the man page, and let DoS attacks continue until the calling software was updated, not the crypt library.
But in your version, you can't preserve backwards compatibility, making an infallible function fallible is a breaking change. For something comparable to libc you might want to (eg Rust std still has mem::uninitialized), but even with security issues, I feel like people would be more inclined to deprecate for a new function (or panic, which does have its own problems).
A good type system also means things are written with better forwards compatibility, and can be safely changed later. The Rust equivalent to `crypt` likely wouldn't exist: a low-level library would just provide hash functions, leaving callers to mitigate DOS etc; a broader library might return some opaque handle with equality/serialization, which you couldn't try to compare to a passwordless user; a high-level library would parse the account store itself. None of these really need to change in incompatible ways, they have all the information to add features and fixes at their own level of abstraction.
eru 15 hours ago [-]
Yes, even in Lean you can just refuse to prove any properties.
But if you have a stronger and more expressive type system, you are more inclined to use it, instead of looking for silly workaround like in C.
skew-aberration 15 hours ago [-]
You wouldn't be concerned about breaking backward compatibility with rust though, would you? A recompile would be needed anyway and the change would be detected and enforced. The rationale for the bad API here (two types in one variable) was that users were ignoring NULL - that's the part which wouldn't happen in rust.
justmedep 14 hours ago [-]
Maybe he meant binary compatibility.
cowlevel 13 hours ago [-]
They would have to make it crypt2.
zbentley 1 days ago [-]
Specifically, that the lack of tagged unions (or really, structured return types in general; you can pretend those are a tagged union if you believe hard enough) in C library interop conventions is a terrible foundation on which to build widely-used APIs.
And that just ends up indicting the C language itself. Can't send errors down a separate channel; folks might not inspect errno. Can't return NULL on error, folks might compare it blindly to another result. No tagged unions, so can't return one. Can't crash on error because there's no scoped error handling. Can't return a struct for checking .error/.result separately, because alignment and equality (and tearing and accidental clobbering and more) make that brittle as hell for widely-used APIs.
C is not a good language for library APIs. Like, really not. It has its uses/strengths, sure, but without even getting into the typical gripes people have about C, libraries--one of the things that it is used for the most--are one of the things for which it is the least well-suited.
"But it has a stable ABI!" So does a land mine.
cozzyd 22 hours ago [-]
you can return a struct just fine... most API's don't though.
cowlevel 13 hours ago [-]
Historically it didn't work well between different compilers. Maybe it still doesn't.
chuckadams 4 hours ago [-]
Passing and returning structs by value has been part of C since C89. Most compilers pass small structs in individual registers, and nowadays with LTO they'll even elide the copy of bigger ones if they can. What's definitely the case though is that by-value structs weren't a common thing when most of the Unix API was written.
kps 2 hours ago [-]
Structure assignment, passing, and returning were added in UNIX V7 C.
eru 16 hours ago [-]
That and in-band signalling is error-prone.
Another example that C programmers love: using 0 to mean infinity. Eg when you have a timeout setting, and 10 means ten seconds, but 0 means infinity seconds instead of zero seconds.
cowlevel 13 hours ago [-]
In general it's a problem of sentinel values. Some languages let you abstract this - I think in Rust and C++ you can define an "intrusive optional" which is like optional but uses an invalid value pattern to store the not-present state. In memory it's the same as a sentinel value but the compiler forces you to unwrap it like an optional.
eru 9 hours ago [-]
I'm not sure how well the 'forcing' works for Rust and C++. You can definitely do this kind of thing in dependently typed languages.
cowlevel 5 hours ago [-]
The type is akin to optional<int>. You can't add 5 to it, you have to dereference it first which crashes if it was null (0 or another chosen value).
Joker_vD 8 hours ago [-]
Amusingly, Win API of all places generally evades this pattern: 0 timeout means "no waiting", and to wait indefinitely, you use timeout of INFINITE which is actually UINT_MAX or, equivalently, -1 (in Win API, timeouts are unsigned, not time_t).
justmedep 14 hours ago [-]
That’s just a very bad API. You could do the same in Rust? There are nice C APIs that would use a nice struct that describes the timeout you want… eg GCD.
eru 9 hours ago [-]
I'm not sure a struct is the right thing here in general. You often want an algebraic datatype.
What Rust calls an enum, but an enum with payloads. In C terms it would be a discriminated union.
selfmodruntime 20 minutes ago [-]
The conclusion is, again!, that C is an outdated tool for secrure software.
l33tman 20 hours ago [-]
Is really the solution to a bug where a function had a wrapper that tried to solve its documented issue of returning *, to make yet another wrapper to try to solve the documented issue of returning true?
sgerenser 21 hours ago [-]
But, but 'always "succeeding" but giving an unmatchable hash is reportedly a better behavior than failing on error.'!!
amluto 19 hours ago [-]
Caveat:
> $ret = crypt($pw, $hash); $ret == $hash
musl is actually safe against this use. OpenSSL takes a bizarre code path, detects that $hash is too short, calls crypt($pw, $other_string), forgets that it detected that the hash was too short, and checks $hash == return value.
See my other comment.
mpyne 16 hours ago [-]
I did note that musl would normally work just fine here. Though I appreciate that your other comment goes into much better detail about what was actually going on in the code, as my analysis basically stopped once I understood the high-level flow of what was going on.
__david__ 9 hours ago [-]
Except OpenSSH qould have worked just fine if musl behaved correctly. Instead musl ignored the spec and tried to hack around badly written programs but ended up creating a security hole in a correctly written one!
idlewords 1 days ago [-]
This is a vastly clearer explanation than the linked page. Thank you!
markasoftware 1 days ago [-]
Yep, the linked page doesn't /feel/ AI written (maybe it's using a grok or muse model whose tone I don't quite recognize yet?) but it kinda just states a lot of facts that make sense on their own but don't cohere into a whole vulnerability like the comment above does.
numeri 24 hours ago [-]
I think it's just Claude prompted to write in a casual, conversational, no caps style.
The surface-level hints are gone, but the attempt to make every sentence pithy, the decisions on what information to present and what to assume the audience knows, and the over-emotionalization are all still there.
pigpop 4 hours ago [-]
It feels very AI written, it's just not the blatantly AI default style. Pretty much any prompt with a "write X in this style: [example text]" will produce writing like this.
lukax 22 hours ago [-]
Of course it is. "one debian box, one alpine box" is an obvious Claudeism.
ghostly_s 21 hours ago [-]
and what same human would embed two linux systems (??) in a blog post just for a demo that prints "pass"/"fail" that hardly anyone will bother to interact with?
larsonian 9 hours ago [-]
The demo is obviously vibe-coded, but that doesn't necessarily have to mean the prose comes from Claude too (although in this case it clearly does).
alexgoodhart 1 days ago [-]
Because Claude wrote the linked page. Claude is impressively bad at communicating these days. with gpt-5.6+ approaching Claude level like 30% of time and Gemini much clearer but also the most annoying 7 year old ever, it’s a pretty lame environment especially for people who r new to everything
pizzafeelsright 1 days ago [-]
I wondered about this myself as the explanations and documents I have been maintaining are becoming increasingly 'slop', but a different kind of slop.
In the past I could get an explanation, where I was able to track the line of thought. Now when I read through the explanation it is like someone has taken two or three explanations and merged them into each other, like there is a combining of iterations without clarity.
example:
The tree is blue.
The tall tree has the color of blue.
the tree, while tall, is blue.
That finally gets squished into: the blue tall tree has height and is evident of blue.
Is this double speak or injected double effort of words?
endymi0n 1 days ago [-]
Until Opus 5.0 I'd have agreed with you, but Opus 5.5 is honestly one of the best I've ever sparred with, vastly better than the rest, including Astra imho. Especially so when prompting for clear top-down, first-principles communication.
ericbarrett 1 days ago [-]
Disagree, I review a lot of Opus 5.5 code and its comments and docs are still pretty inane and Claudish. Is it better than the 4.x and 5.0 era, sure. But it's still nouning the verbs, using colons everywhere, a-not-b, "a foos the b" etc. A lot of lingual knots that need a human hand to untie.
brabel 1 days ago [-]
I was using opus5.5 for everything but since I tried google Argon4 that has been my favorite for writing. It’s clear and to the point.
yearolinuxdsktp 1 days ago [-]
Opus 5.5 is much better, but IME I always have to force it to review its writing (Opus 5.5) to get it to put the most important facts first (and not bury the lede). It loves to park the main point far away after intermediate clauses. It always writes “premise, so consequence.” It takes a prompt asking to review against rules in context for it to fix it.
The ‘concise’ output style setting is bad with Opus 5.5, as it will often spit out a cryptic (is this really English?!), allergic-to-commas concise response, which takes more time to parse.
xnorswap 1 days ago [-]
Oh it loves to bury a lede.
< mundane detail>
< If you don't run this command your computer will explode>
< exposition >
< mundane reminder to follow up something else >
Then 3 back and forths later, your computer explodes, and claude has the cheek to point out when questioned that it did warn you.
phyzome 23 hours ago [-]
Some of the page is also just wrong.
Except it turns out that at least one of those things is because I didn't enable JS, and the static text is wrong!
(Turning on JS did not actually help, because then the page is filled with horrible, cheesy animations and I can't actually read any of it.)
byronsharman 1 days ago [-]
Thank you for explaining it with proper capitalization. I understood your explanation much better than the author's.
dboreham 1 days ago [-]
> musl can return “*” for errors
wtaf ?
za_creature 1 days ago [-]
There is no doubt in my mind that this was intentional.
mmoll 23 hours ago [-]
I agree. The “excuse” for implementing those behaviors is just providing plausible deniability. The commit that lets crypt return “*”, followed closely by a change that lets you easily trigger that behavior sounds just like how somebody would sneakily introduce a back-door into tons of linux systems while making it look as innocent as possible.
zbentley 1 days ago [-]
Well, it was intentional, but not in the way I think you mean.
musl switched to returning `*` so folks stopped accidentally using NULL results, which was causing crashes and risked false-positives comparing NULL to itself.
That was a misfeature and should never have been implemented. But I bet it was an easy band-aid change to make!
ghostly_s 21 hours ago [-]
But choosing "*" as the character to return is an insane choice for anyone who knows the basics of linux sysadmin, or hell anyone who's ever used a search field in any context and thus knows it is ubiquitously used as a wildcard operator.
pdw 18 hours ago [-]
But it fits the common Linux implementation of the function... e.g. the Debian manpage says:
> Upon error, crypt_r, crypt_rn, and crypt_ra write an invalid hash to the _output_ field of their _data_ argument, and crypt writes an invalid hash to its static storage area. This string will be shorter than 13 characters, will begin with a ‘*’, and will not compare equal to _setting_.
cloudfudge 20 hours ago [-]
It's almost sensible if you only think of one side. It's like if /etc/passwd put "NO_PASSWORD" in the field for disabled logins and someone was like, oh, I'll be clever and say crypt() returns "NO_PASSWORD" on error because that's obviously invalid and won't match. But that's exactly what *does* make it match.
bryan_w 14 hours ago [-]
But this also needs the "ddos" patch that was submitted later as well.
> musl switched to returning `*` so folks stopped accidentally using NULL results, which was causing crashes and risked false-positives comparing NULL to itself.
Of all the things to mitigate...
1) crashing should be an option - it noisily indicates "there's a problem" and if it faults and dumps core, you've got a guided tour of your problem.
2) Is testing NULL as a passable option likely? I'd guess a more likely idiom is:
Wise words. I think early in my career I tended towards coding as defensively[1] as possible in a very specific way of "don't crash", as I definitely was not working in codebases I felt I had a full picture of. "I don't know who calls this method, and I don't want to get a call in the middle of the night that I broke prod" so I would say, return an empty string when asked to format a datetime except I'm passed a string or an int. (Yes, I was working in dynamic, untyped languages, too).
Later on, even in untyped languages, I came around more to "Throw loud and specific exceptions when asked to do things that are nonsensical" - it requires more overall cojones though because it may cause "your code" to crash. But the realization I came to was that it's better to crash ('a MaliciouslyLongPasswordError was uncaught') because "AuthManager asked PasswordService to check a password that was 2MB long" than to paper that over until something like this comes out. Or, the more common situation, you're passed garbage, you return something that won't choke the caller at the time, but what it saves to the database in that scenario ends up being corrupt data. 6 months later someone notices, and has to sort out a data mess that may or may not be recoverable.
[1] note that most "code defensively" advice is good. What I was doing could be thought of as a foolish misinterpretation of the phrase.
oasisaimlessly 7 hours ago [-]
FYI, your learned approach is also known as "fail fast" methodology[1][2].
“A good offense (SIGABRT) is the best defense(ive code).”
I dig it.
PokestarFan 24 hours ago [-]
I think the issue is that a lot of callers were not aware that crypt() could error at all and all the issues are downstream of that. If OpenSSH knew that crypt() could error they would have handled it (possibly by inspecting errno).
creata 24 hours ago [-]
I don't think that's the problem. The man page for crypt clearly says that erroring is a possibility and that it's not standardized how it errors:
> The behavior of crypt on errors isn't well standardized. Some implementations simply can't fail (except by crashing the program), others return a null pointer or a fixed string. Most implementations don't set errno, but some do. POSIX specifies returning a null pointer and setting errno, but it defines only one possible error, ENOSYS, in the case where crypt is not supported at all.
In my opinion the problem is that the behavior of crypt is underspecified - it isn't even specified how it signals failure, which is shocking.
Another disturbing decision that crypt's signature takes in the hash its output is supposed to be compared against - if it weren't for that, this half-baked "on error the output won't compare equal to the input" measure that crypt and crypt_* use wouldn't have been possible.
You can't check errno to see if there was a failure because, as the man page says, the value of errno is unspecified when crypt succeeds.
throwaway2037 12 hours ago [-]
> You can't check errno to see if there was a failure because, as the man page says, the value of errno is unspecified when crypt succeeds.
You can set errno to zero before the call. If the value is non-zero after the call, that is a good heuristic for failure.
zbentley 10 hours ago [-]
...unless crypt() is one of the many, many functions which themselves internally call functions that set errno. In which case you can't be sure.
23 hours ago [-]
za_creature 1 days ago [-]
As opposed to the false positives returned by comparing '*' to itself.
Why * specifically? Why not some random guid generated before committing, if it's so important to not let buggy software crash?
zbentley 1 days ago [-]
I'm agreeing with you, to be clear.
But buggy software might not crash; it might compare NULL == NULL and misbehave, instead.
So there's really no good option here, which is why C is a rancid choice of language for publicly-usable APIs. But we're stuck with it. So it goes.
dboreham 24 hours ago [-]
I've been programming in C since 1978 and never until now heard of a function returning "*" on error.
zbentley 24 hours ago [-]
This specific manifestation of "C is terrible for shared code APIs" is rare.
But issues caused by folks forgetting to check errno, miscomparing NULL to NULL when calling multiple APIs, dereferencing external-API-returned NULL due to missed error checking, mishandling inherited errno after successful action, or--the most common--mishandling negative integer return sentinels meant to indicate errors? Those are incredibly common.
za_creature 23 hours ago [-]
This specific library call is quite poorly designed.
Standard C api calls return an integer, which is 0 on success or nonzero on error. Ignoring it is an intentional choice. `errno` is the hack that shouldn't really exist because it's trivially easy to forget to check that and also has issues with threading (patched by making it a thread local) and more recently, coroutines (which admit no general purpose patch because they're a userspace construct).
Another reason why this is terrible is that it's unclear what the returned pointer's lifetime is. Is it until the next call to crypt? If so, is it thread-local like errno? If it's passed on to the user, is it safe to call free() on it, or is there a crypt_free function that should be called instead?
But all of this, and I mean all of it pales in comparison to returning '*' on error.
zbentley 22 hours ago [-]
> But all of this, and I mean all of it pales in comparison to returning '*' on error.
100% agree. This is an egregious failure. But C's error-handling story would be bad even without this, and the badness of C's error handling is what motivated the (bad) decision to return *.
It'd be great if we wrote systems software in a language that supported advanced features like...
checks notes
...returning multiple values.
Hell, even in VBA you had to at least spend the keystrokes to type "on error resume next".
gpvos 20 hours ago [-]
It's just that every other string than "*" would have been a better choice, including even the empty string (which would have been the next terrible choice).
tredre3 23 hours ago [-]
It's not a C thing, it's a Linux thing. The asterisk is relevant to how crypt() is used for /etc/shadow.
If the password field contains some string that is not a valid result of crypt(3), for instance ! or *, the user will not
be able to use a unix password to log in (but the user may log in the system by other means).
zbentley 22 hours ago [-]
No, it's a C thing. They previously returned NULL for invalid passwd lookups, but because there's no way in C to force API users to decide what to do if an error occurrs, that was causing program crashes due to NULL dereferences.
So they made the (very bad, but motivated by C's broken design/conventions) decision to return a magic string instead.
ruslan 20 hours ago [-]
> but because there's no way in C to force API users to decide what to do if an error occurrs,
0. There are tons of libc functions that return NULL on failure, it is conventional way in C to indicate failure.
1. There's no much to decide, just fail to login if crypt() fails with NULL.
2. errno is standard way in C to check error code of a failing function, but in case of login you hardly need this, just fail!
I totally agree that fixing this issue in musl was a pretty dubious solution. Don't fix what's not broken, fix broken application instead. Publish a CVE if you cannot fix it.
I looked through FreeBSD source code for how they use crypt(), it's checked for NULL everywhere!
UPD: Except libpam. :(
__david__ 9 hours ago [-]
Nothing you said is incorrect, but I believe they meant that there’s no way to force a C program to check the NULL correctly—the compiler will happily compile code that gets a pointer back from a function but doesn’t check for NULL before dereferencing it.
In something like Rust the type system won’t let you touch the value until you’ve done something to deal with the error. Even if you just want to ignore it, you are forced to acknowledge that the function is fallible by the type system—it’s a compile error if you don’t.
gpvos 19 hours ago [-]
Returning NULL on error is actually an excellent choice because most programs will crash if they try to use it without checking. A program, any program, crashing is better than this bug. Of course, modern programming languages have better constructs, but this was rare to nonexistent when C was started and if you think C is going away anytime soon you're delirious. musl's decision to change the return value to "*" is ununderstandable and likely intentionally evil.
zbentley 17 hours ago [-]
> musl's decision to change the return value to "*" is ununderstandable and likely intentionally evil.
Dude, they documented their rationale. It was dumb and wrong but totally relatable. Irresponsible users were getting crashes, and I’m sorry to tell you that for a lot of library maintainers
> Returning NULL on error is actually an excellent choice because most programs will crash if they try to use it without checking.
…is not something they want to write in responses to bug reports, even though it is quite true. It’s human nature to want to prevent your stuff causing problems for others, even if the problems are fully their fault. Most people cannot and will not be Steve “so don’t hold it like that” Jobs, for fear that their decision will be just as short-sighted as his was.
In this case, that was obviously the dead wrong call. But I’ve been in that seat, and “closed: user error” for the 100th time feels bad to write. Add to that a touch of misplaced cleverness, and you get “hey…what if we just returned an impossible string instead of an error?”
Evil requires motive or malice, and I’m having trouble coming up with those in this case. These patches weren’t concealed or miscategorized a la xz, the behavior was discussed in the open. They were just, in hindsight, really poor decisions made for ordinary reasons. Hanlon’s Razor and all that.
That’s why I’ve been harping on C being trash for shared-code APIs in these comments: bad decisions are gonna happen, and irresponsible users are forever, so can we maybe use tools that require you to put on a seat belt (mandatory error handling) before starting the car?
__david__ 8 hours ago [-]
> That’s why I’ve been harping on C being trash for shared-code API…
Agreed, but I can’t think of any other widespread ABIs that are better. Rust’s is not stable both C++ and Rust have concepts that don’t map into other languages well, so using them wholesale is a no go. So it would probably need to be a subset of those. Honestly just tagged unions would probably be enough. Maybe the solution is to get native tagged unions into C…
za_creature 9 hours ago [-]
> “closed: user error” for the 100th time feels bad to write.
That's the job of a OSS library author.
nuxi 21 hours ago [-]
Now imagine every other C API that can potentially return NULL did the same thing. Is that in any way sensible?
You can argue that C has broken design/conventions, but the musl decision was really really awkward.
patmorgan23 21 hours ago [-]
This sounds like a program problem not a library problem. Why did the library try to solve it?
za_creature 22 hours ago [-]
Oh, it was magic alright.
cowlevel 15 hours ago [-]
C programs wouldn't accidentally compare NULL to NULL. They'd either check for null explicitly, or they'd pass the null to strcmp and crash.
Of all the possible strings this seems like the absolute worst choice. If I were typing that string I'd pause to think "wait, what if the caller uses this with a regex?"
JdeBP 22 hours ago [-]
One has to remember the context here. The only error possible according to the SUS is ENOSYS, literally that crypt() is not actually implemented in the library. The traditional AT&T Unix crypt() was always implemented, and its encryption did not have failure modes, so there was never any error return to handle. This is why programs were written to never check the result of crypt(). It was only ever NULL on systems where one couldn't use the mechanism at all in the first place.
So the knock-on effect is that libraries that do have errors other than ENOSYS to accommodate, have to return magic 'impossible' values instead of setting errno and returning NULL, because of the weight of existing applications that expect nothing but the standard, traditional AT&T Unix (and indeed old BSD), behaviour.
Check the GNU libc crypt() doco. It defines a whole family of 'impossible' values that signal errors, by dint of listing a load of magic characters that should never be in the encrypted string. Including asterisk.
Also check FreeBSD's OpenPAM for a case in point. Even today, it does not check for a NULL return from crypt() and leaps straight to strcmp().
If you want to see real history here, check out the 1992 comment by Eric Andrew Young, that is still in the source for OpenBSD today (in des/des_fcrypt.c), on how SSLeay's crypt() had to be adjusted so that any single character password field, including "*", did not end up successfully matching itself.
It's worth pointing out that musl tried to do the same as SSLeay, and deliberately returns something different to the original salt so that they will not compare equal. It does not return "*" if the input is actually "*" in the first place, contrary to the headlined cartoon explanation. (The locus of the problem is the caller calling it with a salt that is not "*" and then comparing it with "*".)
boomlinde 12 hours ago [-]
> Check the GNU libc crypt() doco. It defines a whole family of 'impossible' values that signal errors, by dint of listing a load of magic characters that should never be in the encrypted string. Including asterisk.
Although that's a fine distinction for the point being made, as it's still the manual page for the crypt() function that one gets on the operating system, which that other manual page is not.
boomlinde 6 hours ago [-]
That naturally depends on the operating system and the crypt implementation they include.
ruslan 20 hours ago [-]
> Also check FreeBSD's OpenPAM for a case in point. Even today, it does not check for a NULL return from crypt() and leaps straight to strcmp().
Damn, that's a bug! Please file bug report.
JdeBP 18 hours ago [-]
I suggest finding a code path where FreeBSD crypt() ever returns NULL, which would mean one of its internal functions returning non-zero, before getting too excited. They all end in return (0) . (-:
As I said, the callers of crypt() have for a long time been using the knowledge that crypt() on their platform simply never returns NULL. Unix has never been returning NULL from crypt() as far back as 7th Edition, when it simply always returned a pointer to a single internal static buffer.
The reality is that all C library people have to work with a corpus of Unix applications that do not check for crypt() returning NULL that goes back almost 50 years, and an error return mechanism that barely exists outwith the pages of the SUS.
That includes callers in single-user mode in process 1 in some configurations.
Well, crypt_blowfish() has a number of returns with (-1). I'm not familiar with Blowfish, so it's hard for me to trace the possibility of failure here. But what if in a distant future someone adds more hashing algorithms to the list without possessing historical knowledge of this "never return not 0" ?
Failing to check result of crypt() in init cannot do much harm, doing so is equivalent of going system reboot (crash init :-)). Besides, whoever has access to system console can do whatever they want with the system, no password can prevent from that.
JdeBP 8 hours ago [-]
I have no doubt, despite that Marshall Kirk McKusick is still committing in 2026, that FreeBSD developers will come along that do not know history. But they will get pointed to the manual page that (amusingly) says that returning NULL is 'not a standard behaviour' and that AT&T Unix never did that.
NetBSD's manual is more detailed on the subject, and explains the bind that library implementors have been in for decades, thanks to almost 50 years of applications knowing what crypt() 'really' does.
And this despite the SUS trying to shoehorn an error return mechanism into crypt(3) in order to accommodate the problems that operating systems had exporting encryption code, two decades afterwards. (The 1997 GoSolo book explains that this is why there's an ENOSYS error.)
za_creature 21 hours ago [-]
The most charitable interpretation here is that musl authors are incompetent.
There is simply no universe in which a security-conscientious reviewer looks at the code, spots * being a valid return value, remembers that * is the default value in /etc/shadow for "no password login please" and then is all like:
This is fine, you check for * and return + instead.
There are more 32-long character sequences than atoms in the universe. Pick one, any of them instead.
odo1242 1 days ago [-]
Returning a string on error honestly is defensible (especially in a language like C where the type system doesn’t have exceptions or result types), the question is why the string was “*” instead of like “crypt ERROR: input password cannot be longer than 256 characters” lol
mmoll 23 hours ago [-]
No. Musl claims (claimed?) to be strictly POSIX compliant, and POSIX requires crypt to return NULL on errors.
tredre3 23 hours ago [-]
"*" is what's been historically used to represent an invalid/no value in passwd/shadow, so it makes some sense.
Of course given the chance that one might do a direct string comparison between crypt's return and the passwd/shadow file's hash field it was still a not well thought out choice.
gf000 23 hours ago [-]
No, it's absolutely not defensible.
dboreham 24 hours ago [-]
Precisely.
smnc 11 hours ago [-]
> The goal from an attacker’s perspective is to get this to be true: `$ret = crypt($pw, $hash); $ret == $hash` when called by OpenSSH’s sshd.
A typo: `$ret = crypt($pw, $salt)`
paparazzi678 22 hours ago [-]
weird chain of events and could have been handled in a better way for sure
1vuio0pswjnm7 21 hours ago [-]
"The OpenSSL adjustment ensures a minimum hash length for the user before logging them in, even if crypt() succeeds somehow."
OpenSSL??
mpyne 20 hours ago [-]
Sorry, should have written OpenSSH.
forrestthewoods 22 hours ago [-]
Great explanation. This affirms my biases that shell scripting and everything-is-strings is a mistake. Horrific.
Woodi 12 hours ago [-]
Shell is for humans to read and act on it.
Not to programmatically parse 'ls' output or some /proc/some/file.
Eg. git started as Perl scripts then moved to C to realy have data straight from C calls from system.
So for example all Internet things calling "openssl -some -options ...." are pure trash :>>
JoBrad 4 hours ago [-]
Even good, well-founded intentions can end terribly. The mistake is doubling down on those decisions when the mistake has become obvious.
I am also in the camp that CLI tools should return objects that can be parsed (by the shell or otherwise).
BoppreH 1 days ago [-]
In-band signaling[1] strikes again. * should not have been used as special value, neither for "no password" in /etc/passwd nor "error while hashing the password" in crypt. I'm afraid we'll find many more of these abuses, specially in string manipulations. The null was ok, an exception (or equivalent) better, and Result<T> (or equivalent) best.
Exceptions are almost never the right choice. They create extremely complicated control flow extremely quickly, and then you have to test all the different control flows. Java software is an example of how this goes sideways fast. There's no way to create a class that prevents callers from reading the object before the constructor runs (in multi-threaded software, new first hands a reference to the caller, which can put it on the heap, then it calls the constructor), and objects are default initialized, so all fields (even static ones that are assigned in the constructor) can be null.
So, any field de-reference in Java can produce a null pointer exception. Try testing all paths in something like:
There are M * N * P paths. The only way to write testable code with exceptions is to panic on uncaught exception, and to never catch unexpected exceptions. Even then, you're relying on nothing unexpectedly throwing the exception type you caught.
I'd argue Result<T> >> NULL >> sentries >> exceptions for this reason. Sentries are terrible, but at least they're not computationally hard to use correctly.
BoppreH 1 days ago [-]
I'd argue that's the fault of having default nullability, not the exceptions. You would have the same problem if every attribute access returned a Result<T> too.
And I'm not a fan of breaking control flow, but I still find it preferable to nulls. A null may travel far from the place where it was generated before blowing up, making troubleshooting harder, or it might come from a function you called for its side effect but forgot to check the return value. I gotta say I'm also a careful enthusiast of checked exceptions, it's great knowing which ways a function can fail.
But I think the consensus is growing that Result<T> (plus warnings for unused return values) is simply the superior choice.
hedora 24 hours ago [-]
C has what you are asking for, mostly:
int foo f() __attribute__((warn_unused_result))
As long as an error condition on the return value causes a crash on use, then it's mostly the same as result. (The main remaining difference being that you do an implicit `.unwrap()` in C, and in things like rust that's an explicit code smell.)
gf000 23 hours ago [-]
> There's no way to create a class that prevents callers from reading the object before the constructor runs
That can only happen if you leak the reference (given we are talking about java language level semantics and not byte code), which I give you is unfortunately not that hard to do.
As for exceptions, I think they would pair beautifully with software transactional memory, which would work well with effects. In that case (and no side effects) it would not matter what code path we took, we could just go back to a known safe point.
Because ergonomy-wise I still think exceptions win (even though I do like Maybe types, and Rust's ? macro is plenty good):
- they let one focus on the actual happy path code
- no error is ever swallowed, unless explicitly written so (looking at you go with useless if errs that fail to properly handle the error), they are always handled somewhere
- they automatically "unwrap" the Ok result, and automatically bubble up the Err side.
- bubbling up is especially useful because most errors can't be handled at their source (e.g. IO error is meaningless, how you handle it depends on what you want to do in the first place, which is done several layers higher)
- they automatically attach a stackframe/source, while with result types you may get a list of Ok Ok Ok Err and have no idea how that Err got there.
hedora 2 hours ago [-]
As the author of the class, I cannot prevent my caller from leaking the reference, so my public API allows for pre-constructor invocations of all my methods.
Software transactional memory doesn't really work with exceptions, since it precludes the use of catch + finally. It looks more like a top-level panic handler (where the throwing code cannot permute the heap, which is cool). To get exceptions, you'd need nested software transactional memory, which gets into obscure transaction processing theory. There's also the problem that your transaction could side-effect the outside world, then throw and be aborted, so you need to rework how I/O works if you walk down that path.
Exceptions are basically write-only, so I agree that their ergonomics for that use case are pretty good. Panic is better though, since it's really no error handling. Panic is easier to test, and easier to read / understand.
Sohcahtoa82 24 hours ago [-]
> There's no way to create a class that prevents callers from reading the object before the constructor runs (in multi-threaded software, new first hands a reference to the caller, which can put it on the heap, then it calls the constructor)
Which is why the pattern of double-checked locking doesn't work.
It's wrong. The caller doesn't get the reference until the constructor runs. Although it looks like two operations in bytecode, the only thing you can legally do with an uninitialised object reference is call its constructor.
hedora 1 hours ago [-]
In java, the caller gets the reference before the constructor runs. It's in the language spec. It's terrible. You can prove it yourself with something like:
this.dangerous = new Lol();
and then have Lol()'s constructor sleep for 10 sec then set done = true at the end of the contstructor. In another thread:
while(!this.dangerous) { }
assert(this.done) // will throw.
I guess you could use double-checked locking as a class author by having every public method on your class do a if (!this.done) { throw; }, and make sure there are no public fields.
That's pretty lame though.
z3t4 9 hours ago [-]
You should never ignore exceptions! The least you can do is to crash the program. Then you fix the bugs via battle hardening until it no longer crashes. Making the program not crash by ignoring the errors will just mess with the state of the program until you have no idea what caused it. It's better to let it crash, load the dump into your debugger tool to see what went wrong, then fix the bug, and repeat until you have a stable program. Rather then just live by the motto "ignorance is a bliss".
hamilyon2 24 hours ago [-]
> never catch unexpected exceptions
Yes, broad exception catch clauses are simply incorrect code.
Panic is maybe an overreaction, "scratch everything and orderly abort" is sufficient.
Ey7NFZ3P0nzAe 24 hours ago [-]
I'm inclined to say a Panic is absolutely preferable to root auth.
quotemstr 20 hours ago [-]
Exceptions work fine and enable a clean separation of business logic and error handling. Visibility into partially-initialized objects is a language design issue independent of exceptions. In a non-exceptional language, the corresponding access to uninitialized fields would just be bugs, not errors you can see.
Nothing in your post actually supports the idea that exceptions are "computationally hard". The CFG error edges exist however the errors are spelled
hedora 1 hours ago [-]
It is difficult to get to M * N * P control flow edges with O(1) result unwrapping code.
quotemstr 31 minutes ago [-]
You have exactly as many edges if every fallible routine can throw and every infallible one is marked as nothrow.
jedbrooke 23 hours ago [-]
Unfortunately we’re moving to llm based systems which rely entirely on in-band signaling.
__MatrixMan__ 1 days ago [-]
PSA: you can typically declare different types of string so that your type system objects to mixing data of different types even though deep down we know they're both strings.
BoppreH 1 days ago [-]
If you mean type aliases, that's a good idea within a single codebase, but the distinction between different types of strings doesn't survive serialization so it wouldn't have prevented this error. What they need is a way to signal errors out-of-band, like Result<T> or exceptions.
1 days ago [-]
__MatrixMan__ 1 days ago [-]
In python, aliases are separate from typing.NewType, but you're right: it's only enforced within a codebase and then only if the dev cares to run the typechecker.
So yeah, I guess it either needs to happen at runtime, or we just do signaling and user data on separate file descriptors.
Normal_gaussian 1 days ago [-]
This is presuming that the type system is nomitive rather than structural and there is not serialisation/deserialisation or type refinement occurring. A structural difference is much more preferable.
sepeth 19 hours ago [-]
It also looked to me a bit like a semipredicate problem:
This is a side-effect of Unix-y "it's all plain text" design that leans extremely heavily upon in-band signalling (among other design decisions).
If /etc/shadow was a JSON (or XML, even, although that would make me feel gross), this bug would have been much easier to see, because a field that reads '"hash": ""' sticks out a lot* more than a custom microformat that you don't understand.
With data structures, there's never "plain text" - it's all structured. The question is whether you want to use existing formats and tools, or create a new one for every tool and literally make footguns like this.
__MatrixMan__ 1 days ago [-]
A few years back I un-bashed my life as much as possible in favor of nushell (though it's not the only shell to encourage structure). I feel like it has really helped me notice a lot of this-needs-more-structure type problems outside of my shell that I otherwise wouldn't have.
The elegance of the unix-y approach is actually that it's all bits. You can just join things together and shove the bits through, that's a wild simplification compared with having vendor-specific kinds of plugs. But we could have that kind of compatibility while also benefiting from structure.
So I encourage everybody to give a shell with structured output a try, let it rewire your brain, and then ask yourself: how is the OS holding this back from being everything it could be? Because if we build that OS, it won't encourage these problems.
When I imagine it, it would probe either side of a pipe to see if they support negotiation. If not, send the bits through in the same old unixy way. If so, use the newest schema for structured data that is supported by both sides.
jonhohle 1 days ago [-]
At Amazon we had RecordStream which implemented most of the standard Unix tools on structured streams. It was pretty easy to write filters, aggregators, and formatted output in something not much more complicated than sed/awk/cut/join/sort pipelines. There was a convenient structured log format for services that was often used as input.
That looks really cool. I've been heading the opposite direction: posix shell for everything.
I have a bunch of management scripts, etc, that need to run on everything from a potato to a T-1000, and that's the only language that's always available. (OpenBSD sh, FreeBSD sh, busybox, bash, dash, zsh all can cope with it).
I'd love something that's at least JSON-typed (or equivalent), but that can be bootstrapped up on least-common-denominator posix.
Has anyone built something similar? (Yes, I'm being picky -- bun / ts is already a pretty low bar for cross platform support, but these scripts do stuff like install node, so I'm not totally crazy for asking.)
cowlevel 15 hours ago [-]
Python?
hedora 2 hours ago [-]
Not installed universally, and it also doesn't reliably run python programs. (Minor + major semver changes break backwards compatibility, and pip mutates the global system state.)
dns_snek 1 days ago [-]
The issue isn't the config, but the fact that crypt() doesn't have any way to signal failure. In better designed languages you would have exceptions, result types, or something. Even in C you have "return integer indicating success or error" which would've prevented the confusion.
Now the public API is hard to change but you could still avoid the issue by returning a longer token like "==DANGEROUS_CRYPT_ERROR_length==" or "==DANGEROUS_CRYPT_ERROR_unsupported_alg==" which would never match the password field unless someone was really trying to induce the issue.
knome 1 days ago [-]
>but the fact that crypt() doesn't have any way to signal failure
>Upon successful completion, crypt() shall return a pointer to the encoded string. The first two characters of the returned value shall be those of the salt argument. Otherwise, it shall return a null pointer and set errno to indicate the error.
returning a sentinel value of NULL and setting errno is pretty standard C, and shouldn't cause an error.
OP's sshhole link specifically notes that it _should_ return NULL, but some MUSL devs decided they wanted it to return trash instead, and the trash they returned instead of NULL caused this.
All because some devs were too lazy to check the result and another dev decided to be clever and "fix" NULL errors from crypt by sending back 'unmatchable' trash banking on devs not checking NULL.
edit: apparently crypt implementations have been trash ignoring the NULL return for a long time. this isn't only on MUSL, but it's still because people ignored and tried to work around the existing out-of-band error mechanism.
zbentley 1 days ago [-]
This is true, and "trash" is putting it mildly.
But all those sentinels were added for the reason that, when your code is a public API intended to be widely used by who-knows-who, it needs to be really difficult to misuse it. NULL-plus-errno is easy to misuse. Yes, it's a dangerous mistake and your own fault if you do it. But people have been making that mistake for years and don't show any signs of stopping.
> returning a sentinel value of NULL and setting errno is pretty standard C, and shouldn't cause an error.
> OP's sshhole link specifically notes that it _should_ return NULL
I meant there's no mechanism besides returning NULL that would make it hard to ignore the issue (exceptions need to be caught, results need to be unwrapped and their failure handled, an integer return value makes it clearer that there's something there that you need to check). I also missed errno codes when glancing at the manpage earlier which should illustrate the problem.
And yes, whoever first had the brilliant idea to start returning garbage never should've done that, I'm not disputing that.
n_plus_1_acc 20 hours ago [-]
The spec implies a string of at least two characters in the success case, am I reading that right?
mafuy 1 days ago [-]
If this is true, then the entire thread branch here on HN is entirely wrong. What is going on??
fc417fc802 13 hours ago [-]
It's subtly wrong, just like most of the rest of the commentary in this comment section. Yes, crypt is specified to return NULL on failure but if you read the rest of that page it is only permitted to fail with ENOSYS. The spec does not allow for failure for any other reason.
So if you're strictly adhering to the spec in this case then unfortunately NULL is not an acceptable way to signal a generic error condition. Hence all the seemingly bizarre special values. (In a strange turn of events all those experts doing things that looked weird turned out to know what they were doing. Go figure.)
hedora 1 days ago [-]
People don't RTFM, either on HN or when implementing "standards compliant" stuff like MUSL.
No error handling scheme can fix that problem. For instance, if a Rust API says I should return a Result<String, ENOSYS>, I can always write code like `if(ENOSYS) { return Ok("*") } instead of returning Err(ENOSYS).
Then my code would be compatible with MUSL.
zbentley 1 days ago [-]
Sure, but if you didn't do that, and returned the tagged union (in Rust or whatever language), your users would necessarily be forced to handle it in order to process the result.
The only reason this library did the insane thing of returning "*" is because it was extremely easy to cause crashes or false-positives (NULL == NULL) by returning NULL+errno. The reason that was easy is because C lacks a means of making error handling mandatory (exceptions, tuple returns, tagged unions, whatever).
If you return a tagged union, users might mis-handle it, it's true (https://blog.cloudflare.com/18-november-2025-outage/). But they have to handle it, which is a requirement that C APIs are incapable of providing.
hedora 24 hours ago [-]
You have to handle NULL in C, or your program crashes (though I suppose a misguided optimization of strcmp could do "a == b || really_strcmp(a, b)", which gets us back to your bug).
Also, in things like rust you can just do result.unwrap() if you want. In general, language constructs don't really save you from bad programmers.
jstanley 10 hours ago [-]
> You have to handle NULL in C, or your program crashes
But it crashes at runtime, so such a bug can exist for years with nobody noticing.
The existence of such bugs is what motivated the musl developers to return "*" instead of NULL.
zbentley 24 hours ago [-]
> a misguided optimization of strcmp could do "a == b || really_strcmp(a, b)",
In practice, that's less of a risk than:
char* x = function_that_might_return_null_and_error();
dangerous_function();
if (x == NULL) {...}
Or code for which pointer identity is relevant (e.g. accessing slices/subparts of a data structure):
char* x = returns_null_or_a_slice_pointer(data_structure);
char* y = other_function_that_returns_null_or_a_slice_pointer(data_structure);
if (x == y) { puts("you're fucked now!"); }
And for sure there are much better ways to write both kinds of code! But people routinely do not do that.
There's no difference between result and null other than readability (there's a mechanical transformation between the two, where you use unwrap() in rust on an unchecked null dereference, and you read errno in C instead of looking inside the Err() result variant).
To be clear, I strongly prefer Result<> because of the readability, but any dumb thing you can write with NULL + errno can be written with Result (and vice versa).
dns_snek 1 days ago [-]
> I can always write code like `if(ENOSYS) { return Ok("*") } instead of returning Err(ENOSYS)
You can do that but it's going to feel bad to write that and it's going to stand out like a sore thumb to anyone who reads it. But "errno" is just ambient state that's completely invisible unless you happen to have the manpage fresh in your memory.
whoanoway 7 hours ago [-]
[dead]
hedora 24 hours ago [-]
They had to write
if(ENOSYS) {
return "*";
}
after seeing that other implementations use different sentries like "*0" and that the spec says they MUST return NULL. That must have felt bad too.
dns_snek 9 hours ago [-]
Not the library authors but their users!
slashdave 23 hours ago [-]
A lazy attitude like "if we just used <n> language, all our problems would be fixed".
eqvinox 23 hours ago [-]
Yup, that's the way to explain it in order to blame musl.
Here's how to explain it in order to blame openssh:
openssh processes passwords and doesn't want to reveal through timing whether a user has a valid password (or exists at all). So far so good.
What they do if they hit a user with no valid password (i.e. "*") is that they look for another user with a valid password, to use the seed from. A bit weird already, but OK from a perspective of avoiding implementation specifics on seeds and hash algorithms. They know at this point that the result must be to deny access even if the hashed value matches, because that hash is for another user.
So they put the password received over ssh into crypt() with this other user's hashed value to get a seed. And then accept it if it matches, apparently?
Homework: what happens if the password actually matches another user? Can I log in as root if openssh randomly picks my user for the fake salt, and I type my own password correctly?
FLeXMurphy 1 days ago [-]
Except it's not trash. It's a backdoor funded by our glorious leader Kim Jong Un.
panzi 1 days ago [-]
Personally I think there was nothing wrong with returning NULL. "But if people don't check for it it crashes their programs!" Good! Let it crash loudly! Then they notice and fix the issue.
Also I think it could have returned ":". Given the file format of /etc/shadow and /etc/passwd I don't think it's possible to have a : as a value for anything.
hnfong 1 days ago [-]
> Also I think it could have returned ":". Given the file format of /etc/shadow and /etc/passwd I don't think it's possible to have a : as a value for anything.
I wonder if somebody else would have thought the exact same thing in a different (non-/etc/*) context, and then you get the same effect in a more obscure manner.
panzi 15 hours ago [-]
Indeed. Let's keep NULL.
throw10920 16 hours ago [-]
> The issue isn't the config
That's a meaningless statement. There isn't a singular "the" issue like you said. Any one of several different technical decisions would have completely prevented the issue. Including, to my point, /etc/shadow being a JSON file where an account being locked is an `"active": false` field instead of an asterisk with no intrinsic meaning.
BoppreH 1 days ago [-]
No, that's just more in-band signaling!
Some badly designed password hashing algorithms (cough bcrypt cough) have maximum password lengths, ignoring everything after X characters. So it could happen that I set my password to a 100-character string, it gets saved as "==DANGEROUS_CRYPT_ERROR_length==", and then anybody can log in to my account by entering an unrelated long string that returns the same "error".
If a function promises a string result, it should only provide a string if the function succeeded. The null was much better than magic strings.
dns_snek 1 days ago [-]
I know, it's bad. I agree that it should've stayed NULL. But if you really really really think you have an acceptable reason to do something like this (and you probably don't), the least you can do is use a non-trivial token.
BoppreH 1 days ago [-]
If I was really desperate and could only return a valid string, I would probably use a message and add a random component to avoid ever matching two errors. Like SQL, where null != null.
yearolinuxdsktp 1 days ago [-]
Returning “*” instead of NULL to signal failure is the real footgun. Don’t blame the format.
For historical reasons, we could’ve ended up JSON as “hash”: “*” just the same. The /etc/shadow format lets you skip values completely, like “::”, so your argument doesn’t hold.
Terminating an app because of a NULL dereference is the other footgun. Sadly, we still have “panic” in modern languages like Go and Rust.
hedora 1 days ago [-]
"panic" is a good thing. There's always some error condition that your code cannot handle. At some point, the least bad thing to do is crash.
For instance, what if your error handling code throws?
One spectacular example of panic saving the day was the Apollo 11 1202 error. It was basically a watchdog timer that fired if the landing computer was starved of CPU time. A misconfiguration caused the first moon landing to hit CPU overload. When that happened, the computer displayed 1202 and rebooted, then ran things in priority order. Apollo 11 landed in the middle of a tight 'panic, reboot, adjust thrusters' loop. Thanks to core memory, most of the heap survived the reboot, so the reboot step was fast enough for it to mostly land without human interaction. The pilot had to make ~1 adjustment to get it to actually hit the landing site, but the panic-ridden computer handled the rest of the descent.
sshole-type APIs < panicking APIs < required error handling.
Panic is fine, but fundamentally a whole-program crash is undesirable in a really wide range of use-cases. I don't want other SSH connections to crash (or the server to stop accepting valid logins until someone restarts sshd) if one person tries to log in with a credential that triggers a panicking bug in crypt(). Same for crashes in request handlers and so on.
The language/runtime itself doesn't (and probably cannot, at least if it's a general purpose language) distinguish between local and global error conditions, so aborts end up being either unnecessarily damaging, or languages like Go end up normalizing panic "localization"/handling in dangerous ways. Isolation using threads isn't robust enough. Isolation using processes is far too fiddly, slow, and opaque.
The better solution is required error handling. If user code must engage with the possibility of an error state in order to access a result, that's a huge step forward. They can do the error handling wrong, but if you make most people have to do it at all most of the problems will go away.
There are other good solutions too, like Erlang's process boundaries, but those tend to have drawbacks that make them inapplicable to a lot of programming areas. Tagged unions, on the other hand, are only inapplicable in C library code because of language defects and terrible community conventions around ABIs.
"I forgot to check for an error" should not be implicated in bugs in 2026; we have had a large menu of possible alternatives (checked exceptions, unchecked exceptions, tagged unions, etc.) for decades. But we're still over here writing C (and Go), languages pathologically incapable of providing this ability.
hedora 1 days ago [-]
SSH specifically has separate processes per connection the last time I checked, so a panic does the right thing: The attacker's connection drops uncleanly.
This is also why you can upgrade SSH over SSH without having the connection drop mid-stream.
Again: You cannot avoid "I forgot to check for an error". What happens if you get a sigpipe on stdout when printing the "I crashed" line? If Linux overcommits memory? If you turn off overcommit, port all software to check for + cleanly handle NULL from malloc (a substantial effort!), but then get a machine check exception from a double bit ECC error during an infallible read?
zbentley 24 hours ago [-]
Sort of. It does some of the validation/auth code in the sshd parent daemon.
It has to: if it did 100% of the post-accept(2) work in a subprocess it would be pretty easy to remotely forkbomb/PID-exhaust a system by spamming invalid logins.
> You cannot avoid "I forgot to check for an error"
Yes you can. Checked exceptions and tagged unions both force calling code to be written with error-awareness.
In SSH's case, "I forgot to check" might lead to crashes (which might knock over sshd so new connections are impossible), or it might lead to false-positives via comparing NULL == NULL.
I'd much rather that boneheaded API users have to write code like
How do you handle out of stack space on 'authenticate_user'? What about the machine check exception loading the DRAM page that contains "successfully authenticated"?
throw10920 14 hours ago [-]
> is the real footgun
This contains a false snuck premise - that one and only one thing was the footgun. This is categorically not true. An obtuse, poorly-understood, purpose-built text-based DSL is absolutely a footgun and if it was replaced with a JSON and inactive accounts had been indicated by `"active": false` instead of a freaking asterisk then this would absolutely would not have happened.
> For historical reasons, we could’ve ended up JSON as “hash”: “” just the same.
That's dishonest, because it intentionally conceals the fact that that's a
vanishingly* unlikely scenario.
People have brains that, factually, are far more likely to see a single context-less character "" and not realize the implications of it than to see `"hash": ""` and think "oh, that's normal".
This is just straight-up psychologically true.
> The /etc/shadow format lets you skip values completely, like “::”
Totally irrelevant.
> so your argument doesn’t hold.
False `::` is both (1) still harder to understand than `"active": false` and (2) even though the syntax existed it wasn't used here because everyone thought they understood the system.
Don't make excuses for this insane text syntax. It is, factually, harder to understand than a decently-written JSON format.
BoppreH 1 days ago [-]
Yes, absolutely. Take for example this cURL command:
Random parts of it will be interpreted by the shell, others by cURL, or the filesystem, or the remote server. They each have their own encoding rules, nested. I count ten distinct separators and nine ways of escaping/tagging. It's one command. No, just no.
We have seriously overused strings and we're now paying the price.
dolmen 1 days ago [-]
Also: free text is the input format of LLMs...
UqWBcuFx6NV4r 1 days ago [-]
At this point I’m resigning myself to think that we’ve just gotta wait for some people to die. It feels very dramatic and glib (no pun intended lol) and whatever to say, but at this point it feels like there’s a certain hostility in some parts to changing anything from the platonic ideals of however computing happened to work in 2004. “Oh we don’t need to mess with it!”
xorcist 1 days ago [-]
How did you arrive at that conclusion? If anything, this vulnerability happened because someone decided to change the behaviour of crypt() without fully understanding the consequences. This specific vulnerability did not exist in 2004.
timhh 23 hours ago [-]
I agree with him. I only managed to get asterisks enabled by default in sudo because sudo-rs isn't owned by people who have decided that the 70s was the perfect decade and nothing should ever change. And look at the insane pushback when it did!
I can't even get an obvious UX improvement through PAM because they like it broken.
I think this is a bit unfair. The way I see it, a lot of people are saying that changing breaks lots of old stuff, and risks complete fragmentation. Those are real costs to change. It might still be worth it, but let's not pretend that everyone is resisting change just because it's change.
abirch 1 days ago [-]
The costs to change are dropping significantly with AI.
All I see is sausage makers advocating for a sausage-based diet.
raverbashing 1 days ago [-]
> "it's all plain text" design that leans extremely heavily upon in-band signalling
Honestly I've been shaking my fist at these since 0 becomes True/Success in Bash and the opposite everywhere else
xorcist 1 days ago [-]
Error code 0 means success in a lot more places than just bash.
cwillu 1 days ago [-]
That's not specifically bash though, that's process exit codes
raverbashing 1 days ago [-]
Yes. But it's still contradictory
OnlineTux 19 hours ago [-]
The question you are asking when querying a process exit code is "did this program exit with an error?". The default assumption is "no, it completed successfully", so 0 (false) is the logical expected default return.
A function is expected to produce a result - so the question is "what was the result?". !0 (something) is the logical expected default return.
I've never found it to be contradictory.
xigoi 9 hours ago [-]
But in shell, 0 is true, not false. The program called true returns 0.
Dylan16807 7 hours ago [-]
"True has 0 errors" is the straightforward part.
"False pretends to have an error" is where it gets weird. It's not that they got booleans backwards, it's that they're stretching a tenuous analogy a bit too far.
killerstorm 1 days ago [-]
I'd say another UNIX-y thing - "C is good enough" - is part of the problem. C makes programming painful which results in all sorts of hacks.
E.g. in Pascal you can can define a record and then defined file of record to have typed I/O. I.e. it comes with built-in serialization format supported by compiler. Not a good one - but perhaps a step in a good direction.
OTOH C people starving for serialization often believe that `struct` byte layout made by a specific version of a compiler is a serialization format - a view not endorsed by the standard.
adwn 23 hours ago [-]
Pascal's record typed files have the exact same problems regarding endianness and word size as directly writing structs to files in C, so I don't see how they're any better. They're also a complete deadend and unusable for pretty much any file format from the last 40 years+, since you can't even handle a header or magic bytes at the start.
mkl 1 days ago [-]
Even using "no_password" and "error_while_hashing" would have prevented it. The problem was picking a generic, meaningless signal string.
mort96 1 days ago [-]
That would've happened to fix it, because "error_while_hashing" and "no_password" happen to not be valid password hashes. The right way would've been to have different syntax for hashes and non-hashes; you could've made it so that a hash starts with a '#' and if it doesn't start with a '#', the rest of the string should be ignored.
Or, for backwards compatibility, start non-hash strings with a character which can't appear in a hash, such as '@'. Your examples would then become '@no_password' and '@error_while_hashing'.
SoftTalker 1 days ago [-]
Well neither * nor ! (the two characters I've seen to indicate "no password") will appear in a valid hash. The problem was using a character that could also be returned as an error indicator, IIUC.
mort96 23 hours ago [-]
Yeah but it's not forwards compatible with future hashes where '*' is suddenly a legal hash, which is what happened here. You need a grammar which lets you tell the difference at the parsing stage between "this is a password hash" and "this account has no password". That's not the same as letting the value '*' flow through the system as normal and hope it doesn't happen to match the hash.
SoftTalker 19 hours ago [-]
True, but if they would have picked ?, or #, or any character other than * with its designated special meaning in the /etc/shadow file, then we would not be in this pickle.
cowlevel 15 hours ago [-]
hashes can't start with * and there's no reason to think they ever will
deniska 1 days ago [-]
Technically they did use a character which can't appear in a hash, '*'.
mort96 24 hours ago [-]
Right but not in a "rigorous" way. They kept the format of the file the same: the string between the colon characters is a hash. They exploited the fact that an asterisk happens to never appear in a hash, and just let the system believe the asterisk was supposed to be a hash to be matched again.
In a sense, they should've changed the parsing logic to return a struct with a field 'password_hash: Option<String>', and defined the grammar of the field to be: "a hash string, or '*' for no login". The bug is, in part, caused by the haphazard way they don't change any formal grammar or contract and just relied on the fact that '' happens to* never be a valid hash.
It's the difference between "happens to work" and "defined to work by construction". The former approach breaks once a hash function is invented where '*' is a legal hash (such as this new musl crypt() hash function where all strings longer than 256 bytes hash to '*'). The latter keeps working regardless because the asterisk is treated as "no password" already at the parser stage.
hobo123 5 hours ago [-]
Should have set crypt_errno.
1 days ago [-]
amluto 1 days ago [-]
The whole story is a bit worse than this. Musl already worked around this type of failure. See the second-to-last line here:
The vulnerable path in musl is only a little bit vulnerable, because "*" is not actually a sha256 encrypted configuration and should not reach this code: musl's crypt(..., "*") will not call that code! Turns out that sshd detects that the password hash is too short and does not forward the salt!
Look, no else statement! If that if doesn't get taken, then salt remains NULL. And the sshd code is really quite messy and doesn't keep track of the fact that this code path is inappropriate for validating an existing password hash.
A workaround in the sha256 path in musl wouldn't even have prevented the bug!
I'm going to give some credit to musl for unwisely using "*" and some credit to OpenSSH for having nonsensical control and data flow.
tchebb 23 hours ago [-]
> (hey, this workaround is wrong if the salt is "x"!)
When the salt is "x", it'll return "*" and still not match, won't it? The code is correct as long as the second argument is the hash that's being checked. And like you say, the sha512 path isn't relevant since you only get there if the salt started with "$".
> Turns out that sshd detects that the password hash is too short and does not forward the salt!
I think you're misunderstanding the intent of that check. It was added by the 2016 change linked in the writeup[1] and causes OpenSSH's xcrypt() to select and use a salt from a different line of /etc/shadow to prevent timing attacks. It is correct for a crypt() implementation that returns NULL on error.
> When the salt is "x", it'll return "*" and still not match, won't it? The code is correct as long as the second argument is the hash that's being checked. And like you say, the sha512 path isn't relevant since you only get there if the salt started with "$".
I think you're right. That's what I get for doing a very very quick read.
> I think you're misunderstanding the intent of that check. It was added by the 2016 change linked in the writeup[1] and causes OpenSSH's xcrypt() to select and use a salt from a different line of /etc/shadow to prevent timing attacks. It is correct for a crypt() implementation that returns NULL on error.
I don't care what the intent of the check is. If you're doing a fake password hash to obscure timing, remember that it's fake and don't let the user log in! Don't deliberately pass nonsense into the live check and rely on the nonsense check failing to prevent the fake check from accidentally working. And don't have a code path that thinks it's checking the real password but passes in the wrong second parameter to crypt().
Or, better yet, have an entirely separate function fake_check_password() that does the fake check, returns void, and cannot possibly be confused with the actual password check code.
echion 14 hours ago [-]
> If you're doing a fake password hash to obscure timing, remember that it's fake and don't let the user log in!
I think this is the crux. Instead of doing the right thing, they tried to codegolf to get the desired behavior and it got real subtle, real fast.
this is a little flip: maintaining a security-critical program naturally pushes one to minimize code changes because a 30-line change is usually easier to review than a 100-line change. Except when it's not.
tchebb 22 hours ago [-]
> you're doing a fake password hash to obscure timing, remember that it's fake and don't let the user log in
That would be a much better approach. I see your point now.
NewJazz 1 days ago [-]
The formatting in your comment got messed btw
amluto 1 days ago [-]
Thanks, fixed
aljgz 22 hours ago [-]
My general rule of thumb is: "no magic strings". This is much easier to implement in higher-level languages and when writing application/web code than system programming.
Don't return empty string/asterisk/anything else. Instead, return a failure value.
When absolutely needed, choose the magic string with more caution:
One string showing the SHA-256 of a non-existing hash? It's a good idea for it to be a valid hash. Now we have a new risk: if it's broken once, it's broken for all. So it's a good idea for it to be a random one for each machine and/or account. Even this might have problems I can't think of, but it seems to be as secure as an account with a real password.
If this is not a good decision for some reason, choose something unmistakable: "--ACCOUNT-HAS-NO-PASSWORD--". This is much less likely to be mistakenly used in a different context, is much easier to look up in a code base than "*", and is self-documenting.
sunnybeetroot 21 hours ago [-]
Agreed, unfortunately there is so much shit programming that does this that AI is trained on that I continually see it generating it in my code.
My AI usage involves telling it to do things and setting up lint rules to catch all the bad programming it’s learnt.
saguntum 13 hours ago [-]
100% agreed on no magic strings
One potential point of disagreement: it's easy to implement structural prohibition of magic strings in low-level languages too if you consider Rust a low-level language (which I've heard before, even though I didn't really consider it to be one for our application of it).
I'm not a Rust expert but do run a few thousand lines of it in prod for some perf/correctness critical code, and I wish it didn't have the reputation of being so difficult to comprehend. There's so much resistance to introducing it or new languages in our large org, that I'm surprised we were able to introduce our thin slice at all. I am not sure I'd use it where you need easy async primitives like goroutines, but it prevents whole swathes of bugs that are repeated over and over again with devastating impact, like the OP's bug, via thoughtful language design.
Maybe this resistance is an artifact of where I work more than large organizations generally, though. Curious about others' experiences. I'm not really an evangelist for the language but would really like to work in a more polyglot environment one day.
kykeonaut 3 hours ago [-]
MITRE scored this as a 8.1 with a high attack complexity. [0]
I fail to see how the attack complexity of this vulnerability is high, as all it requires is attempting an ssh login with a password longer than 256 characters to an affected alpine image with port 22 open. Am I in the wrong?
Above all, I’m a little sad that the musl developers decided to make a lot of “well it’s convenient” changes to core crypto functions — at least without the level of auditing that would have caught this! It’s tarnished their reputation in my eyes a little.
rustyhancock 1 days ago [-]
I think everyone is over fixated on AI finding complex bugs beyond a typical person's understanding either to locate or exploit.
I think for any one of those bugs there might simply be half a dozen footguns or bugs that once it's spelled out you wonder how anyone let it slide.
I think that's the case where LLMs might indentify most.
It's challenging though as the current AI audits produce so many false positives it's hard to know when it's being pedantic or when it's genuinely caught a human being a bone head
jamal-kumar 1 days ago [-]
Finding more and more that these complex bugs people gloss over like you are now are being used in exploit chains and the end result is still my clients getting hacked
svnt 1 days ago [-]
This one might be hard to find using the author’s approach of throwing everything at various builds of ssh.
It is not really that hard to find in the wild if you had a device built this way, when overflowing buffers on embedded devices is a standard attack vector.
It might be hard to root cause, but it would not be that hard to find.
rpcope1 1 days ago [-]
Honestly the ideas around musl are interesting, especially being able to statically link, but given the sort of nonsense around how they handled pthreads, and generally other trade offs, it's hard to imagine wanting to use it over glibc (which everyone shits on) if you don't absolutely need static linking.
pelagicAustral 1 days ago [-]
Everything in this industry is about convenience.
0xbadcafebee 1 days ago [-]
Musl has a history of introducing (and not fixing) major bugs in their library that aren't in glibc; I would not depend on it for production
trebligdivad 1 days ago [-]
In many ways each individual decision makes some sense;
wanting to stop loads of programs being affected by a (remotely triggerable?) crash
wanting to avoid time based attacks
wanting to be able to set accounts to not have a password
Each one relies on a perfectly valid argument - that * can't match because we all know it'll be rejected by the other thing.
yangm97 1 days ago [-]
I would be more concerned about the lack of explicit return types, one of the arguments for strongly typed languages is that you should not have ambiguous data and yet…
Granted I mostly work with high level programming languages so I may be missing something.
preg_match 23 hours ago [-]
Yes this is C, C is not strongly typed at all and the type system is very primitive. NULL ptr is the return value you should probably use if something goes wrong. But you can’t, say, return a sum type with the result and an error. It’s not uncommon at all to use magic sentinel values in C to indicate error conditions.
Retr0id 1 days ago [-]
> Each one relies on a perfectly valid argument - that * can't match because we all know it'll be rejected by the other thing.
But that's not a perfectly valid argument, and it never was.
aidenn0 19 hours ago [-]
I guarantee that every single developer of musl in 2012 had at least account on their machine with "*" as the password hash. Also crypt() is a standardized function and the standard states that a null pointer is returned on error.
That being said, OpenSSH did mismanage their attempts at handling a garbage return-value from crypt(), so there's definitely some blame to go around.
woah 1 days ago [-]
If they had just all used a different garbage symbol would this have been avoided?
zarzavat 1 days ago [-]
The real problem here is the C language. Our industry should have dumped it a long time ago and we’re now paying the price of prevarication.
oneeyedpigeon 1 days ago [-]
Which of the several flaws in this bug chain would have been materially different with a language other than C?
nh2 1 days ago [-]
Any use of sum types would have prevented this immediately.
The whole thing exists because APIs that return a NULL value that "nobody checks", plus a string that you shouldn't use if you actually did check the null value.
In languages with sum types, this cannot happen.
Either you get your hash, or no result at all.
You couldn't get eg NULL and *.
xorcist 1 days ago [-]
This argument is weak. C, and specifically libc, has errno. The error code is there. Not in band.
But there has to be something in the payload field when it is not applicable. A null is perfectly fine.
The argument for changing that to "*", if I understand the above linked discussion correctly, was that some users of this api threw away errors and a dummy value would be harder to ignore. Maybe one could argue that these behaviours are less common in Rust or Go, but not because of a technical difference.
The world is not perfect and sometimes you really need to work around bad users. Even in Rust or Go. I am not convinced this was such a case. If you are convinced it was it would still have been better to select a less trivial dummy value (such as "*:ERROR:*").
lemonwaterlime 3 hours ago [-]
In Haskell, one would return a Maybe or an Either here.
Maybe
——————
Just hash
Nothing
Either
——————————
Left err
Right hash
The program would be designed around not allowing this failure mode to even be possible.
forresthopkinsa 22 hours ago [-]
> But there has to be something in the payload field when it is not applicable
Not in languages with ergonomic sum types. A `Result` or `Either` will have no payload field on the error type. The compiler won't let you dereference it.
aidenn0 19 hours ago [-]
This is wrong. errno is not to be used to detect errors, only to get the cause of the error once an error has been detected.
jppittma 1 days ago [-]
C has unions. C has sum types. People have been doing tagged unions in C for longer than rust has been a language
creata 1 days ago [-]
It is extremely unidiomatic to use tagged unions in C for error handling. It's definitely possible, but awkward, and the compiler doesn't check that you checked the tag.
K0nserv 1 days ago [-]
I don't think I've ever seen a C function that uses unions for this purpose though?
I can think of two common patterns that I've seen frequently:
* Return NULL for errors (sometimes with a global errorno() function).
* Return an integer error with any return values as out parameters.
So if it is possible to improve, why doesn't anyone do it?
Woodi 12 hours ago [-]
Because usually NULL is good enough and you need to have domain use case to make library for it.
> * Return an integer error with any return values as out parameters.
You must first standarize all that (usually negative) integers, make docs, check for it, do not make mistakes in your if's or case's...
And errno is just that - integers with a meaning. errno is even thread local now and quite safe to use but people on internet scream loudly when someone try to use that not app global varible :) I guess their llms statistically have less new texts feeded in ;)
At some point it is just better to return pointer to struct with all info you can imagine.
And how you think other languages implemented their Error type returns ? Just a struct or union with tag. Because current CPU's operate on simple type registers there is no other sane way to do it.
zarzavat 6 hours ago [-]
That's a universe apart from real sum types with exhaustiveness checking and pattern matching.
aliceryhl 1 days ago [-]
Yet the authors of crypt did not use them. The C language is not designed in a way to encourage people to write code that correctly uses tagged unions (or nullable pointers for that matter).
knome 20 hours ago [-]
because POSIX dictates the shape of crypt from a function written in 1973 and unions/structs were only introduced to the C language in 1976.
the fault for sshhole isn't just "C bad", it's a series of poor choices in workarounds falling together to create a vulnerability.
being condescending about decisions made 50 years ago, before any of the better languages you're mad the developer didn't use even existed, is silly.
if you want to lay blame, lay it at the feet of more recent devs that started returning values instead of NULL because people were using crypt incorrectly. likely sha512 returned a fake value instead of a real one because prior hashes couldn't fail. the interface should have been updated then, but instead they worked around it.
jppittma 1 days ago [-]
Where are you going with those goal posts?
Philpax 1 days ago [-]
Nowhere. It's the same argument as upthread. A language that forces downstream consumers of an API to correctly deal with all possible outcomes would have prevented this.
rini17 1 days ago [-]
Goalposts are fine. If crypt did return union type, C still allows cowboy programmers to straight extract the char* without any check whether it's valid. So we'd end up in exactly the same situation.
cannonpalms 1 days ago [-]
I don't think the goal posts were moved. The comment you replied to referred to the "use"--not the existence--of sum types.
1 days ago [-]
dolmen 1 days ago [-]
Nope.
Because in fact all the changes were about changing the function signatures. The function signature is enforced by the linker. The semantic of strings that go through isn't.
Returning 2 values is possible in C (via pointers input parameters). The main issue here is that libc is a strong API contract based on C function signatures, that nobody can change, only augment with new symbols. Such strong API compatibility constraints happen in any programming language, and workaround to fit new behaviours in old APIs happens everywhere.
K0nserv 1 days ago [-]
I don't think nh2 is saying that this is how the "issue" should've been solved here, but that it's a fatal flaw of C itself (yes I know, unions, but no one uses them).
But also, its not like there were many alternative languages when the POSIX standards were designed so it's fair that it works like this.
aidenn0 19 hours ago [-]
Even C has a better type system than C: All modern C compilers support an extension that lets you flag a function's return-value as requiring a null check.
muvlon 1 days ago [-]
C's lack of all of
- sum/union types
- multiple return values/tuple types
- exceptions
means that most ways of returning an error from a function are not available in C. Because of this, many C APIs, including `crypt` in this case, resort to in-band error signalling by using a sentinel value (like NULL, a negative integer, "*" etc.).
Most other languages lack one or two of the above, but very few lack all three, and so most have developed a different culture around error reporting (Result in Rust, multiple return values in Go, exceptions in Java, Python etc.).
tialaramex 1 days ago [-]
One of the small but important significances of Rust is that it very often gets to have the same machine code as a correct "in-band error signalling" strategy from C but with source code that the maintainers can actually understand.
And really often when your "in-band error signalling" in Rust is more expensive you realise (perhaps to your horror) that your C wouldn't be correct, so you just dodged a bullet. For example Option<OwnedFd> is, like OwnedFd, a 32-bit value, Unix has always reserved -1 to mean invalid, so Rust can treat that as a niche. But the Windows equivalent Option<OwnedHandle> can't do this because, well, cue Raymond Chen: https://devblogs.microsoft.com/oldnewthing/20040302-00/?p=40...
creata 1 days ago [-]
> One of the small but important significances of Rust is that it very often gets to have the same machine code as a correct "in-band error signalling" strategy from C but with source code that the maintainers can actually understand.
But aside from NonZero- and NonNull-based layout optimizations, I don't think Rust is doing anything special to obtain that machine code - it's just relying on LLVM to optimize things.
tialaramex 8 hours ago [-]
Well, all of the niche optimisations, so that includes every normal C-style user-defined enumeration because (unlike C or C++) Rust says those aren't just the integers wearing a funny hat.
But sure, for the most part LLVM will do The Right Thing™ here without much help and Rust is just giving better ergonomics to the same backend.
Gibbon1 7 hours ago [-]
I remember mucking with returning multiple values as anonymous structs. It was tantalizing close but no banana.
I think with C23 you got half a banana now. Still can't do it because C23 didn't adopt proposals to allow returns of untagged/anonymous structs.
If you could then you could write
auto rtn = Foo();
if(rtn.err)
printf("err=%i", rtn.err);
else
printf("Foo=%i\n", rtn.foo);
sscaryterry 1 days ago [-]
The first 2 it has, and the 3rd, you really don't want.
C has sum types/unions? What are you talking about? You don't need multiple return. You can use a struct and it's the same thing.
adgjlsfhk1 1 days ago [-]
not in a way that forces the user to actually handle them properly. C is a weakly typed language and as such is pretty badly suited to systems programming.
uecker 1 days ago [-]
C is not a weakly typed language, this is just nonsense. There is also [[nodiscard]] which you can use to make sure a return value is handled.
muvlon 1 days ago [-]
What is a weakly typed language to you? To me, implicit type conversions are a big part of the definition, and C is flush with those (void* converts to any pointer, all kinds of number conversions, arrays decay to pointers, functions convert to pointers, lvalue conversion strips const/volatile etc.).
However, I also think "weakly typed" is not a well-defined concept, everyone seems to apply it differently. In particular, it seems to mostly mean "playing it loose with types in a way that I don't like".
uecker 1 days ago [-]
Yes, it is not well-defined. But most conversions, function calls, and accesses are strongly typed in C, so I would not call it weakly typed.
Implicit conversion could be seen as weak typing, but it is mostly limited to specific arithmetic operations in C. I would see implicit conversions as problematic if they allowed operations that make no sense. Decay of arrays and function to pointer conversion also do not seem problematic to me for this reason.
Void pointers are also weakly typed, but they are specifically designed for this. So if you do not want this behavior, one would simply not use them. It is a bit like using "unsafe" in Rust.
There are certainly stronger typed languages, but on the full range C is more on the strongly than weakly typed side in my opinion. A language that allows arbitrary implicit conversions between pointers, would be weakly typed, or which allows calling functions with wrongly typed arguments (which was true in the past for C a long time ago)
adgjlsfhk1 22 hours ago [-]
C is definitely not the most weakly typed (e.g. Javascript/PHP are both weaker), but IMO C is weaker than python in that python will give you a runtime type error if you do something invalid, while in C you cast something and just get UB. Most higher level strongly typed languages are able to make their casts checked because they have generics to remove the need for 99% of them, but C is missing those features and thus has to rely much more heavily on unchecked and unsafe runtime casts.
uecker 20 hours ago [-]
Casts are escape hatches just like "unsafe" in Rust. You can write safe generics in C that either do not use casts or wrap all unsafe operations behind type-safe interfaces.
muvlon 1 days ago [-]
C unions are sum types if you just "draw the rest of the fucking owl". C has sum types the way x86 assembly has function calls. You see, there are "call" and "ret" instructions. Just figure out calling convention and the stack yourself, it's easy. And yes I can stick the tag next to the union myself in C and write a bunch of functions that hopefully deal with it correctly. I can even write some macros that look like pattern matching and maybe convince myself that it's a language feature, but it just isn't.
And it's very evident why the difference matters. People gravitate to what's readily available and ergonomic. That's why every C API under the sun (most definitely including libc itself) does janky stuff like returning -1 or "*" on error instead of building their own bespoke Result-style abstraction. A programming language is not just the things that are in some way possible within it, but more importantly the things that are easy in it.
uecker 1 days ago [-]
While I agree that this should come out of the box, the "rest of the fucking owl" is a tiny macro or function, so not exactly something difficult to do complex as "drawing an owl" would imply.
dvdkon 24 hours ago [-]
Sure, when I last wrote a sizeable chunk of C, I wrote those macros and with a bit of help from new-ish compiler warnings, I got a reasonable error handling system in C. But whenever I interacted with anyone else's code, they apparently didn't get the memo that this is possible and desirable.
A language should encourage writing robust/secure code, not just make it possible.
uecker 23 hours ago [-]
While I see the point, I am not entirely sure I agree. I kind of like that C is not an opinionated language which tells you exactly how to write things. This comes at the cost that many people will do things in a bad way, and this is difficult to fix, but still probably easier than getting them to move to a different language.
In any case, I do not think the issue here is caused by not having a pre-defined result type. Checking for null would have been fine. The issue was that to accommodate old and broken code a compromise was made. But as long such compromises are done even where fixing such old code would be quite easy, it is clear that changing to another language is completely unrealistic. So what are we even talking about? The real issue is that nobody wants to spend money or effort maintaining the old code. No language can fix this.
muvlon 1 days ago [-]
There are different amounts of effort you can pour in to this depending on your requirements, and we can argue about those, but to me the evidence is clear: It's too much of a hassle for library authors to actually go and do it. Everyone just uses sentinel values instead.
1 days ago [-]
K0nserv 1 days ago [-]
For the other part of the issue (the truncation in SHA512), in Rust you could've had:
In any case, you can not change the API of an existing function. But returning NULL for error has required by POSIX would have been just fine.
dmantis 1 days ago [-]
Wouldn't Rust's Result or Go's error checking prevent that?
While not mandatory, a person would probably use it instead of the * hack.
Simply because the calling side wouldn't have an option to forget the check like with the NULL case
ninkendo 1 days ago [-]
Rust and go still make it pretty easy to not check results if you don’t care about the success value (lots of code just does `let _ = foo()`), but if you do want the value (like with a hashing function), Rust makes it really hard to not check the error too. There’s no syntax to say “assume this worked and give me the success value, including whatever garbage is there if it’s not set”:
- result.unwrap() will panic if it failed
- result.ok() throws away the error but gives you a None value instead (still can’t use it as if it’s a success)
- matching on the Ok variant and doing nothing on the Err variant, you only get to use the success value inside the Ok arm
Etc etc.
The problem is when the compiler says “NULL is a valid instance of T”, which go lets you do, and rust doesn’t (without using the unsafe keyword and intentionally casting a null pointer, which you don’t do by accident.) Go lets you write code like `var hash string; hash, _ = hashfn()` and it compiles and now you have null instead of a valid hash.
65a 4 hours ago [-]
Minor nit, strings are never nil in Go, so that would be "" instead. Usually nil is only a valid instance of pointer-to-T. The point stands though, a common pattern is "defer foo.Close()", the Close() signature almost always returns an error and almost no one checks it...which is probably fine, but not automatically fine.
tialaramex 1 days ago [-]
> NULL is a valid instance of T
This by the way is Tony Hoare's famous "Billion Dollar Mistake"
uecker 1 days ago [-]
A NULL pointer for error would be a safe interface where you get a trap when there is no value.
f1shy 1 days ago [-]
I understand that other languages could help avoiding the semi-predicate problem. That being said, in C there are better ways around semi-predicates, as inelegant as they may be, is possble, is not an inherent problem of the language, but may a bad habit/idiom of it.
randomNumber7 1 days ago [-]
Do you work in industry or academia?
01HNNWZ0MV43FF 1 days ago [-]
There is no industry, it's just people
WesolyKubeczek 1 days ago [-]
You must be new here. While I applauded the idea of musl when it first came out (coincidentally, glibc wasn't having the best time in its development then), the developer's arguments why <some behavior drastically different from every libc under the sun> was akchyually correct, which was why you had to fix a program that hadn't been broken for 30 years and now suddenly was, made me put him under the "clowns" category and never look at musl seriously again.
When later it turned out that musl's procrustian stack size made Python not really runnable under it, it cemented my resolve to not go near musl.
And now we have this.
acdha 1 days ago [-]
I’m also reminded of the clowns pushing “it’s smaller!” container images based on social media vibes rather than analysis where, yes, not using Debian+glibc saved tens of megabytes on the base OS but did so at the expense of measurable performance reductions & compatibility issues like the one you mentioned, and literally every single one of those dudes I worked with was shipping containers pushing a gigabyte so the size reduction had no measurable impact on deployments.
NewJazz 1 days ago [-]
Uh, isn't this the case of the opposite? The musl devs trying to accomodate broken programs?
throwawayffffas 1 days ago [-]
I am sorry, a little? I am deeply disappointed that libc decided to do the same just with another sequence.
And why because some dude was complaining sshd or whatever was crashing because it didn't handle nulls? Just handle the null my guy you already know why your bullshit is crashing.
IshKebab 24 hours ago [-]
Let's be real here, the main culprit is C. People love to say "but you can still get logic errors with Rust", and no shit of course you can. But it's much less likely. You wouldn't have gotten this bug.
semiquaver 1 days ago [-]
A little?!? I’m sorry, but musl is a joke, this is yet another proof of that fact, and I have no idea why anyone uses it in production, they have only themselves to blame after the many warning signs.
wowczarek 1 days ago [-]
In my last org I was in charge of internal testing and qualifying software releases for network equipment, and designing test harnesses and tests for these. Significant install base. All the major vendors. Many years. Basically doing the vendor QA for them, because we couldn't afford things to fail. As you do - I wish.
This was not security testing as such - mostly latency, performance and hardcore network "can we break it" and "does it comply" stuff, but also a lot of cable and module / line card pulls, pluggable and fiber breaks, random reboots and pre-empting a lot of possible operator inventiveness. At one point we considered SEU bit flip testing on rented cyclotron time.
And always we battled against the top brass urging us to speed up the process and don't bother with "really basic" testing, because we observed software quality going down over the years as the push for release and time to market stared to dominate.
So we always started with sanity testing - and that included oversized password and CLI input strings. Always. And we caught things more than once, but never like this.
We were right after all.
Kuraj 20 hours ago [-]
> We were right after all.
Exhilarating, isn't it?
wowczarek 20 hours ago [-]
Hardly. I say this with a faint, bored "huh" note rather than with a triumphant "I told you so", and I move on.
It's about maintaining solid engineering practices and far above proving who's right or wrong.
dabinat 21 hours ago [-]
This is why I don’t fully buy the argument that software that’s been around for a long time is more secure and reliable because it’s been battle-tested. There’s been a lot of time for cruft to accumulate until no-one can ever fix it because everyone is expecting it to function in the broken manner it’s always functioned. And some of the more obscure configuration options get a lot less scrutiny.
procaryote 19 hours ago [-]
Musl, which is vulnerable, is much younger and less battle tested than glibc, which is not vulnerable though
aidenn0 19 hours ago [-]
This is not an obscure configuration option. Your password file almost certainly has at least one account with "*" as the password hash.
Integration of different software is the stage most likely to introduce uncaught bugs. That is what happened here: musl intentionally broke the documented API of a function used by OpenSSH. Then Alpine integrated this incorrect implementation with OpenSSH.
sidneythekidney 1 days ago [-]
Fun!
I was able to reproduce this on Alpine 3.23.7 with OpenSSH_10.2p1.
Luckily PermitRootLogin [0] hasn't allowed logins with passwords for root since 2015 [1]. I'm guessing that's at least part of why this went undiscovered for so long.
Did anyone get it to work? I tried on the official alpine docker image, enabled password login for root. The shadow entry has ! as password by default, this didn't work with a > 256 character password (i also did passwd -l which also does !). I manually changed it to * and it still didn't work.
Am I doing anything wrong here?
OpenSSH_10.3p1, OpenSSL 3.5.9 29 Sep 2026
PRETTY_NAME="Alpine Linux v3.24"
markuman123 1 days ago [-]
It's because alpine has updated all openssh-server packages already. Even from EOL alpine versions.
bonyt 1 days ago [-]
Ahh! Wasn't working on 3.16, which does indeed have an updated package.
But, got it to work on 3.15 - which requires changing the ! to a * in the shadow file in addition to other steps (making another user with a hash, setting PermitRootLogin to yes, etc.).
Didn't notice that the openssh-server-10.3_p1-r2 was released today.
atmosx 1 days ago [-]
Nope, wasn’t able to reproduced it in my firecracker VM, which isn’t publicly accessible anyway.
f311a 1 days ago [-]
As far as I understand, it wont work on default setups.
You need musl-c, no pam and openssh.
zamadatix 1 days ago [-]
In Alpine, these would all be the case for an out of the box install with openssh added unless you specifically install openssh-server-pam and configure sshd_config to use PAM (Alpine is already musl-c based and doesn't default to PAM).
Aachen 1 days ago [-]
Doesn't it also need a user to be added with a password entry?
I only ever see Alpine Linux used for containers which don't need accounts with interactive user logins, which is not to say it doesn't exist, just that it doesn't seem like Alpine comes with all preconditions met out of the box
zamadatix 1 days ago [-]
Isn't it the opposite, accounts which have no password and hence had * instead of a hash?
> accounts that should never take a password at all (root, your key-only deploy user, ldap users, the default user on alpine's aws and azure images) take any password longer than 256 characters.
But yeah, Alpine for persistant installs isn't unheard of - just not as popular as the container use case. They offer official VM, Xen, Pi, and bare metal images users deploy as well. I actually had a fleet of VMs to update!
Edit: The Alpine notice made it clearer to me that at least one other account has to have a password, it's just the target account that doesn't. So if you truly have no users with passwords in your container deployment then those passwordless users aren't vulnerable to this bug, but any of the image types which have a user will likely have root exposed.
> The attacker needs the name of an account whose password field is * (an account with no password set, which is normal for key-only accounts), and needs the machine to have at least one other account with a $5$ or $6$ password hash.
SSHing into test@ with one 'wrong' password then one 'long' password doesn't work. I have UsePAM no and PasswordAuthentication yes in my sshd_config.
1 days ago [-]
hagbard_c 1 days ago [-]
No, didn't work for me open OpenWRT x64 running in a container but that is probably 'cause I'm using dropbear instead of openssh-server. That's what OpenWRT uses by default so the impact there will be minimal assuming that most installations don't change to openssh-server - it isn't installed by default.
necovek 20 hours ago [-]
On top of OpenWRT on my router (running 25.12) running dropbear instead of openssh-server, I also do not have any accounts in /etc/passwd|/etc/shadow with "*" for a password hash and an actual shell (all are set to /bin/false).
emayljames 1 days ago [-]
Did you try one incorrect standard password fail first?
justinsaccount 1 days ago [-]
I can't reproduce it either. I used alpine:3.24.1 and openssh is openssh-server-10.3_p1-r2
I tried adding PermitRootLogin yes and PasswordAuthentication yes. Tried adding another user with a valid password.
Verified shadow entry has a *. Can't login as root or a test user. Tried another incorrect password first as well.
edit: can reproduce after `git revert 384e25e6bd1f2e7b30095900e48b570d46953510` in aports and rebuilding openssh without the fix. It also accepts the password on the 2nd+ attempt, NOT the first.
1 days ago [-]
ryanw02 1 days ago [-]
[dead]
so_it_be 1 days ago [-]
[dead]
wolrah 1 days ago [-]
Also how many times do we have to learn that letting badly programmed software die when unexpected things happen is the better option because it means people actually notice that there's a problem. Silent compatibility hacks are the worst, they guarantee problems never get fixed and the stack of hacks never gets smaller.
Microsoft has historically been one of the leaders on the wrong side of this one, but they have also demonstrated how effective it is to call out badly behaved software and make it annoy the users with UAC. When Vista came out, so many applications needed elevated privileges to run because they were badly designed to write user data in system locations, but now their application was more annoying to use and came with a scary dialog about it making changes to your computer that other better-designed software didn't get.
More often than not if a program didn't actually intend to change system settings it was just a matter of badly chosen default paths which are pretty trivial to fix, but so many vendors just would not do it despite years of complaints from NT admins wanting to not have to spend hours with Process Explorer figuring out specific permissions to grant limited users enough access. Once their failures were put front and center, calling their application out as the one getting things wrong every time the user ran it, they suddenly had motivation to fix it.
simias 1 days ago [-]
For security it's better, but in many cases if you're not writing security-oriented software it can be worse from a user perspective. I remember the first job I had over 20 years ago was working on VoD streaming and one of the "tricks" to improve stability was mmap'ing the 0 page so that NULL dereferences wouldn't immediately segfault. Ugly as hell but when you were working with all sorts of more or less well coded media codecs and security wasn't a concern at all (worst case scenario the user would stream movies without paying) it made some amount of sense.
I suspect that Microsoft has the same history: the entire Windows platform was built with single-user PCs in mind, mostly only dealing with trusted input. The users were also expected to be tech-illiterate for the most part and unable to troubleshoot deep technical issues.
More recently I'm still working on video transmission, but on the production side this time. I also originally adopted the mentality that I should crash early if some unexpected situation was encountered, but I ended up walking that back a lot, because having a critical piece of live media production reboot mid FIFA worldcup final is generally Not Great. Degraded functionality is usually preferable. I still crash early in dev builds, but the production code just logs an error and continues if at all possible.
That said, I definitely agree with you for something like password handling. In this case, I do think it's warranted to crash the program without survivors if something is amiss.
pmontra 1 days ago [-]
> if you're not writing security-oriented software
It's hard to define what's security oriented and what is not. For example, is this HN web app security oriented? Not in the way of sshd but it must keep our accounts data separated and it must not let people exfiltrate data from the db, where for example it is storing the emails of some of us.
funcDropShadow 1 days ago [-]
It depends on the time. Twenty or thirty years most software was not security-oriented. Nowadays, everything is online and therefore everything needs to be vigilant about security.
pixl97 1 days ago [-]
> because having a critical piece of live media production reboot mid
It's not great, but these days a trickster with AI will find some way to start broadcasting porn to a billion people because of the flaw, so may end up being the better option.
whoanoway 1 days ago [-]
[dead]
stackghost 1 days ago [-]
> because having a critical piece of live media production reboot mid FIFA worldcup final is generally Not Great.
Story time?
simias 20 hours ago [-]
There's not much else to say honestly, for these very important events there will usually be several layers of redundancy for the most critical feeds, so the production is always able to send something to the satellite uplink even while our devices reboot due to a crash, but brutally losing 90% of your inputs for ~1 minute while the machine reboots is not a pleasant experience for anyone...
pizlonator 1 days ago [-]
> Also how many times do we have to learn that letting badly programmed software die when unexpected things happen is the better option because it means people actually notice that there's a problem.
100%
zahlman 1 days ago [-]
Agreed. I feel like this attitude has been holding Python back, too. It's one thing to criticize "move fast and break things", but it's also not good to ignore already existing breakage just because the pieces still fit together neatly, and it's not good to refuse to move at all.
0xbadcafebee 1 days ago [-]
> letting badly programmed software die when unexpected things happen
Go ahead and program a flight controller this way, I'm sure nothing bad will happen.
dvdkon 24 hours ago [-]
I'd still rather have the software crash and alert a human that to continue in a faulty state. The way to add robustness in this case is to add backup systems, not "ON ERROR RESUME NEXT".
21 hours ago [-]
bottlepalm 13 hours ago [-]
> badly programmed software
Is there a test or check I can use to determine that software is 'good'?
Oh there isn't? Good/bad is subjective? We need to scape goating hacks as bad software, as if anyone is capable of perfect code. Perfect code would require knowing the system inside and out, the stack up and down - no one knows all that, no one is perfect. Sufficiently advanced AI though can rip through all of that, find the weakest spot and zero day it, bringing the entire house of cards down.
Software you thought was good can still be hacked. There is no way to create provably unhackable software.
Which means all software is packed with vulnerabilities just waiting to be discovered, no matter how good you think it is.
drdexebtjl 1 days ago [-]
Now everything installs to a user local folder, runs as the user, there are no UAC prompts, and malware can just freely tamper with other applications, act as the user, and harvest all data the user has access to.
Hey, but at least it can’t touch the System32 folder.
Great job guys.
hnfong 1 days ago [-]
> Microsoft has historically been one of the leaders on the wrong side of this one
Objective-C ahem nil messaging...
sscaryterry 1 days ago [-]
Objective-C is an Apple thingie :)
hnfong 1 days ago [-]
Yes and thus I dispute that Microsoft leads on this front :P
usefulcat 1 days ago [-]
By way of NeXTSTEP
SoftTalker 1 days ago [-]
NeXT certainly used it extensively but they didn't invent it. Originally developed by Brad Cox and Tom Love in the early 1980s, per !w.
quotemstr 1 days ago [-]
> Also how many times do we have to learn that letting badly programmed software die when unexpected things happen is the better option
Sadly, we will never learn this because failing fast runs counter to intuition, and a class of person will never override it.
1 days ago [-]
zicixisi 1 days ago [-]
[flagged]
evgpbfhnr 1 days ago [-]
So to recap, with a non-upgraded sshd you need:
- an account with /etc/shadow hash set to '*'
- (note root won't work by default because of PermitRootLogin prohibit-password default)
- attempt twice, first attempt fails second long password is accepted (that took me a while to understand)
In practice passwd -l seems to lock with a ! prefix, and new account without passwords are created with a single ! which is also not affected, I'm not sure how big the impact actually will be in practice and don't understand the recommendation to upgrade urgently right now... Just don't use `*`?
EDIT: ah, some default container images apparently have a default alpine user with that hash, so I guess that's bad for them. This makes a bit more sense now.
jack1142 24 hours ago [-]
> some default container images apparently have a default alpine user with that hash
Do you have an example? Looking at Alpine's advisory, I can only see that their cloud images (i.e. what you use to run a VM, not a container) have a default alpine user, no container images. And even with those, cloud images with cloud-init are unaffected with default config due to `PasswordAuthentication no` being inserted by cloud-init into sshd_config. So that leaves... Cloud images that use Tiny Cloud for provisioning. That's an instance bootstrapper authored by Alpine from the looks of it so I'm not sure how many people would prefer that over cloud-init (which a lot of VM/bare metal management software may already make easy to use) or just a non-cloud image.
cowlevel 1 days ago [-]
Why do you need to attempt twice?
babagan0ush 1 days ago [-]
> Just don't use `*`
Well but, alas, a non-zero number of user probably do use it, though and "upgrade X" is a clearer and better instruction to communicate than "change this setting where maybe you don't even know what it means and it might break your workflow, but saves you an upgrade".
evgpbfhnr 1 days ago [-]
In some contexts (e.g. embedded) you'd be surprised at how reluctant some people are to upgrade when they can run through a dozen of hoops for workarounds such as changing hash instead...
1 days ago [-]
0812037123158 1 days ago [-]
It's amazing how similar this exploit payload is to the 20+ year-old Quake 3 vulnerabilities found by Luigi Auriemma (aluigi), which also used the exact same 256-to-1024 character range and nothing more than a long string of AAAA.... Here you're feeding >256 A's up to a 1024-byte limit to trip musl's defensive length cap, whereas back in the mid-2000s, Luigi was famous for publishing one-line .cfg scripts where the entire exploit was literally "say aaaaa..." repeated a few hundred times overflowing server's memory.¹ ²
A great example is Jedi Academy, a 2003 Quake 3 engine game whose multiplayer is still actively played today (its open-source engine port, openjk, is even in the Debian repos btw). Luigi had two classic scripts for it on his site: Jedi Academy <= 1.011 in-game server buffer-overflow (jamsgbof)¹, which crashed the server just by pushing a string of as past the 1024-byte console buffer, and Quake 3 engine in-game players kicking (q3msgboom)², where a wall of as hit the engine's 1022-char truncation limit and the leftover bytes confused the parser into disconnecting everyone on the server.
Kind of wild that 20 years later, the exact same payload goes from crashing Quake 3 matches from the tilde console to getting root on some SSH servers.
Offtopic: this is a very cool way to present commits and code excerpts. Love classic Windows styling.
Were those pictures with links generated using a publicly available tool that we could use, or were it custom (vibe)coded specifically to create this page?
dpacmittal 1 days ago [-]
This looks entirely vibe-coded to me. No one is going to put so much effort for a vuln.
chris_wot 1 days ago [-]
To be honest, I love this sort of vibe-coding. Explains it really well, in an interesting way. If this is vibe-coding, then the AI has done a great job.
marcelox86 1 days ago [-]
It was fun to see this type of style around the release of Opus 4.6, but its really tired now. I've seen countless landing pages like this...its basically ritual
bigstrat2003 23 hours ago [-]
There's nothing great about this site. It should give you a succinct summary of the vulnerability right up front, so you can determine if it applies to you. The history lesson is not relevant to the vast majority of visitors.
etatester 12 hours ago [-]
Yes but it's not your average vibe coded site, it's still decently done and clearly art-directed by a human rather than "make cool website for me plz"
grumpymuppet 5 hours ago [-]
This kind of thing is a constant worry of mine, but I've never had the (mis?)fortune to look at this particular code.
Signatures/verification (typically) only provide a boolean guard on what you want to be controlled behavior.
To properly protect against cla controlled behavior, the check must be incorporated into the EXECUTION of the behavior (with a safe fail).
This is hard and I think a lot of pentesters will continue to have lucrative work hunting issues like this down for a long time.
Fundamentally, the problem is the "type" of the verification and the execution.
dev_l1x_be 1 days ago [-]
From the advisory: musl’s crypt() returns the string * as an error indicator when the key is over 256 bytes long
Are these people mad? Why on Earth would anybody indicate error like this?
21 hours ago [-]
23 hours ago [-]
neuroticnews25 1 days ago [-]
That's a well-made site, yet I feel nothing.
oneeyedpigeon 1 days ago [-]
I thought it was a great write up. It did feel a bit AI-coded, but whether it actually was or not, it's still a good bit of work imo.
phyzome 1 days ago [-]
I dunno, there were some strange discrepancies in the explanation (and captions). Didn't really hang together.
user43928 1 days ago [-]
Looks good overall, but I was wondering if eg. "the timer learns nothing" and "no X, no Y, no Z" was slop rather than a deliberate choice.
fwip 23 hours ago [-]
Slop, no question. There's at least 10 Claudisms on the page, and I'm not that good at smelling them. Some of the sentences might have been human-written.
aphysically 16 hours ago [-]
fwiw pangram believes the text is AI as well
xnorswap 1 days ago [-]
It would be weird to spend this much effort doing the site by hand, for a bug that the author says was found by a swarm of agents.
cpach 1 days ago [-]
LLM output fatigue?
amelius 1 days ago [-]
Apple sideways traffic light fatigue.
1 days ago [-]
hagbard_c 1 days ago [-]
Good to know I'm not the only one annoyed by that superfluous piece of ${whatever} signalling.
riffraff 1 days ago [-]
I didn't hate the writing even if it's AI, but I deeply disliked the animations. I don't know if that's LLM fatigue or not.
gblargg 19 hours ago [-]
I left after trying to scroll to a concise description of the issue. I don't want a theatrical presentation.
mrguyorama 1 days ago [-]
It has a crap load of text but says very little and is very bad at conveying the details because it thinks it has to tell some weird story.
marcelox86 1 days ago [-]
I had this classmate that had their ChatGPT instructions to respond in this same laidback, lowercase, gen-z style. i recognized the output as a lot of llm writing almost immediately
efilife 14 hours ago [-]
Yes. Reading this feels like a LLM pretending to write carelessly. And failing miserably
tasuki 1 days ago [-]
Maybe that's on you though. I enjoyed it, though the discrepancy between the `*sshole` name and the `sshhole.com` domain made me twitch. How is anyone[0] obsessive enough to find this and write it up and yet not obsessive enough to make the amount of `h` consistent?
[0]: Be they human or LLM. Or an entity with any proportion of the two. I do not judge nor discriminate.
glibg10b 1 days ago [-]
Look closely. The second h is behind the word and almost as dark as the background
QuantumNomad_ 11 hours ago [-]
Yes, but if you select the text and copy paste it it comes out with a single h, and also the <title> of the page has it with a single h and a double h.
<title>*sshole · sshhole.com</title>
Strange and confusing.
I mean I understand that in “*sshole” it’s referencing the * that is central to the issue and putting it so it looks like a censored version of “asshole” while also referencing that it’s an ssh hole. And the domain is more simply ssh hole instead of sshole. Is a way maybe it does makes sense to do both. But it did seem at least at first strange to me that the domain is one thing and the name is a slightly different but almost same thing. Like there was a typo when registering the domain, or like the author changed their mind about the name slightly after registering the domain.
tasuki 1 days ago [-]
Oh. Thank you!
Levitating 22 hours ago [-]
It has very low information density. This would read 10x if it were just a few paragraphs of text. Instead we need to render an animation every few sentences. Not a fan.
jonstewart 1 days ago [-]
What is dead can never die. All blueteamers are ironborn these days.
bonplan23 1 days ago [-]
I do actually. My guess is that this was not just one-prompted, but someone took the time to refine it based on their tastes and expectations until it matched something they would actually like.
However, if someone already developed a strong AI-slop-tingle it will still be triggered by some telltale signs and let their subconsciousness know that they can't possible enjoy this.
BeetleB 22 hours ago [-]
I think at some point I'm going to get banned from HN for repeatedly saying this, but here we go:
This thread highlights why I often say that at some point in the future, the obsession with whether something was written by AI or not will be classified as a mental disorder.
It is tiring to virtually every day see this discussion play out in some HN thread or other. It's also just noise. I don't think your comment, nor the thread it inspired, added anything of value to the discussion.
0x0 1 days ago [-]
For OpenWRT, AsusWRT etc, I believe they use dropbear ssh by default instead of OpenSSH, which apparently isn't affected?
dhx 1 days ago [-]
Yes, correct for OpenWrt. OpenWrt defaults to dropbear for both server and client. However, more advanced users may optionally choose to replace it with OpenSSH, or install OpenSSH side-by-side, to gain additional features of OpenSSH.
Dropbear added the 100 character password limit in 2019.77, so earlier versions might be susceptible.
mkj 15 hours ago [-]
Nope, they are OK.
RVuRnvbM2e 1 days ago [-]
Wait so musl, famously a stickler for standards, just changed the behaviour of a POSIX function?
traceroute66 1 days ago [-]
To be fair, I have less than zero sympathy for anyone who runs SSH on any platform without `PasswordAuthentication=no` in 2026.
Its not asking for trouble, it is there will be trouble, not if but when.
As for the people who say "well, I run SSH on port 942329053052". Well guess what, the bots make short work on that. We tried it at $work as an experiment, multiple bots found it in less than a couple of hours. That's why your SSH config must be hardened, preferably running on a hardened OS too.
dwedge 1 days ago [-]
> Its not asking for trouble, it is there will be trouble, not if but when.
Did any of those bots brute force a password? Because a 20 character random password is comparable in brute force complexity to an SSH key. The experiment you ran was just as valid for SSH keys on that port, and had no bearing on password vs key.
traceroute66 1 days ago [-]
> Did any of those bots brute force a password?
The point is they found the security by obscurity so-called "hidden" port in less than a couple of hours.
At that point, what they did is irrelevant. The point is that security by obscurity does not work in 2026.
> Because a 20 character random password is comparable in brute force complexity to an SSH key.
You can't be serious with your comment.
No its not.
And even then, SSH key is asymmetric, you're not sending the private key to the server. But with a password you are, which leaves you vulnerable to many attack vectors that simply do not exist with keys.
minitech 1 days ago [-]
Other points are fine, but why do you say a 20-character random password isn’t comparable in brute force complexity to an SSH key?
TZubiri 1 days ago [-]
> The point is that security by obscurity does not work in 2026.
I feel the opposite, note how most all AI vulns discovered use source code level analysis, empirically keeping source code hidden is more secure.
kerchoff’s principle is misconstrued by a voyeuristic internet that wants to see source code
Ardon 1 days ago [-]
This is a sampling error - we're only hearing about the public disclosures.
(almost) No company actually has secret source code, only source code not publicly viewable. And the LLMs crawling those don't belong to people who make public disclosures.
ShinyLeftPad 17 hours ago [-]
> LLMs crawling those don't belong to people who make public disclosures
Using LLMs is a great way to empower the governments behind them (US and PRC) to cultivate a garden of undisclosed bugs in your closed commercial codebases.
Aachen 1 days ago [-]
> > Because a 20 character random password is comparable in brute force complexity to an SSH key.
> You can't be serious with your comment. \n\n No its not.
If you differ in opinion, maybe provide evidence? Especially for a claim like long randomly generated value 1 being absurdly ("can't be serious"-level) weak as compared to long randomly generated value 2
If you mean it's log(62^20,2)≈119 bits of entropy instead of 128+ and so it's theoretically weaker when we build a Dyson sphere around the sun to brute force that key (after somehow obtaining the shadow file, since an online attack would be way too slow), then sure
Ecco 1 days ago [-]
How is password auth weaker than key auth? Bugs in auth code can happen in both pathways. And apart from stupidly short passwords both should be equally impossible to brute-force.
In other words, conceptually, at the end if the day, isn’t a key just another name for a password, really?
minitech 1 days ago [-]
Password gets sent to the server. Private key does not get sent to the server, and authentication protocol can be forwarded safely, enabling things like ssh-agent, FIDO SSH keys. This is solvable with passwords (see: augmented PAKE), but `PasswordAuthentication` isn’t that. But there are also settings where none of this really matters, yes, to be fair if that’s what you’re referring to.
SSH certificates are cool too.
TZubiri 1 days ago [-]
password doesn’t get sent to the server, hash does.
but I guess that the hash is always the same unlike with pub priv key mechanism.
mkj 1 days ago [-]
No. The password gets sent to the server. That said, I still enable passwords on servers, but don't ever share passwords between trust domains.
traceroute66 1 days ago [-]
> password doesn’t get sent to the server, hash does.
Kind of irrelevant what gets sent to the server and what happens on the server.
Password is already easily defeatable long before it even leaves your computer, thanks to keyloggers and/or password manager breaches and/or old-school shoulder surfing (assisted in the 2026 era by covert filming and AI).
All of which are not vulnerabilities with hardware-backed keys which you don't even need a Yubikey for these days since most modern computers and smartphones have secure enclaves and/or TPMs. (Yubikeys are still great though).
Borealid 1 days ago [-]
What people think of as a very strong password often has <100 "security bits" of entropy.
Keys are usually at least 128 bits of entropy, and it's easy to take them up to 256.
Yes, both are a shared secret, but password auth - even with challenge-response - is weaker than key auth exactly because key auth provides more guess resistance.
Elucalidavah 1 days ago [-]
But the SSH keys are based on asymmetric crypto, not shared secrets.
dagss 21 hours ago [-]
That makes it impossible to steal the key if you MITM, but there is still the same chance of guessing it. (Assuming password has as many bits of pure randomness, such as machine generated password.)
traceroute66 1 days ago [-]
> What people think of as a very strong password often has <100 "security bits" of entropy.
The best password in the world will still be defeated by a simple keylogger or breach of your password manager.
scotty79 1 days ago [-]
2^100 is a lot of things to try though
jandrese 1 days ago [-]
Especially if you have Fail2Ban set up.
traceroute66 1 days ago [-]
> if you have Fail2Ban set up.
Worth noting that if you are running a remotely recent version of SSH it has everything you need built-in. No need for Fail2Ban on SSH these days if your config is fully modernised.
Borealid 4 hours ago [-]
Could you maybe explain a bit more about how one sets up OpenSSH to block brute-force login attempts with only its built-in features?
belorn 1 days ago [-]
Fail2Ban is kind of required unless one like running out of sockets and are unburden by high cpu and disk load.
jandrese 1 days ago [-]
There is regular bot traffic attacking SSH, but it's not "saturate your downlink" level of traffic. If you're getting attacked that heavily then you're being specifically targeted.
Aachen 1 days ago [-]
Huh? I run ssh (OpenSSH) on port 22 with root login enabled and no fail2ban on a single-core laptop cpu from fifteen years ago. The load average is 0.00 even with the small mail and web server that runs on it
What cpu and especially what disk load are you talking about? Also not running out of sockets, not sure if that's RAM-dependent (the box has 2GB)
teddyh 9 hours ago [-]
I believe this depends on your exposure level. If you have one, or maybe no, domain names pointed at your server, you are probably fine. If you have tens, or hundreds, of domains pointed at your server (i.e. you are a web site hoster), you will probably need something like fail2ban.
Aachen 7 hours ago [-]
On another server with the same setup, there's various services and websites running, so load average is never zero there (due to scrapers) but ssh has yet to catch my eye in htop. Whenever I've looked, even the amount of cpu that a Windows VM uses constantly while idle is many times higher than what ssh uses
belorn 15 hours ago [-]
Working as a sysadmin, and with disk encryption the constant log writes generated from ssh was enough to cause heavy load on severs. Long lasting connections from bots also forced us to increase the maximum number of sockets, as some scan waves was enough to hit the maximum and block other connections. Fail2ban has mitigated those issues to the point where neither has been an issue.
epihelix 14 hours ago [-]
> How is password auth weaker than key auth?
Well, it's a heck of a lot weaker in this instance (at least when `PasswordAuthentication no`)
8organicbits 1 days ago [-]
Password reuse is the problem. The password hash is stolen from some other site and brute forced offline.
dwedge 1 days ago [-]
Same issue with key reuse. Unless you have a policy to rotate them and verify which host they login from, someone's unencrypted hard drive from 5 years ago is more likely to have a key than password.
traceroute66 1 days ago [-]
> someone's unencrypted hard drive from 5 years ago is more likely to have a key than password.
Even 5+ years ago, hardware-backed SSH keys were a thing (Yubikey). The only difference today is that its 100% easier to do keys properly because of the widespread commoditisation of secure enclaves in desktops and smartphones to support FIDO, its basically free of charge now because you're not obliged to buy a Yubikey.
ant0ni 1 days ago [-]
you claimed my favorite game's character name!
1 days ago [-]
ghm2180 1 days ago [-]
100% and with local tool calls many AI systems will do ssh "for you" and end run around any security theatre that you may have going on with LLMs.
kccqzy 1 days ago [-]
I run SSH with password authentication on but the port binds to a ULA address. It’s fine. No need to be so absolutist.
traceroute66 1 days ago [-]
> No need to be so absolutist.
Why not ?
Even if you don't want to go all the way to SSH certificates, SSH public-key authentication is laughably easy to do.
I also don't buy the argument "but its on a ULA address". Attacker's old favourite these days is finding a way in (North–South) and then using that as a stepping stone (East–West). Also the general concept of "lower security is OK because its on the LAN" is so 1997.
kccqzy 1 days ago [-]
Yeah because if attackers are already finding a way in, they can easily read my ~/.ssh/id_ed25519. A password is strictly better, although not by much because lateral movement is very easy anyways. The only exception is if the private key is actually on a HSM, which is the case for all my SSH private keys on iOS.
cozzyd 22 hours ago [-]
or password protect your key, at least.
traceroute66 1 days ago [-]
> read my ~/.ssh/id_ed25519.
No excuse for doing that these days.
You can use a Yubikey or you can use your computer's secure enclave or TPM with FIDO at no extra cost.
A password is never, ever "strictly better". Seriously, you're just making shit up now.
kccqzy 1 days ago [-]
Your comment is self-contradictory. In one sentence you say “these days” and in another you say “never, ever.” These are contradictory. What if your computer is so old that it doesn’t have a TPM?
I was already pointing out that if you have secure enclave for your private key you should use key-based authentication. That’s what I do on iOS. Otherwise a password is better than your private key in plain text on disk. So much malware steal your private SSH keys (along with your other saved passwords and browser cookies); no malware that I know of performs social engineering and tricks you into giving up your SSH passwords.
tarxvf 3 hours ago [-]
N=2 but non-22 ssh ports get almost zero traffic for me at home and at work.
Not zero - but very nearly zero.
shrug.
cozzyd 22 hours ago [-]
Running on that port would be perfectly safe as it's outside the range of valid port numbers.
znnajdla 1 days ago [-]
This particular bug works even with password authentication turned off.
sjamaan 1 days ago [-]
Where did you get that idea? The entire bug is about password hashing. With password auth off, that's not even attempted.
kijin 1 days ago [-]
The exploit works for a user account that has no password. This is often confused with turning off password auth, especially in containers that have only one usable account.
It's not just the parent commenter. I'm sure somebody out there is going to remove their account password instead of editing their sshd_config in an attempt to "patch" this vulnerability, actually exacerbating the problem.
tialaramex 1 days ago [-]
Bug > are password logins still on? they are by default
You > This particular bug works even with password authentication turned off.
No. The bug is related to password authentication. If you do not have password authentication then you are not affected.
What freaks some people out is they chose parameters where they do have password authentication but they expect to always fail. That's a bad choice, like OK you did fit a cheap manual keyhole, but you threw the keys away thinking now nobody can open that - you realise somebody with a cheap jiggler can open it right ?
znnajdla 1 days ago [-]
read the article again
traceroute66 1 days ago [-]
> This particular bug works even with password authentication turned off.
Does it ?
Somebody here has already said "You can set PasswordAuthentication=no or update to remediate"
We don't use musl at $work anyway, so I've no means to test.
unglaublich 1 days ago [-]
I mean there are 65k possible ports so it's the equivalent of guessing an 16 bit password, i.e. two (extended) ascii characters. It's not very surprising that 'multiple bots' found it: a single `nmap` command will give you the open ports in a few seconds.
bawolff 24 hours ago [-]
Its kind of cool to see such simple to follow logic bugs instead of usual memory safety issues. Each step made sense in isolation.
kibwen 23 hours ago [-]
The terrifying decision for the API to return "*" on failure is a second-order consequence of memory unsafety, because it was a half-baked attempt to mitigate a common source of UB that people were encountering in the wild.
bawolff 23 hours ago [-]
i'd say it is more a consequence of not having exceptions or other way of being explicit about error values.
kibwen 22 hours ago [-]
That was a contributing factor, but there are plenty of other instances of C APIs that make use of in-band signaling like this. That the API was changed from returning one in-band signal to another is because the original value was causing programs to crash unpredictably due to UB.
hypfer 1 days ago [-]
What is out there that uses musl without PAM and hosts an OpenSSH SSHd with PasswordAuthentication?
As in: "Actually out there"
The hypothetical Alpine exists, yes, but what runs that like that?
Bender 1 days ago [-]
What is out there that uses musl without PAM and hosts an OpenSSH SSHd with PasswordAuthentication?
Possibly public SFTP servers. I have allowed null passwords in the past for anonymous public SFTP and I run Alpine Linux exclusively for my servers both VM's and physical. The shadow package (PAM) did not get installed by default. I installed it but I had to specify it in the installation so there are likely people without it. There is probably a way to find them on shodan.io.
I will activate one for people to play around with for today:
sftp hn@ping.ohftp.org
There's no firewall and no password. Hack away HN.
Hopefully some had a chance to have some fun. I moved sshd back behind haproxy. There's still no firewall but ssh should be invisible now.
jbyers 1 days ago [-]
A large number of Docker containers?
Edit: Sure, a small minority presumably run sshd under these conditions. But a small minority times the number of Alpine containers running in the world gets to a large absolute number.
Edit 2: You really don't need to look very hard to find public container examples at million-download scale.
ekidd 1 days ago [-]
Many, many Docker containers rely on "docker exec" or the kubectl equivalent to gain CLI access when needed. To have sshd in an Alpine Docker container, you're generally need to be running init and a bunch of demons, not just a single standalone service.
I'm guessing in practice relatively few people are actually exposed. But if they are, it's catastrophic.
stingraycharles 1 days ago [-]
Docker containers run sshd?
hypfer 1 days ago [-]
> Edit 2: You really don't need to look very hard to find public container examples at million-download scale.
It would be super helpful if you would answer a "what, specifically" question with a "this, specifically" itemized list or not answer at all.
This is not helpful.
hypfer 1 days ago [-]
Docker containers that run sshds?
figassis 1 days ago [-]
Agent sandboxes today probably
illusive4080 1 days ago [-]
From the other thread’s source [1], it affects musl without PAM. You can set PasswordAuthentication=no or update to remediate.
Wow. I wonder which of the common fuzzers would have found this a decade ago. If literally any password over 256 chars works, that means nobody has ever fuzzed openssh on musl linux
sbrjdfgk 7 hours ago [-]
I think the following should be done. Either this, or eventually DataKrash will happen. International coordinated emergency should be declared with an immediate order to shut down all non-critical internet exposed services. Critical services should be put under armed guard and remote access should be disabled except for whitelisted critical staff. Then lustration of programmers should begin. Everyone who ever coded in C(rime) language and any descendant languages would be barred from this profession at least for next 30 years. Then maybe new software will be written with due care, and any new services should be put online only when written without any C - no OS code in C, no library code in C, no C conventions, etc
thewillowcat 1 days ago [-]
This is an unpleasant era to live through, but at least on the other side we may actually have real tools for writing secure software.
mmoll 1 days ago [-]
The musl vulnerability looks a lot like an intentional backdoor, doesn’t it?
Cakez0r 1 days ago [-]
Choosing * instead of NULL is indeed very fishy coincidence. Of all the return values they could've picked... What are the odds?
Arch485 8 hours ago [-]
About 1 in 256 I'd reckon
procaryote 19 hours ago [-]
Also the coincidence of 3 weeks later adding "oh, no passwords above 256 chars" making it easy to trigger
mrleblanc101 3 hours ago [-]
Microsoft Teams has flagged the sshhole.com as a malicious domain and now there is a org wide investigation where I work happening right now lol
tomaskafka 1 days ago [-]
At this point it feels to me we aren’t discovering security bugs but a decade of planted backdoors.
jephs 1 days ago [-]
Never attribute to malice that which yada yada yada
And like who even needed to plant back doors? If you're the NSA you already had an agent swarm in 1993, but it was made out of dudes from Berkeley and New Haven instead of racks in Prince William County and 印西 (and they were literal secret agents I guess, pretty metal). You can get back doors just by foraging, no need for agriculture.
ex-aws-dude 1 days ago [-]
No the biggest lesson of all this AI stuff to me is that we are just not smart enough as programmers
If you think about it that’s the whole idea of neural nets
We aren’t smart enough to design the function we want by hand as programmers so we need to learn it through weights
This lesson just keeps recurring in many different ways
actually this looks very underhanded indeed, what are the odds they pick the same sentinel character?!
jephs 1 days ago [-]
Like 1 in 5? The set of go-to characters for that kind of thing is pretty small.
phyzome 23 hours ago [-]
Why restrict yourself to a single sentinel character?
robinpie 1 days ago [-]
Pretty big? For this purpose, my instinct would be to select an ASCII character that's not alphanumeric, not directional, and not a common separator, and that's like, what, a set of 11 characters?
tialaramex 1 days ago [-]
You have considerably underestimated both how fallible humans are, and how high their tolerance is for risks which aren't immediately visible.
This isn't even just the software industry, humans are like this about everything. Imagine explaining to something rational that (a) humans didn't realise that they won't be able to competently operate a motor vehicle while watching TokTok dances, and (b) when we finally made that illegal because it's such an obviously bad idea they would keep doing it anyway.
This is why those "What if it builds SkyNet?" conspiracies were so stupid. Why on Earth would you build SkyNet when you can just invent Weeaboo and the idiot humans will die playing ?
ImPostingOnHN 1 days ago [-]
True, but also, that is exactly the pretext that would allow someone to maliciously, intentionally commit a backdoor.
There was even a contest for hiding/obfuscating malicious code, e.g. as an honest mistake: https://www.underhanded-c.org/ . And what do you know, the most recent, common approach shown on the page, is: improper comparison inputs that give unexpected comparison results, which is... well... this submission.
tomaskafka 1 days ago [-]
Yes, "this looks like an underhanded c challenge" was my first thought when reading this. See one mistake by one project, inject the matching intentional mistake into another project (that also conveniently fixes some known bug), and boom - plausibly deniable backdoor.
If I had a free afternoon, I'd ask a bunch of agents to track the people responsible and their other contributions (but usually the scheme is "contribute valuable patches to build reputation, and only inject an exploit when safe to do so").
zicixisi 1 days ago [-]
[flagged]
hk__2 24 hours ago [-]
I expected let another AI slope website and instead that was a real human writing about what they found. Sure they used LLMs to find this, but they used their real fingers to write this. See, this isn’t that difficult, and it’s a lot more enjoyable to read.
phyzome 23 hours ago [-]
It's definitely an AI slop website even if a human wrote the text. I found it pretty hard to get through the garbage animations.
Edit: My bad as pointed below this is not in OpenSSH but in musl and it appears in the relase notes https://www.openssh.org/releasenotes.html#10.6p2. I had scrolled straight to the bottom of the website because the animations annoyed me. Still would like a CVE for proper tracking though!
doodlesdev 1 days ago [-]
OpenSSH considers this to be a vulnerability in musl and not OpenSSH, but shipped a fix anyways since that helps mitigate impact... Whether you agree with that or not, it's up to you :)
mkj 1 days ago [-]
Probably because mitre take days to weeks to give out CVE numbers (if ever). Just get the fix out there.
draxen 1 days ago [-]
I avoided using alpine-based containers ever since finding out a lot of Linux server software is actually Linux + SystemD software. The union set API of both Linux and SystemD seems to be the default "Linux API" today.
otterley 22 hours ago [-]
Container images rarely ship with an init service of any kind (systemd or otherwise); the expectation is that your application is the root process. This includes Alpine-based images.
puzzlingcaptcha 1 days ago [-]
but alpine famously does not use systemd but openrc?
vel0city 1 days ago [-]
That's their point. The software they want to use often assumes its being run on a platform with systemd, so running that software on Alpine which makes a point to not use systemd causes challenges for them compared to using a different distro.
sam_lowry_ 1 days ago [-]
I use alpine-based containers whenever I can just to keep the world safer from SystemD rotting Linux.
Never enabled sshd there... except on my PostmarketOS PineBook Pro... oh shit!
andunie 1 days ago [-]
What is the argument against systemd?
Is it mainly that it is too big?
I feel like everybody still loves the unix philosophy, but the previous tools were too bad, systemd just filled user demand for easier tools.
The moment someone makes a pid 1 software that is good and still modular, open and pluggable I think that will be an instant success.
pona-a 1 days ago [-]
Yes the other tools at the time were pretty bad, but after and around the time systemd came out, they did get much better than the sysv-init days.
Check out dinit, it's basically the same declarative philosophy as systemd, but without the bloat, and with a much simpler config API. Or s6 if you specifically care about "modular", though unless you're an embedded dev, you probably don't actually want that.
sam_lowry_ 1 days ago [-]
Parts of it are good, like systemd-boot, parts of it are horrendous, like systemd-logind.
But systemd is not a technical problem, it's Redhat (and IBM) having too much control over what happens to the init system.
And of course there is Pottering and his vile Microsoft stint, then another vile stint to enforce age verification on Linux.
otterley 22 hours ago [-]
Eh, HNers are just noisier and crankier than most. The silent majority are fine with (and even like) systemd. I have no desire to go back to SysV init.
bspammer 1 days ago [-]
The last few weeks have made me feel very good about my policy of zero internet-accessible ssh ports, ever. On AWS, use SSM. For a VPS or home server, access everything over WireGuard.
Aachen 1 days ago [-]
Such solutions always felt like swapping one risk for another. With good separation (running the VPN on a separate host from the SSH login, otherwise a bug in the VPN solution still might result in a shell on the box), at best you'd need two vulnerabilities before someone gets in, but it's not fundamentally different unless you find a way simpler protocol where you can be more certain that it's secure than ssh is (for all its scrutiny and track record of no pre-auth bugs for decades)
Putting services behind ssh forwarding was the way to secure something you're worried about rather than the other way around, but Wireguard actually makes a good argument nowadays. Iirc they did proofs for the cryptography and it has good design in terms of attack surface, and it's old/mature enough now that I'll say it's probably better than exposing SSH. On the other hand, when this thing running in your kernel gets compromised, you're slightly worse off than with someone logging into a passwordless service account as in the current bug
bspammer 1 days ago [-]
One thing that makes me feel a lot more confident about exposing a WireGuard port is it’s completely invisible unless you have the host’s public key. Nothing will show up on a port scan.
Its security model doesn’t rely on that fact of course, but it’s the cherry on top.
stillpointlab 24 hours ago [-]
A common complaint I've seen (and I experience on occasion) is AI agents being a bit silly in their overprotective coding style. They nit pick and catch every single little error. This can be frustrating since a lot of the time they are trying to handle edge cases that are so unlikely to happen, yet to cover them requires significant additional code.
This might be the beginning of the reason we shouldn't be reading the code. It's like in chess engines where the players complain about how weird, alien and unintuitive the moves are. But 100 steps down the line the engine finds some edge case exploit that gives it enough advantage to win.
So we may be crossing a line, right now, where the non-intuitive alien code being generated by agents is seeing edge cases down the line that might allow exploits. And if we "correct" the agent, we may be leaving ourselves open to exploits we don't see and have difficulty understanding.
akiarie 21 hours ago [-]
The answer is not to reduce our understanding. That is the worst possible conclusion we could draw from what we're seeing. The answer is to increase it.
stillpointlab 2 hours ago [-]
That is similar to saying to Magnus Carlson that he should just increase his understanding of chess.
The point of my comment is, no amount of "increased understanding" puts a human on equal footing to a chess engine.
ShinyLeftPad 17 hours ago [-]
> non-intuitive alien code being generated by agents
You're saying it as if humans can't write that too.
stillpointlab 2 hours ago [-]
I am saying that we won't be able to write it, in the same way that Magnus Carlson (or whoever your top chess player would be) cannot compete with a chess engine.
I am arguing that, much like the "best" chess move is unintelligible to best chess players, truly secure code will be unintelligible to humans.
ShinyLeftPad 1 hours ago [-]
To make good code intelligible is compiler's job
rob 5 hours ago [-]
Man, that author is really committed to that lowercase aesthetic. Even the entire personal website. I wonder what's going on with that? We used to do that back on AIM informally. I mostly just see Gen Z and young kids still doing that now. Pretty crazy how all the stuff that used to be trendy comes back.
KMnO4 4 hours ago [-]
I don’t think you should let it bother you. I still found it more pleasant to read than the average article’s LLM slop
Kotlopou 1 days ago [-]
Tangent: Pangram says the text is 60-ish% AI-generated. Not that surprising given the look of the page and the work being done with AI in the first place, but it is interesting to me because this doesn't look like your standard wall-of-text AI purple prose. So it's notable that a) you can generate convincing text in this style and b) Pangram detects it anyway.
yuck39 1 days ago [-]
Panagram is just guessing.
chekibreki 1 days ago [-]
PSA for OpenWrt users: If you are getting the post quantum message on login which points to openssh.com, you are likely using dropbear nonetheless. Check in System->Software. I was confused and worried why I seemingly had the vulnerable OpenSSH installed, but its just dropbear using an OpenSSH link.
So AFAIU, you are fine.
throooooo 1 days ago [-]
The bugs and commits that "fix" them reveal somewhat of a shitshow for how musl is managed. All projects have bugs, but I can't help but question why the wheel was reinvented here. Many toes have been stubbed along the way.
jaekwon 8 hours ago [-]
minor nit: for Alpine's root, the * came with 3.16 in 2022, so that path is 4 years old.
the sshd/musl part dates to the 2016 CVE-2016-6210 fix, so other * accounts (key-only users, LDAP users) have been open for 10 years.
the website is amazing, I just think these details would clarify things.
p4bl0 1 days ago [-]
Nice web site. However, I've got a server with 14 alpine-based docker containers in production. Among them 13 are widely distributed images (like web server, databases, etc.) and one is a custom build image. Only one of them have an SSH server publicly exposed, Forgejo, but it's its own internal SSH server which only accept public keys and Git commands. On the internal network, only 3 of respond to SSH. I haven't tweaked their default SSH config. None were vulnerable to this attack.
dwedge 18 hours ago [-]
I tested an unpatched forgejo (enabling passwords temporarily, and even adding root to permitted users) and couldn't trigger this.
ETA: I could trigger it. You need to enable passwords and add a second user with a valid password, then git@ works. Obviously this still means the default forgejo is safe as you say.
thatxliner 5 hours ago [-]
This site is beautiful I'm going to say
gustavorg 9 hours ago [-]
My antivirus won't let me access the site, stating that it contains a threat: URL: Phishing.
pluc 1 days ago [-]
A+ name
hypfer 1 days ago [-]
A* name
kleiba2 1 days ago [-]
A{256,} name
henry700 1 days ago [-]
*
mceachen 1 days ago [-]
*0
matevskial 8 hours ago [-]
I have alpine linux on an arm device, and I have all the prerequisites for the exploit, yet I am unable to exploit myself. All of the Who is affected? from alpines security advisory applies to me. What am I missing?
matevskial 7 hours ago [-]
Update: I think I got the length of the password to enter wrong. I successfully exploited myself.
bschne 24 hours ago [-]
it's like the reverse of the Max Levchin story about almost losing the master key for PayPal's payment credentials (not gonna spoil further than that)
Did we declare "break the world Thursday" and nobody told me? I feel like it's three weeks in a row that we've had a once-in-a-decade security issue pop up.
vova_hn2 1 days ago [-]
I think we should get used to eternal "break the world Thursday", just like our predecessors had to come to terms with Eternal September [0].
They list OpenWRT as affected -- but by default OpenWRT runs dropbear not OpenSSH.
Dropbear is unaffected.
cube00 1 days ago [-]
> except a lot of programs never check for NULL. they just compare whatever crypt() gives back with the stored field, hit the NULL, and crash
> so musl (the libc under alpine) goes ok fine, crypt() just won't return NULL anymore. if it fails, it returns a single character instead.
Considering it's been accepted that crashing if malloc returns a NULL is your fault, did musl really feel the need to accommodate bad implementations like this or is this just AI speculation?
red75prime 1 days ago [-]
"unfortunately, a large portion of programs which call crypt are not
prepared for its failure and do not check that the return value is
non-null before using it. thus, always "succeeding" but giving an
unmatchable hash is reportedly a better behavior than failing on
error." [1]
Such a questionable decision! "Rather than let a bug in a caller (not handling the valid null output and thus crashing) surface, let's just silence it and pretend that the caller is correct! A potential clash in some situations (resulting in this vulnerability) is better than a crash!"
Maybe I should update my plea to fail loudly (https://alejo.ch/3gk) to list this as an example.
boomlinde 12 hours ago [-]
An interesting thing about C is that there is no guarantee that NULL dereferencing will "surface". It's undefined behavior.
NetBSD and libxcrypt both have similar workarounds. NetBSD has a paragraph on the reasoning.
The behavior of crypt() on errors isn't well standardized. Some imple-
mentations simply can't fail (unless the process dies, in which case they
obviously can't return), others return NULL or a fixed string. Most
implementations don't set errno, but some do. Version 2 of the Single
UNIX Specification (``SUSv2'') specifies only returning NULL and setting
errno as a valid behavior, and defines only one possible error (ENOSYS,
``The functionality is not supported on this implementation.'') Unfortu-
nately, most existing applications aren't prepared to handle NULL returns
from crypt(). The description below corresponds to this implementation
of crypt() only. The behavior may change to match standards, other
implementations or existing applications.
OpenBSD and glibc both have similar simple, stupid-free return value implementations. OpenBSD:
The function crypt() returns a pointer to the encrypted value on success, and NULL on failure.
GNU:
On success, a pointer to the hashed password is returned. On
error, NULL is returned.
The distro I use on the desktop uses libxcrypt, though. Probably more distros are. Its, NetBSD's and musl's workarounds probably make sense in some way if you have no clue what goes into your userland and are willing to write stupid software to accommodate for that.
01HNNWZ0MV43FF 1 days ago [-]
It's always a people problem. Doing a bad patch on my side is easier than bothering anyone else to fix their code
Good4boothee 1 days ago [-]
> did musl really feel the need to accommodate bad implementations like this
I don't know about this specific case but in general? Yes, totally. No one is going to fix all existing software that depends on libs behaving exactly as glibc, you have to accept some trade-offs. Either your new implementation stops being perfectly simple and clean, or no one uses it.
Palomides 1 days ago [-]
it's not "glibc behavior", returning null on error is in the spec
NewJazz 1 days ago [-]
To be fair the API is older than the POSIX spec.
vova_hn2 1 days ago [-]
See also: Hyrum's Law
ghm2180 1 days ago [-]
Thanks you to the team that found this. Exceptionally well explained and funny at the same time. What is most important is how people just checking stuff( remember the .xz hack that was tried?) has save the internets ass so many times. We should give good security researchers UBI for life for just doing good work.
glennfiedler 1 days ago [-]
It's time to switch to OpenBSD
mkj 1 days ago [-]
It would be nice if the passwd crypt functions didn't have timing dependence on password length. Then there'd never need to be hacky workarounds like what musl did, and various tricks to avoid exposing whether users exist on systems.
No idea if it's likely to happen though.
1 days ago [-]
schappim 23 hours ago [-]
We use Alpine quite a bit so I built our team ssh-audit[1]. Feel free to use it if you want.
OpenBSD probably did (which doesn't use musl), RedHat probably did (or some other distro using glibc), but they all don't use musl.
bensyverson 1 days ago [-]
Yeah, I am mystified as to how this could have happened.
throwawayffffas 1 days ago [-]
I learned maybe a year in software, should not match/happen are famous last words.
This is just negligent. Not a fan of rewrite in rust, but I find it highly unlikely this would happen in that community.
dabinat 22 hours ago [-]
Rust tells you off if you don’t handle Results. Sure, you can still just unwrap() it and have it panic, but at least that’s a specific decision instead of forgetting to check for NULL like in this case.
leonjza 1 days ago [-]
Managed to reproduce this, with a lab[0] you can play with.
Can we all marvel at the site. The author says they did the discovery on November 3. Since then they apparently bought a domain, set up the webpage, created animations, and a cartoon. All kinds of stuff, and got it to explode on hackernews. A span of 2-3 days. How long ago was this not even conceivable?
paaloeye 1 days ago [-]
Not the worst site, but it's clearly a publicity machine from wakefield.co, whom I haven't heard of.
Also, exploitation conditions are super narrow, albeit, any password is wild.
Horrors behind comprehension all around. The code, the error case being “*”, the loose patches plastered over. musl and C are unserious
philipwhiuk 1 days ago [-]
Magic characters are awesome.
stogot 1 days ago [-]
Of all the named vulnerabilities, I appreciate the history lesson and screenshots of how it got there
ChrisRR 1 days ago [-]
Can we just have a site with the text and images without the awkward scrolling crap? Hijacking basic features is an instant click away from sites for me
Edit: It seems to work fine on mobile. I'm not sure why the scrolling behaviour was added on the desktop version of the site
israrkhan 1 days ago [-]
how come such a thing can exist for more than a decade and get unnoticed? Weird times.
ctrlaltdileep 1 days ago [-]
That's kind of shocking. I guess the swarm of AI related vulnerability findings continues.
vdfs 1 days ago [-]
It's shocking that it took so long with the amount of investment and hype you'd expect this kind of bugs to be already fixed, but it still need a human to tell the agents what to do, which could also work with any fuzzing framework
gosen 1 days ago [-]
I really like creativity in the name, the way website built and the content
"tl;dr" was a bit of a downgrade though, but that's fine
it's a nice way to promote the company behind it
muddi900 1 days ago [-]
I would like to read this but the scrolling is completely broken on iPad Safari
jbirnick 23 hours ago [-]
Am I the only one who is shocked that such important and security-relevant software just does unorthodox stuff like "well let's just return `*` if it fails" and other kind of "hacks" ?! The developers must know that this is very bad software engineering and will definitely cause issues at _some_ point in time, no?!
hoppp 23 hours ago [-]
Was this exploited in the wild?
tonnydourado 1 days ago [-]
Am I missing something or is this written up to seem a lot worse than it actually is?
It starts with a bold claim: "ssh accounts that should not accept a password will accept any password between 256 and 1024 characters". But then it keeps adding qualifiers:
- sshd compiled with musl
- not using PAM
- password logins enabled
- at least one account with '*' as the password "hash"
- at least one OTHER account with a valid password
- server is aligned with 2 ley lines
- hacker is wearing a blue t-shirt and green socks
Ok, the last two are an exaggeration, but still. The conditions are all ANDed, miss a single one and you're fine.
On top of that, as someone else in these comments reminded, the target account needs to have a valid login shell or other exploitable program assigned to it, and these passwordless accounts tend to have nologin or something similar.
To me this seems to fall on the "technically possible, practically very unlikely" bucket, kinda like the PHP vulnerability from the other day. Maybe this has gone unnoticed for so long because it just doesn't happen.
pizlonator 1 days ago [-]
There are a lot of systems that meet those requirements.
> - sshd compiled with musl
So alpine as well as a bunch of other embedded Linuxes. Huge install base.
> - not using PAM
What you get if you try to have a minimal setup. Default sshd config if you install from source.
> - password logins enabled
The default. Super common.
> - at least one account with '*' as the password "hash"
Almost every Linux system.
> - at least one OTHER account with a valid password
Almost every Linux system.
So sure, there are caveats, but they cover a massive install base.
This is a super bad bug
tonnydourado 1 days ago [-]
How many alpine installs are running sshd? Most containers using it as a base image aren't. Embedded Linux is unlikely to be running it, too. And if one IS running sshd, how often is it with unchanged defaults? I don't buy the inference here.
I'll concede that embedded linux might be my blindspot here. Maybe cheap web hosting, too, but that seems much less likely to be using musl.
svnt 1 days ago [-]
Embedded linux is your blindspot here.
__MatrixMan__ 1 days ago [-]
Rumor has it that Cisco has been hurriedly patching something and refusing to tell their customers what it is. I'm thinking it's this.
tchebb 1 days ago [-]
Most embedded systems I've seen use glibc or uClibc. I agree it would make sense for them to use musl, but the embedded space moves very slowly.
pizlonator 1 days ago [-]
Yes
andix 1 days ago [-]
A lot of embedded systems use musl and no PAM. Some of them might use Alpine, but the distribution doesn't matter that much.
3m4r 1 days ago [-]
Most of those would be using dropbear rather than openssh.
tonnydourado 1 days ago [-]
And ssh? I'd assume most would use proprietary or industry-specific standard for data IO, or just https, for domestic IoT stuff
andix 1 days ago [-]
And also ssh in many cases.
pera 1 days ago [-]
I thought Dropbear was pretty much the defacto SSH server for embedded applications, is OpenSSH+musl actually a common combination for these scenarios?
andix 2 hours ago [-]
Some use Dropbear, some use openssh.
This vulnerability clearly is not one that affects everyone like the PHP vulnerability did. But it's exposure is still significant and very easy to exploit. As easy as it gets.
We don’t know that dropbear isn’t vulnerable. The bug here is in musl and OpenSSH is just being a bro by working around it
hnfong 22 hours ago [-]
I wouldn't say the bug is only in musl. Musl is probably the "worst" offender here (at least choose something other than "*"!), but the change in sshd was also a little suspect. i.e. the "let's find a user and take their salt (and hashing algorithm) and use it to compute and compare the hash to avoid timing attacks" change.
I haven't thought it through, but I suspect in an ideal world, sshd shouldn't have accepted the user password regardless of the outcome of crypt() if the user did not exist and it only pretended to go through the motions to avoid a timing attack.
Did look into their patch yesterday(?) which seemed to mitigate this issue which they (rightfully IMHO) consider caused by musl. But it seems their fix is to do a sanity check on the supposed hashed password char length, and thus still theoretically still allowing the possibility of those "!!!INVALID_PASSWORD!!!" suggestions to fall through....
gsich 22 hours ago [-]
Tried it, doesn't work.
lucideer 23 hours ago [-]
> Most containers using it as a base image aren't.
On top of the embedded linux blindspot, this statement seems unfounded.
> if one IS running sshd, how often is it with unchanged defaults?
By definition, I would imagine the answer to this is most of the time.
qwm 1 days ago [-]
For what it's worth, default Alpine installs don't have an account with * as its password hash if you use the root account as your main one. Not sure if that's also true for ones where you set up a user, maybe those ones are vulnerable.
utopiah 1 days ago [-]
I'm genuinely curious what's the Venn diagram here because I feel a lot of embedded systems precisely have one condition but not the other, e.g. password yes but not necessarily multiple accounts, e.g root only, while using musl. Namely almost all for almost all, but if only one is not overlapping another then it's actually very few.
yjftsjthsd-h 1 days ago [-]
>> - at least one account with '*' as the password "hash"
> Almost every Linux system.
But not, notably, on Alpine, which uses '!' instead.
Edit: or maybe yes on Alpine? I checked my desktop and it has zero *s in shadow, but Alpine's post seems to say they do use * sometimes
tremon 1 days ago [-]
I thought they had a difference in meaning, i.e. * means the user has no password and ! means that password logins are disabled for this user, but the man page does not make it clear if there's a distinction.
From man 5 shadow:
> A password field which starts with an exclamation mark means that the password is locked.
> If the password field contains some string that is not a valid result of crypt(3), for instance ! or *, the user will not be able to use a unix password to log in (but the user may log in the system by other means).
LinaLauneBaer 1 days ago [-]
does this really matter? It could be used to do something devastating. The number of systems that match those requirements is only one factor. If just a handful of those are highly critical for humanity it could lead to disaster. Eg. hacks into fridges, centrifuges etc.
rhet0rica 1 days ago [-]
Updates since this was posted:
- blue t-shirts and green socks have been added to the default loadout for new hackers, since it's optimal according to AI and costs nothing to the attacker in terms of compute
- there are so many ley line "maps" out there that basically any server placement counts as aligned with at least two, so this one's free also
Onavo 1 days ago [-]
Blue t shirts and green socks?
RetardedBastard 1 days ago [-]
[dead]
petecooper 1 days ago [-]
Vulnerability-branded merch? Unsure.
b112 1 days ago [-]
Someone owns a blue shirt/green socks clearance warehouse, this is all just and ad? I've seen weirder. I can imagine visiting SanFran in a few months, and it looks like a weird sci-fi movie scene, everyone dressed the same.
dcrazy 1 days ago [-]
SF has puffer jackets. Milan has men in blue suits. Paris has women whose wardrobes consist of black and white.
alexchamberlain 1 days ago [-]
> password logins enabled
Disabling sshd password logins is security 101, and the default for some cloud provided images these days. If password logins haven't been disabled, what other doors have been left open anyway?
twothreeone 24 hours ago [-]
Now the question for the man: would fil-c prevent it :D
1 days ago [-]
vadansky 1 days ago [-]
> > - password logins enabled
> The default. Super common.
This always seems bonkers to me. I think SSHD should give you ONE password free connection of setup to copy your key over and then it just shuts the door behind it.
hnfong 1 days ago [-]
I think it has to be a time limited FREE offer. (like, seriously, what if the user doesn't take the offer?)
It might actually a good idea, but I would likely avoid it since my built-in Bayesian filter would just mark it as spam.
diath 1 days ago [-]
Not sure why you're getting downvoted as from security standpoint that would be the right thing to do; but actual good server providers take it up a notch and allow you to upload the public key through their admin panel during server set up and the password you get only works locally for sudo etc. There's no reason why a fresh Linux setup should ever offer a publicly available SSH server with password based auth out of the box.
hnfong 1 days ago [-]
Alpine Linux (which uses musl) images are popularly deployed by those who use docker, and apparently the issue is there on the default install.
Not sure whether the PHP vulnerability actually had been exploited, but I'm pretty sure there are literally thousands of machines out there affected by this ssh issue right now as we speak.
This isn't going to end well.
---
Addendum (which may or may not be related to the above) --
Everyone has heard of the tale of the boy who cried wolf. The main moral of the story is not to troll your fellow townsfolks so they don't distrust you. But everyone with a sane mind gets that.
The (IMHO) under-appreciated lesson is actually: don't be those villagers. Believe stories you are given, even if you might look like a fool for believing in them. Don't assume what happens is the same as what happened in the past. Look at the facts yourself. Trust and verify.
VorpalWay 1 days ago [-]
You don't typically run sshd in containers, instead you only run whatever single service you set up your container for.
Sure there are native Alpine installs too, but that is far less common of a use case.
tonnydourado 1 days ago [-]
Who's out there running sshd inside a docker container? It's almost MORE work than exec-ing into the container.
hylaride 24 hours ago [-]
I've done it in the past with "serverless" sftp servers, which is how a lot of inter-company batch transfers happen for historical reasons, especially in the financial sector. I don't recall what images I used, but it was far easier to patch with a CI/CD pipeline. We did use keys, though.
cluckindan 1 days ago [-]
Probably a lot of companies which have their own k8s cluster and segregate kubectl access to operations only. Developers still need a way to their app backend in some cases.
chuckadams 1 days ago [-]
Kubernetes already comes with better solutions for that sort of thing. I could see a ssh bastion container for bridging to legacy infrastructure though. Manage its keys right and it's probably more secure than a lot of alternatives.
jbvlkt 23 hours ago [-]
Even if you do that why would you allow it outside VPN?
cluckindan 6 hours ago [-]
You wouldn’t, but it would still remain an avenue for lateral movement
zbentley 1 days ago [-]
Several CICD solutions which offer "shell into container to debug a build failure".
You'd think they'd all do the smart thing and exec a shell into the container's namespace. You'd be wrong.
Jolter 1 days ago [-]
I have done that in the past, in order to be able to automate copying files to the container during testing or data migration. It was an ugly hack and now I need to make double sure none of those containers are still running…
Brian_K_White 1 days ago [-]
Everyone who has access to their container and not the host except by some terrible vnc or web console.
andix 1 days ago [-]
Alpine in containers is usually not affected. Most images/containers haven't ssh/sshd installed at all. Remote access is usually done from the host system.
63stack 1 days ago [-]
You could start with trust and verifying what you wrote first, the popular usage of alpine in docker containers don't run sshd.
hnfong 23 hours ago [-]
Even if 0.1% of those machines run sshd I guesstimate we'd still be talking thousands.
1 days ago [-]
tcdent 1 days ago [-]
Really, "password auth enabled" should be enough to be a non starter these days.
qwm 1 days ago [-]
you'd be surprised
itemize123 1 days ago [-]
only 1) is a narrowing filter. the rest is basically the majority.
VorpalWay 1 days ago [-]
Most linux distros use PAM, so 2 is also a narrowing filter. Debian, Ubuntu, Fedora, Arch, etc all use PAM.
jack1142 1 days ago [-]
What if you narrow it down to musl-based distros (which none of the above are). Do most musl-based distros use PAM?
VorpalWay 1 days ago [-]
Alpine is mostly used in containers, where you don't run sshd anyway. It apparently doesn't use PAM by default though.
OpenWRT also doesn't use PAM by default apparently, and there is a risk of people using password login, which you should never do past initial setup.
But outside specific niches, musl is rare, with good reason. Performance is a joke compared to glibc, especially the allocator. The only reason to choose musl in my opinion is for memory constrained devices (OpenWRT etc) or for building static binaries where performance doesn't matter but portability does. I consider the "small container" use case questionable: performance will suffer, and you could share large base layers instead which would amortize the cost over many different images.
kasabali 1 days ago [-]
AFAIK OpenWRT doesn't use openssh
jack1142 1 days ago [-]
Yeah, OpenWRT doesn't use openssh by default - it uses dropbear. You can install openssh, if you want more features and there seem to be 2 packages - one with PAM support (openssh-server-pam) and one without it (openssh-server). PAM-less one is the one without the suffix so perhaps it's more likely you'd install that one out of the two, but only if you were trying to switch to using openssh in the first place.
anamexis 1 days ago [-]
Those distros are all filtered by 1, no?
edf13 1 days ago [-]
Yes - immediately had me in a mild panic that I’d missed another AI discovered crit bug!
Had to quickly scan the rest of the homepage of HN to confirm.
alienbaby 1 days ago [-]
I suppose there is the chance you get to prep a system setup at one time, so you can access it at a later time?
ShinyLeftPad 18 hours ago [-]
Yes, 100% written to oversell it as hard as possible. Still a vuln but karma farming too.
CodeWriter23 23 hours ago [-]
Yes, missing the rest of the world that does not conform to your self-references.
arunc 24 hours ago [-]
> - hacker is wearing a blue t-shirt and green socks
You had me there :D
j45 1 days ago [-]
I think you're missing the reason why it's relevant now.
LLMs will find it.
They work endlessly at finding the 'and' scenarios.
This should have been closed up a long time ago.
bigbuppo 24 hours ago [-]
Every time I see one of these named exploit sites I always think, "Hey look at this guy, he wants attention."
cubefox 24 hours ago [-]
> kinda like the PHP vulnerability from the other day
What? Pretty sure Cloudflare detected widespread attacks almost immediately after it was published. We are probably lucky that not much more happened because of responsible disclosure and the large media attention led to widespread admin awareness.
huslage 1 days ago [-]
It’s irrelevant how widespread it is. It’s a bad bug that should be fixed. Why do people always say stuff like this? If you’re running a system that is exploitable then you deserve to know about it and to be provided a fix. It doesn’t matter all that much how widespread it is. The CVE system doesn’t exist to cater only to highly exposed systems.
its_ethan 1 days ago [-]
It matters how widespread it is because humans have limited time and resources to solve problems. If these conditions only exist in 0.001% of installs, is it worth the man hours to fix it? Alpine may think not, because like others have mentioned, there are likely other higher probability vulnerabilities being exposed in order to meet the conditions for this to even matter. A judgement call can (and clearly has?) be made to prioritize other work.
The good news is that considering Alpine is open-source, you can be the one to make the judgement call that it is worth fixing and go out there and fix it ;)
tchebb 23 hours ago [-]
A trivial-to-exploit auth bypass is always worth fixing, no matter how few systems are affected right now in practice. My question is whether it's worth giving the vulnerability a name and making a webpage for it.
its_ethan 16 hours ago [-]
Sounds like you should go fix it then!
gf000 23 hours ago [-]
0.001% of all Linux installs (including containers, CI workers, embedded, etc) is still a huge huge number. It should be taken seriously until proven otherwise
its_ethan 16 hours ago [-]
ok then, you should go fix it! and thanks for pretending the entirely made up number of 0.001% actually matters
jappgar 42 minutes ago [-]
almost seems like a planted backdoor in alpine.
expect more of these
stevenhubertron 1 days ago [-]
I’m a smart person and none of this makes any sense to me. It’s like purposefully written poorly.
If The UNIX-HATERS Handbook should ever get updated, this deservers its own chapter.
titzer 1 days ago [-]
I'm starting to realize that trusting some random people who wrote the gruddiest, boringest, most torturous low-level stuff might have been a bad plan.
At this point I feel embarrassed in front of the AIs.
oefrha 1 days ago [-]
This is a super duper annoying site for people who just want to know if their systems are vulnerable, after patching like a dozen critical vulnerabilities in other software over the last week and are almost at the breaking point of resigning.
Can we have a “don’t treat me like five/toddler” / “idgaf about the explanation” quick link to a section for adults who need to patch systems?
(And if your reaction is “just scroll down, stupid”, I’m happy to report that with the scroll jacking, I scroll not enough / past the critical sections nine out of ten times.)
Thanks for mentioning that before I tried to scroll down. Forewarned, I was able to click on the "2026" link once the left-handed "scrollbar" (not the right word but I can't figure out what the right word should be) appeared, which let me jump past most of the verbiage and get to the "are you affected?" section just a couple pages later.
jamal-kumar 1 days ago [-]
[flagged]
alper 10 hours ago [-]
At this point we should all agree that software engineering is an elaborate well paid joke.
peanut-walrus 22 hours ago [-]
What this tells me is that musl+openssh-server is an incredibly uncommon setup, as "at least 257 character string" is literally the third thing I try after "admin" and "12345" when greeted with a password input.
anilakar 12 hours ago [-]
Joke's on you, my busybox only supports generating MD5 hashes so I'm safe.
angry_octet 12 hours ago [-]
The best part of this exploit is undoubtedly the name, *sshole.
symlinkk 3 hours ago [-]
Wow seriously no unit or integration test for this?
thadt 20 hours ago [-]
Theory: “With enough eyes, all bugs are shallow.”
Corollary: “Nobody looks”
The proof of this corollary is left as an exercise for the reader.
nialv7 1 days ago [-]
This AI slop write-up is so fucking annoying to read. I don't even mind the AI generated website. Why can't you even bother to spend like an hour to properly sum up the exploit yourself?
bombcar 1 days ago [-]
The real name of this is sshole - the is the "star" of the show, after all.
yreg 1 days ago [-]
*sshole, the * is the star of the show
You can use \ to escape formatting.
bombcar 1 days ago [-]
Ugh! HN got me, too
1 days ago [-]
cozzyd 22 hours ago [-]
Good thing my yocto images use glibc...
hinkley 21 hours ago [-]
What did you just call me?
shaolinspirit 1 days ago [-]
came for the name, stayed for the story
temp123477 14 hours ago [-]
One h or two?
DANmode 11 hours ago [-]
SSH olé
MarkBlah 1 days ago [-]
Haha insane, great find and write up
nasso_dev 1 days ago [-]
something interesting to note here is that while this isn't a memory safety vulnerability, a modern systems programming language with better error handling could have easily avoided this whole situation. error handling in C is a PITA, and we're paying the cost of decades of pretending that it's not a problem
harrouet 1 days ago [-]
I need to check my fridge...
Eleg007 18 hours ago [-]
Love the website
qwel-1asj 1 days ago [-]
All that site is lacking is marquee and furry creatures. AI slop sites are so horrible with their blinking animations, l33t dark background and chaotic text.
gabucino 1 days ago [-]
rich felker was such a f* (funny) guy back in the mplayer days too
never changed i guess
krick 16 hours ago [-]
What is this idiotic manner of typing full articles in lowercase? Don't people learn basic orthography in school anymore? This is fucking unreadable.
meindnoch 23 hours ago [-]
I refuse to believe this is not a backdoor.
andreagrandi 1 days ago [-]
This is fine :)
lclc 1 days ago [-]
One more reason to build your (own) Docker images on 'FROM scratch' (basically nothing).
blueflow 1 days ago [-]
No, the Alpine containers don't come with sshd preinstalled. You'd have to manually install via dockerfile and leave it unconfigured.
Good4boothee 1 days ago [-]
No one runs sshd in containers, at least I hope so. What would even be the point when you have docker/kubectl commands available?
Dwedit 1 days ago [-]
Using LXC (not the same as docker) and running openssh inside the container.
tgml 11 hours ago [-]
quick fix?
ubermonkey 23 hours ago [-]
I desperately want someone to create a single-letter subdomain here.
sneak 24 hours ago [-]
It’s dumb and negligent to be missing “PasswordAuthentication no” in 2026. It’s been so for at least ten years. OpenSSH should have turned off password auth in the defaults years ago.
henry700 1 days ago [-]
at this point my only reaction is "are you fucking kidding me"
varjag 1 days ago [-]
just thank god for dropbear
dcchambers 1 days ago [-]
This one feels negligent. How does this go uncaught for 10 years? How did it make it into production period?
We need to re-think EVERY assumption about security.
DonHopkins 1 days ago [-]
This is bottom-up branding I can get behind. ()*()
1 days ago [-]
HackerThemAll 1 days ago [-]
I just created an Alpine container, did "apk add openssh" and this doesn't work for me. I assume it's been updated already.
1 days ago [-]
jdw64 1 days ago [-]
I liked that this site explained the vulnerability in an easy-to-understand way.
ordu 1 days ago [-]
I can't understand it. All this scrolling and pictures are distracting me, can't concentrate really.
vavkamil 1 days ago [-]
Yep, scrolling through my phone and reading slowly to understand. I'm seeing distracting animations below what I'm reading. And when I go to them, there is no way to restart them. So it completely misses the point.
jdw64 1 days ago [-]
I think it varies from person to person. Since I'm kind of from the TikTok generation, I don't have much trouble reading this kind of animation, but some people prefer static sites. That's just how it is.
I also found scroll-triggered animations hard to read at first, but these days a lot of sites use them, so I've gotten used to them.
ordu 1 days ago [-]
There is a myth that TikTok generation has ADHD and can't concentrate, if it were true I'd expect static sites would be easier for you, like to me. I mean, I struggle sometimes to keep attention and not to wander off mentally. With static text it is easier to fix, you have just to jump back a bit, when you notice that you've forgot to read while you were reading. Or when you are lost at too many trees you can kind of look at the forest, at the text as a whole and find your place there. But with all this scrolling it is hard to jump backward/forward at will. It feels tiring and annoying.
> how did I ran into it? [...] i had agent swarms. a lot of them. let's not get into it
Can we please now normalize AI usage in development? Just some other day people asked for "NO AI used" badges for software as measure for quality, and praised i.e. NetBSD for being AI-free. Yes, we come from a phase of AI slop, but it has become obvious that there are plenty of legit usecases where AI beats human-only effort nowadays.
__MatrixMan__ 1 days ago [-]
You've got a great point. But let's also acknowledge that there's a pretty stark contrast between these:
- I found a way to get AI to help me while I made this
- I asked AI to make this
The people you're objecting to are complaining about the latter, this is an example of the former.
neuronexmachina 1 days ago [-]
There's a decent-size "any AI usage is theft" contingent that reacts dramatically to any usage of AI whatsoever.
rglover 1 days ago [-]
Not sure why but a lot of people seem to be missing that you don't have to look at AI as a binary "you're either with us or you're with the terrorists."
Just do HITL (human in the loop). Use an LLM to help you author, but steer the AI in whatever direction you want. It's like having a slightly delusional copy of yourself to pair program with. If you take the time to provide an LLM with some additional skills files and the like, with you in the pilot seat, you can get solid results (it doesn't look as cool as a hands free agent setup but the quality far surpasses anything they can do solo).
SrslyJosh 1 days ago [-]
It's a stupid and wasteful way to automate the process.
vova_hn2 1 days ago [-]
> Can we please now normalize AI usage in development?
1. It is already normalized outside of a tiny, but very vocal bubble that lives mostly on Bluesky (sometimes also on Reddit/HN/Mastodon).
2. I don't think that even the most staunch AI hater would object to using AI for security research as long as all the artifacts (code, docs) that were produced based on the research are human-written. "Yes, there is a bug in my program, but I refuse to fix it, because I know that you found it using AI" would be a pretty weird position. Not sure, how would you defend it.
littlecranky67 1 days ago [-]
I feel like there is a lot of pushback from delusional engineers, that won't accept their coding skills have been rendered useless - so they come up with tons of reasons to avoid AI, because it just has to be bad. Anything to hold up the ilusion you are an well-paid white-collar worker that can pick employers anytime. And I say this myself coding since the late 90s and making a living off it - it is a painful pill to swallow, but the genie does not go back in the box so just accept it and move on.
cowlevel 1 days ago [-]
This is a pretty well done AI generated website, congrats you have proven it's not solely for slop.
kevinfiol 23 hours ago [-]
Are articles like this deliberately designed to make it to the frontpage of HN?
tonymet 24 hours ago [-]
what was done to report & patch the vuln? Why publish it in this format? It’s presented like a whistleblower who’s exhausted every approach, but the effort to do the right thing is not clear.
karel-3d 1 days ago [-]
does this have CVE or something?
thomcrnl 20 hours ago [-]
cve-2026-123158
mschuster91 23 hours ago [-]
> on alpine's official aws and azure images that's the default alpine user, and it has passwordless doas. so it's root
good fucking god. if I get this correctly... you can insta-own any aws/azure instance that was set up with alpine as long as it has ssh enabled, which it probably has.
globular-toast 1 days ago [-]
Tbh the most disappointing part of this is musl (and later glibc) changing the function to stop callers crashing because they didn't RTFM.
sliqqq 23 hours ago [-]
jesus christ wtf
TacticalCoder 23 hours ago [-]
Can we hear something from those who say that using port-knocking is "security through obscurity"?
Really: from the XZ backdoor in SSH (which went live on some distros) to this...
What's you rationale for calling port-knocking "security through obscurity" and not "defense in depth"?
Put it this way: if I advertize that my server is using port-knocking, that the sequence changes every 30 seconds (like TOTP) and that as soon one sequence is used, it's unusable (so you've only got one attempt to SSH in every 30 seconds, even if you know the sequence)...
Where's the security through obscurity?
But really: even if had musl and PasswordAuthentication allowed (I don't have neither), how would an attacker gets through the port knocking?
More fundamentally the question is: in which way is your system that does not use port knocking more secure than the one who do?
(remember: we're talking about remote exploits by simply having exposed TCP ports open or SSH listening)
Wanna have a talk too about the "solutions" proposed by "security experts" here "because SSH is too old" and that happen to be crufts on top of SSH that had remote root exploitable backdoors? (sorry, holes)
damatar 20 hours ago [-]
[flagged]
ethanhood_okroh 16 hours ago [-]
[flagged]
23 hours ago [-]
ravivarmat 12 hours ago [-]
[flagged]
mari_dtfy 1 days ago [-]
Disclosure: I work at Detectify. A few practical notes for anyone checking their own machines:
- It affects Alpine's openssh-server (built without PAM), or openssh-server-pam with UsePAM no, when PasswordAuthentication is enabled. Alpine says every release they checked is affected, 3.16 through 3.24 and edge.
- Fixed openssh packages are out for all of those, including the older 3.16-3.20 releases. Upgrade and restart sshd (rc-service sshd restart). If you can't upgrade yet, set PasswordAuthentication no.
- To check a box: apk info -v openssh-server for the version, and sshd -T | grep -iE 'usepam|passwordauthentication' for the effective config.
- Containers built FROM alpine that run sshd are easy to miss in an inventory. So are the official tiny-cloud images (3.18.11 through 3.24.3), which ship an alpine user with no password and passwordless doas, so a bypass there means root.
- After patching, look in /var/log/messages for sshd lines saying "Accepted password for" an account that has no password.
1 days ago [-]
rom1v 1 days ago [-]
[dead]
asclepios_v 1 days ago [-]
[dead]
v1b3m 1 days ago [-]
[flagged]
dadadat 1 days ago [-]
[flagged]
Krea8ly 5 hours ago [-]
[dead]
fleroviumna 2 hours ago [-]
[dead]
royaleagle7 10 hours ago [-]
[flagged]
tremarley 1 days ago [-]
[dead]
PhaseTwobig 24 hours ago [-]
[dead]
tonetheman 1 days ago [-]
[dead]
Docpixelkb 14 hours ago [-]
[dead]
thomcrnl 21 hours ago [-]
[dead]
TZubiri 1 days ago [-]
what’s the moral of the story?
alpine stocks are crashing?
musl stocks are crashing?
glibc stonks are up
alternative implementations in general slightly down
sneela 1 days ago [-]
[flagged]
BugsJustFindMe 1 days ago [-]
> Also, what's with this damn trend of typing in lowercase only?
The goal from an attacker’s perspective is to get this to be true: `$ret = crypt($pw, $hash); $ret == $hash` when called by OpenSSH’s sshd. This is how sshd validates that the user logging in knew the account password.
The `$hash` is the “settings” to the crypt call and will normally look like “$6$--gibberish” for users with passwords set, but could look like “*” or “!” for users not intended to be able to login.
The bug is from the intersection of all of these things:
1. musl can return “*” for errors, rather than leave the return value NULL. Other libc have adopted something like this too, but seem to use a different result “*0”.
2. musl can also return “+” for errors, but only does so if $hash starts with “*”. This is intended to eliminate the possibility of an error causing `$ret == $hash` to be true, however it assumes that applications calling `crypt()` will set $pw and $hash to values corresponding to the `$ret == $hash` comparison to be made immediately afterwards.
3. OpenSSH can be convinced to pass the hash of a different user if the hash is “*” or “!” to avoid timing attacks that can enumerate the user list, so it will set $pw and $hash to correspond to different users. $hash will be a real “$6$--gibberish” style hash in this case, and OpenSSH expects `crypt()` to either return NULL or return a hash string. But now there’s two hashes OpenSSH uses. It does: `$ret = crypt($pw, $hash2); $ret == $hash1`;
4. This should normally not be a problem because OpenSSH will pass in valid $pw and $hash2 that should not lead to errors in hashing, so the $ret that musl generates should be a real hash that cannot match the “*” or “!” $hash1 for the login-disabled user. However as a DDoS mitigation, musl will return error if the password is too long when using SHA-2-based hashes (the default, it seems).
5. This error will result in a non-NULL result of “*” or “+”. Since OpenSSH passed in a real hash ($hash2), musl will return “*”, which compares equal to the actual user hash $hash1 for the disabled-login user.
The OpenSSL adjustment ensures a minimum hash length for the user before logging them in, even if crypt() succeeds somehow.
It is the opposite that is unsafe, that is, when the returned value can't be NULL, but it stills returns a pointer. C++ has references in an attempt to fix this problem.
Because C tends to prioritize performance over safety, a NULL pointer dereference is considered undefined behavior and the compiler can invoke nasal demons if it makes the code go faster, but with the right options, you can have it checked and invoke what in other languages would call a panic handler, but on UNIX is the SIGSEGV signal. You can then attach a debugger or read the core file and see what went wrong.
Note: I am not talking about "char *" for strings, that one is unsafe, but NULL is not. It is just a special case of an option type, or a result type, or an error union. Modern languages just extended this idea to objects other than pointers, and with more than one type of error.
You don't even need a fancy result type. A simple Option<String> would've forced downstream users to consider the failure case, avoiding this situation in the first place.
Now what?
However a better API (to me) that I would have written was:
That way btw the function doesn't need to allocate memory inside, which is (being a mostly embedded programmer) something I don't like at all. Plus returning an error enum makes it clear instead of having to look at errno.But POSIX API is not something well designed, it dates back to the '80 so... probably today would be written completely differently
If NASA were to use the equivalent of `pub struct Newtons(f32); pub struct PoundsForce(f32);`, where the types Newton and PoundsForce were not interchangeable, this would never have happened[1].
[1] I'm aware that that story is a bit more complicated than that, that CSV files were involved (which are, again, not type safe), and that, in that particular scenario, more would have to be done to preserve type safety across formats, vendors and codebases, but my point still stands.
https://lexi-lambda.github.io/blog/2020/11/01/names-are-not-...
> The newtype is opaque, so the only way to observe it is to convert it back to a [f32].
These are not the same, and your next sentence suggests why. Option has but a single error type, but result types and error unions allow arbitrary types of error.
That depends on how you interpret the pointer type, but in any case there are multiple interpretation and just the fact that these exist will create confusion and type unsafety.
I had never heard that phrase, and was delighted at the answer when I googled it its origin, thank you:
http://www.catb.org/jargon/html/N/nasal-demons.html
> “When the compiler encounters [a given undefined construct] it is legal for it to make demons fly out of your nose” (the implication is that the compiler may choose any arbitrarily bizarre way to interpret the code without violating the ANSI C standard)
I've heard nasal demons invoked as a literal possibility. No they are not a literal possibility.
Which is zero, to be clear. There's not even a standard USB device class for a nasal demon releaser
A null check elision is not a demon up my nose - it's a null check elision.
That's... a bizarre choice, considering how religious C is about not making any assumptions about what a machine can and cannot do, even going to the trouble of working with non-IEEE floats and int types that are not a multiple of 8 bits.
That's a bit weird? In Rust, if you have an thing that returns Result<String, String> that's still type safe, even though success and error types are the same.
I think I understand what you want to express, maybe. You might say it's in-band vs out-of-band error signalling?
These types are not the same:
Nit: the type constructor is `Result`: given two types `A` and `B` (not instances of those types, but actual types) it gives you a new type `Result<A, B>`.
`Ok` and `Err` are called "variants" and can be used to construct a new _instance_ of `Result<A, B>`.
I think almost all (tech) arguments stem from simple disagreements and/or different assumptions made on the definition of a (subjective) term.
Unless the point of your argument isn't to understand OP but simply to "win."
You don't have to think C is sound or good to distinguish a disfavored API style from mathematical unsoundness. There are infinitely many type safe APIs and languages that are, despite their soundness, ugly and hard to use.
Let's not project every dimension along which a language can be good or bad onto the axis of safety.
"Make invalid states unrepresentable" is such a useful adage when designing even basic systems, much less the gates to control your server.
Maybe I'm wrong, but it looks to me like the exact same incentives and means to screw things up exist in C and in Rust for this particular bug.
For example, in a Rust web app, you can choose to use String for encoded URLs, unencoded URL fragments, escaped html, unescaped HTML, usernames, etc. Or you can choose to use newtypes and carefully design all APIs to only take and return the correct newtypes.
The problem is not that you can write bad stuff in rust, the problem is that this particular usage pattern can't be sanely coded in C. This is a public API that should inform users and enforce certain properties. A Results type does exactly that, you would have to go way out of your way to force invalid stuff through that type alone.
But in your version, you can't preserve backwards compatibility, making an infallible function fallible is a breaking change. For something comparable to libc you might want to (eg Rust std still has mem::uninitialized), but even with security issues, I feel like people would be more inclined to deprecate for a new function (or panic, which does have its own problems).
A good type system also means things are written with better forwards compatibility, and can be safely changed later. The Rust equivalent to `crypt` likely wouldn't exist: a low-level library would just provide hash functions, leaving callers to mitigate DOS etc; a broader library might return some opaque handle with equality/serialization, which you couldn't try to compare to a passwordless user; a high-level library would parse the account store itself. None of these really need to change in incompatible ways, they have all the information to add features and fixes at their own level of abstraction.
But if you have a stronger and more expressive type system, you are more inclined to use it, instead of looking for silly workaround like in C.
And that just ends up indicting the C language itself. Can't send errors down a separate channel; folks might not inspect errno. Can't return NULL on error, folks might compare it blindly to another result. No tagged unions, so can't return one. Can't crash on error because there's no scoped error handling. Can't return a struct for checking .error/.result separately, because alignment and equality (and tearing and accidental clobbering and more) make that brittle as hell for widely-used APIs.
C is not a good language for library APIs. Like, really not. It has its uses/strengths, sure, but without even getting into the typical gripes people have about C, libraries--one of the things that it is used for the most--are one of the things for which it is the least well-suited.
"But it has a stable ABI!" So does a land mine.
Another example that C programmers love: using 0 to mean infinity. Eg when you have a timeout setting, and 10 means ten seconds, but 0 means infinity seconds instead of zero seconds.
What Rust calls an enum, but an enum with payloads. In C terms it would be a discriminated union.
> $ret = crypt($pw, $hash); $ret == $hash
musl is actually safe against this use. OpenSSL takes a bizarre code path, detects that $hash is too short, calls crypt($pw, $other_string), forgets that it detected that the hash was too short, and checks $hash == return value.
See my other comment.
The surface-level hints are gone, but the attempt to make every sentence pithy, the decisions on what information to present and what to assume the audience knows, and the over-emotionalization are all still there.
In the past I could get an explanation, where I was able to track the line of thought. Now when I read through the explanation it is like someone has taken two or three explanations and merged them into each other, like there is a combining of iterations without clarity.
example:
The tree is blue.
The tall tree has the color of blue.
the tree, while tall, is blue.
That finally gets squished into: the blue tall tree has height and is evident of blue.
Is this double speak or injected double effort of words?
The ‘concise’ output style setting is bad with Opus 5.5, as it will often spit out a cryptic (is this really English?!), allergic-to-commas concise response, which takes more time to parse.
Except it turns out that at least one of those things is because I didn't enable JS, and the static text is wrong!
(Turning on JS did not actually help, because then the page is filled with horrible, cheesy animations and I can't actually read any of it.)
wtaf ?
musl switched to returning `*` so folks stopped accidentally using NULL results, which was causing crashes and risked false-positives comparing NULL to itself.
That was a misfeature and should never have been implemented. But I bet it was an easy band-aid change to make!
> Upon error, crypt_r, crypt_rn, and crypt_ra write an invalid hash to the _output_ field of their _data_ argument, and crypt writes an invalid hash to its static storage area. This string will be shorter than 13 characters, will begin with a ‘*’, and will not compare equal to _setting_.
That seems orchestrated.
https://lists.llvm.org/pipermail/llvm-dev/2019-June/133308.h...
Of all the things to mitigate...
1) crashing should be an option - it noisily indicates "there's a problem" and if it faults and dumps core, you've got a guided tour of your problem.
2) Is testing NULL as a passable option likely? I'd guess a more likely idiom is:
Wise words. I think early in my career I tended towards coding as defensively[1] as possible in a very specific way of "don't crash", as I definitely was not working in codebases I felt I had a full picture of. "I don't know who calls this method, and I don't want to get a call in the middle of the night that I broke prod" so I would say, return an empty string when asked to format a datetime except I'm passed a string or an int. (Yes, I was working in dynamic, untyped languages, too).
Later on, even in untyped languages, I came around more to "Throw loud and specific exceptions when asked to do things that are nonsensical" - it requires more overall cojones though because it may cause "your code" to crash. But the realization I came to was that it's better to crash ('a MaliciouslyLongPasswordError was uncaught') because "AuthManager asked PasswordService to check a password that was 2MB long" than to paper that over until something like this comes out. Or, the more common situation, you're passed garbage, you return something that won't choke the caller at the time, but what it saves to the database in that scenario ends up being corrupt data. 6 months later someone notices, and has to sort out a data mess that may or may not be recoverable.
[1] note that most "code defensively" advice is good. What I was doing could be thought of as a foolish misinterpretation of the phrase.
[1]: https://www.martinfowler.com/ieeeSoftware/failFast.pdf
[2]: https://en.wikipedia.org/wiki/Fail-fast_system
I dig it.
> The behavior of crypt on errors isn't well standardized. Some implementations simply can't fail (except by crashing the program), others return a null pointer or a fixed string. Most implementations don't set errno, but some do. POSIX specifies returning a null pointer and setting errno, but it defines only one possible error, ENOSYS, in the case where crypt is not supported at all.
In my opinion the problem is that the behavior of crypt is underspecified - it isn't even specified how it signals failure, which is shocking.
Another disturbing decision that crypt's signature takes in the hash its output is supposed to be compared against - if it weren't for that, this half-baked "on error the output won't compare equal to the input" measure that crypt and crypt_* use wouldn't have been possible.
You can't check errno to see if there was a failure because, as the man page says, the value of errno is unspecified when crypt succeeds.
Why * specifically? Why not some random guid generated before committing, if it's so important to not let buggy software crash?
But buggy software might not crash; it might compare NULL == NULL and misbehave, instead.
So there's really no good option here, which is why C is a rancid choice of language for publicly-usable APIs. But we're stuck with it. So it goes.
But issues caused by folks forgetting to check errno, miscomparing NULL to NULL when calling multiple APIs, dereferencing external-API-returned NULL due to missed error checking, mishandling inherited errno after successful action, or--the most common--mishandling negative integer return sentinels meant to indicate errors? Those are incredibly common.
Standard C api calls return an integer, which is 0 on success or nonzero on error. Ignoring it is an intentional choice. `errno` is the hack that shouldn't really exist because it's trivially easy to forget to check that and also has issues with threading (patched by making it a thread local) and more recently, coroutines (which admit no general purpose patch because they're a userspace construct).
Another reason why this is terrible is that it's unclear what the returned pointer's lifetime is. Is it until the next call to crypt? If so, is it thread-local like errno? If it's passed on to the user, is it safe to call free() on it, or is there a crypt_free function that should be called instead?
But all of this, and I mean all of it pales in comparison to returning '*' on error.
100% agree. This is an egregious failure. But C's error-handling story would be bad even without this, and the badness of C's error handling is what motivated the (bad) decision to return *.
It'd be great if we wrote systems software in a language that supported advanced features like...
checks notes
...returning multiple values.
Hell, even in VBA you had to at least spend the keystrokes to type "on error resume next".
So they made the (very bad, but motivated by C's broken design/conventions) decision to return a magic string instead.
0. There are tons of libc functions that return NULL on failure, it is conventional way in C to indicate failure.
1. There's no much to decide, just fail to login if crypt() fails with NULL.
2. errno is standard way in C to check error code of a failing function, but in case of login you hardly need this, just fail!
I totally agree that fixing this issue in musl was a pretty dubious solution. Don't fix what's not broken, fix broken application instead. Publish a CVE if you cannot fix it.
I looked through FreeBSD source code for how they use crypt(), it's checked for NULL everywhere!
UPD: Except libpam. :(
Dude, they documented their rationale. It was dumb and wrong but totally relatable. Irresponsible users were getting crashes, and I’m sorry to tell you that for a lot of library maintainers
> Returning NULL on error is actually an excellent choice because most programs will crash if they try to use it without checking.
…is not something they want to write in responses to bug reports, even though it is quite true. It’s human nature to want to prevent your stuff causing problems for others, even if the problems are fully their fault. Most people cannot and will not be Steve “so don’t hold it like that” Jobs, for fear that their decision will be just as short-sighted as his was.
In this case, that was obviously the dead wrong call. But I’ve been in that seat, and “closed: user error” for the 100th time feels bad to write. Add to that a touch of misplaced cleverness, and you get “hey…what if we just returned an impossible string instead of an error?”
Evil requires motive or malice, and I’m having trouble coming up with those in this case. These patches weren’t concealed or miscategorized a la xz, the behavior was discussed in the open. They were just, in hindsight, really poor decisions made for ordinary reasons. Hanlon’s Razor and all that.
That’s why I’ve been harping on C being trash for shared-code APIs in these comments: bad decisions are gonna happen, and irresponsible users are forever, so can we maybe use tools that require you to put on a seat belt (mandatory error handling) before starting the car?
Agreed, but I can’t think of any other widespread ABIs that are better. Rust’s is not stable both C++ and Rust have concepts that don’t map into other languages well, so using them wholesale is a no go. So it would probably need to be a subset of those. Honestly just tagged unions would probably be enough. Maybe the solution is to get native tagged unions into C…
That's the job of a OSS library author.
You can argue that C has broken design/conventions, but the musl decision was really really awkward.
https://www.underhanded-c.org/
So the knock-on effect is that libraries that do have errors other than ENOSYS to accommodate, have to return magic 'impossible' values instead of setting errno and returning NULL, because of the weight of existing applications that expect nothing but the standard, traditional AT&T Unix (and indeed old BSD), behaviour.
Check the GNU libc crypt() doco. It defines a whole family of 'impossible' values that signal errors, by dint of listing a load of magic characters that should never be in the encrypted string. Including asterisk.
Also check FreeBSD's OpenPAM for a case in point. Even today, it does not check for a NULL return from crypt() and leaps straight to strcmp().
* https://cgit.freebsd.org/src/tree/lib/libpam/modules/pam_uni...
If you want to see real history here, check out the 1992 comment by Eric Andrew Young, that is still in the source for OpenBSD today (in des/des_fcrypt.c), on how SSLeay's crypt() had to be adjusted so that any single character password field, including "*", did not end up successfully matching itself.
* https://cvsweb.openbsd.org/annotate/src/lib/libcrypto/des/At...
It's worth pointing out that musl tried to do the same as SSLeay, and deliberately returns something different to the original salt so that they will not compare equal. It does not return "*" if the input is actually "*" in the first place, contrary to the headlined cartoon explanation. (The locus of the problem is the caller calling it with a salt that is not "*" and then comparing it with "*".)
No. https://man7.org/linux/man-pages/man3/crypt.3.html
My guess is that you are reading the libxcrypt crypt(3) manual: https://www.mankier.com/3/crypt
That's not GNU libc, but frequently deployed on Linux systems.
* https://manpages.debian.org/trixie/libcrypt-dev/crypt.3.en.h...
Although that's a fine distinction for the point being made, as it's still the manual page for the crypt() function that one gets on the operating system, which that other manual page is not.
Damn, that's a bug! Please file bug report.
* https://cgit.freebsd.org/src/tree/lib/libcrypt
As I said, the callers of crypt() have for a long time been using the knowledge that crypt() on their platform simply never returns NULL. Unix has never been returning NULL from crypt() as far back as 7th Edition, when it simply always returned a pointer to a single internal static buffer.
* https://github.com/dspinellis/unix-history-repo/blob/Researc...
The reality is that all C library people have to work with a corpus of Unix applications that do not check for crypt() returning NULL that goes back almost 50 years, and an error return mechanism that barely exists outwith the pages of the SUS.
That includes callers in single-user mode in process 1 in some configurations.
* https://cvsweb.netbsd.org/bsdweb.cgi/src/sbin/init/init.c?an...
* https://cgit.freebsd.org/src/tree/secure/lib/libcrypt/crypt-...
Failing to check result of crypt() in init cannot do much harm, doing so is equivalent of going system reboot (crash init :-)). Besides, whoever has access to system console can do whatever they want with the system, no password can prevent from that.
* https://man.freebsd.org/cgi/man.cgi?query=crypt&sektion=3&fo...
NetBSD's manual is more detailed on the subject, and explains the bind that library implementors have been in for decades, thanks to almost 50 years of applications knowing what crypt() 'really' does.
* https://man.netbsd.org/crypt.3#RETURN%20VALUES
And this despite the SUS trying to shoehorn an error return mechanism into crypt(3) in order to accommodate the problems that operating systems had exporting encryption code, two decades afterwards. (The 1997 GoSolo book explains that this is why there's an ENOSYS error.)
There is simply no universe in which a security-conscientious reviewer looks at the code, spots * being a valid return value, remembers that * is the default value in /etc/shadow for "no password login please" and then is all like:
This is fine, you check for * and return + instead.
There are more 32-long character sequences than atoms in the universe. Pick one, any of them instead.
Of course given the chance that one might do a direct string comparison between crypt's return and the passwd/shadow file's hash field it was still a not well thought out choice.
A typo: `$ret = crypt($pw, $salt)`
OpenSSL??
Not to programmatically parse 'ls' output or some /proc/some/file.
Eg. git started as Perl scripts then moved to C to realy have data straight from C calls from system.
So for example all Internet things calling "openssl -some -options ...." are pure trash :>>
I am also in the camp that CLI tools should return objects that can be parsed (by the shell or otherwise).
[1] https://en.wikipedia.org/wiki/In-band_signaling#Computer_dat...
So, any field de-reference in Java can produce a null pointer exception. Try testing all paths in something like:
try { foo.sideEffect1().sideEffect2()...sideEffectN(); } catch { bar.sideEffect1().sideEffect2()...sideEffectM(); } finally { quuz.sideEffect1().sideEffect2()...sideEffectP(); }
There are M * N * P paths. The only way to write testable code with exceptions is to panic on uncaught exception, and to never catch unexpected exceptions. Even then, you're relying on nothing unexpectedly throwing the exception type you caught.
I'd argue Result<T> >> NULL >> sentries >> exceptions for this reason. Sentries are terrible, but at least they're not computationally hard to use correctly.
And I'm not a fan of breaking control flow, but I still find it preferable to nulls. A null may travel far from the place where it was generated before blowing up, making troubleshooting harder, or it might come from a function you called for its side effect but forgot to check the return value. I gotta say I'm also a careful enthusiast of checked exceptions, it's great knowing which ways a function can fail.
But I think the consensus is growing that Result<T> (plus warnings for unused return values) is simply the superior choice.
That can only happen if you leak the reference (given we are talking about java language level semantics and not byte code), which I give you is unfortunately not that hard to do.
As for exceptions, I think they would pair beautifully with software transactional memory, which would work well with effects. In that case (and no side effects) it would not matter what code path we took, we could just go back to a known safe point.
Because ergonomy-wise I still think exceptions win (even though I do like Maybe types, and Rust's ? macro is plenty good): - they let one focus on the actual happy path code - no error is ever swallowed, unless explicitly written so (looking at you go with useless if errs that fail to properly handle the error), they are always handled somewhere - they automatically "unwrap" the Ok result, and automatically bubble up the Err side. - bubbling up is especially useful because most errors can't be handled at their source (e.g. IO error is meaningless, how you handle it depends on what you want to do in the first place, which is done several layers higher) - they automatically attach a stackframe/source, while with result types you may get a list of Ok Ok Ok Err and have no idea how that Err got there.
Software transactional memory doesn't really work with exceptions, since it precludes the use of catch + finally. It looks more like a top-level panic handler (where the throwing code cannot permute the heap, which is cool). To get exceptions, you'd need nested software transactional memory, which gets into obscure transaction processing theory. There's also the problem that your transaction could side-effect the outside world, then throw and be aborted, so you need to rework how I/O works if you walk down that path.
Exceptions are basically write-only, so I agree that their ergonomics for that use case are pretty good. Panic is better though, since it's really no error handling. Panic is easier to test, and easier to read / understand.
Which is why the pattern of double-checked locking doesn't work.
https://en.wikipedia.org/wiki/Double-checked_locking
That's pretty lame though.
Yes, broad exception catch clauses are simply incorrect code.
Panic is maybe an overreaction, "scratch everything and orderly abort" is sufficient.
Nothing in your post actually supports the idea that exceptions are "computationally hard". The CFG error edges exist however the errors are spelled
So yeah, I guess it either needs to happen at runtime, or we just do signaling and user data on separate file descriptors.
https://en.wikipedia.org/wiki/Semipredicate_problem
If /etc/shadow was a JSON (or XML, even, although that would make me feel gross), this bug would have been much easier to see, because a field that reads '"hash": ""' sticks out a lot* more than a custom microformat that you don't understand.
With data structures, there's never "plain text" - it's all structured. The question is whether you want to use existing formats and tools, or create a new one for every tool and literally make footguns like this.
The elegance of the unix-y approach is actually that it's all bits. You can just join things together and shove the bits through, that's a wild simplification compared with having vendor-specific kinds of plugs. But we could have that kind of compatibility while also benefiting from structure.
So I encourage everybody to give a shell with structured output a try, let it rewire your brain, and then ask yourself: how is the OS holding this back from being everything it could be? Because if we build that OS, it won't encourage these problems.
When I imagine it, it would probe either side of a pipe to see if they support negotiation. If not, send the bits through in the same old unixy way. If so, use the newest schema for structured data that is supported by both sides.
https://github.com/benbernard/RecordStream
I have a bunch of management scripts, etc, that need to run on everything from a potato to a T-1000, and that's the only language that's always available. (OpenBSD sh, FreeBSD sh, busybox, bash, dash, zsh all can cope with it).
I'd love something that's at least JSON-typed (or equivalent), but that can be bootstrapped up on least-common-denominator posix.
Has anyone built something similar? (Yes, I'm being picky -- bun / ts is already a pretty low bar for cross platform support, but these scripts do stuff like install node, so I'm not totally crazy for asking.)
Now the public API is hard to change but you could still avoid the issue by returning a longer token like "==DANGEROUS_CRYPT_ERROR_length==" or "==DANGEROUS_CRYPT_ERROR_unsupported_alg==" which would never match the password field unless someone was really trying to induce the issue.
https://pubs.opengroup.org/onlinepubs/009604599/functions/cr...
>Upon successful completion, crypt() shall return a pointer to the encoded string. The first two characters of the returned value shall be those of the salt argument. Otherwise, it shall return a null pointer and set errno to indicate the error.
returning a sentinel value of NULL and setting errno is pretty standard C, and shouldn't cause an error.
OP's sshhole link specifically notes that it _should_ return NULL, but some MUSL devs decided they wanted it to return trash instead, and the trash they returned instead of NULL caused this.
All because some devs were too lazy to check the result and another dev decided to be clever and "fix" NULL errors from crypt by sending back 'unmatchable' trash banking on devs not checking NULL.
edit: apparently crypt implementations have been trash ignoring the NULL return for a long time. this isn't only on MUSL, but it's still because people ignored and tried to work around the existing out-of-band error mechanism.
But all those sentinels were added for the reason that, when your code is a public API intended to be widely used by who-knows-who, it needs to be really difficult to misuse it. NULL-plus-errno is easy to misuse. Yes, it's a dangerous mistake and your own fault if you do it. But people have been making that mistake for years and don't show any signs of stopping.
The root cause here is that C is fundamentally a rotten choice for public APIs of any kind. I griped more here: https://news.ycombinator.com/item?id=50381800
> OP's sshhole link specifically notes that it _should_ return NULL
I meant there's no mechanism besides returning NULL that would make it hard to ignore the issue (exceptions need to be caught, results need to be unwrapped and their failure handled, an integer return value makes it clearer that there's something there that you need to check). I also missed errno codes when glancing at the manpage earlier which should illustrate the problem.
And yes, whoever first had the brilliant idea to start returning garbage never should've done that, I'm not disputing that.
So if you're strictly adhering to the spec in this case then unfortunately NULL is not an acceptable way to signal a generic error condition. Hence all the seemingly bizarre special values. (In a strange turn of events all those experts doing things that looked weird turned out to know what they were doing. Go figure.)
No error handling scheme can fix that problem. For instance, if a Rust API says I should return a Result<String, ENOSYS>, I can always write code like `if(ENOSYS) { return Ok("*") } instead of returning Err(ENOSYS).
Then my code would be compatible with MUSL.
The only reason this library did the insane thing of returning "*" is because it was extremely easy to cause crashes or false-positives (NULL == NULL) by returning NULL+errno. The reason that was easy is because C lacks a means of making error handling mandatory (exceptions, tuple returns, tagged unions, whatever).
If you return a tagged union, users might mis-handle it, it's true (https://blog.cloudflare.com/18-november-2025-outage/). But they have to handle it, which is a requirement that C APIs are incapable of providing.
Also, in things like rust you can just do result.unwrap() if you want. In general, language constructs don't really save you from bad programmers.
But it crashes at runtime, so such a bug can exist for years with nobody noticing.
The existence of such bugs is what motivated the musl developers to return "*" instead of NULL.
In practice, that's less of a risk than:
Or code for which pointer identity is relevant (e.g. accessing slices/subparts of a data structure): And for sure there are much better ways to write both kinds of code! But people routinely do not do that.To be clear, I strongly prefer Result<> because of the readability, but any dumb thing you can write with NULL + errno can be written with Result (and vice versa).
You can do that but it's going to feel bad to write that and it's going to stand out like a sore thumb to anyone who reads it. But "errno" is just ambient state that's completely invisible unless you happen to have the manpage fresh in your memory.
Here's how to explain it in order to blame openssh:
openssh processes passwords and doesn't want to reveal through timing whether a user has a valid password (or exists at all). So far so good.
What they do if they hit a user with no valid password (i.e. "*") is that they look for another user with a valid password, to use the seed from. A bit weird already, but OK from a perspective of avoiding implementation specifics on seeds and hash algorithms. They know at this point that the result must be to deny access even if the hashed value matches, because that hash is for another user.
So they put the password received over ssh into crypt() with this other user's hashed value to get a seed. And then accept it if it matches, apparently?
Homework: what happens if the password actually matches another user? Can I log in as root if openssh randomly picks my user for the fake salt, and I type my own password correctly?
Also I think it could have returned ":". Given the file format of /etc/shadow and /etc/passwd I don't think it's possible to have a : as a value for anything.
I wonder if somebody else would have thought the exact same thing in a different (non-/etc/*) context, and then you get the same effect in a more obscure manner.
That's a meaningless statement. There isn't a singular "the" issue like you said. Any one of several different technical decisions would have completely prevented the issue. Including, to my point, /etc/shadow being a JSON file where an account being locked is an `"active": false` field instead of an asterisk with no intrinsic meaning.
Some badly designed password hashing algorithms (cough bcrypt cough) have maximum password lengths, ignoring everything after X characters. So it could happen that I set my password to a 100-character string, it gets saved as "==DANGEROUS_CRYPT_ERROR_length==", and then anybody can log in to my account by entering an unrelated long string that returns the same "error".
If a function promises a string result, it should only provide a string if the function succeeded. The null was much better than magic strings.
For historical reasons, we could’ve ended up JSON as “hash”: “*” just the same. The /etc/shadow format lets you skip values completely, like “::”, so your argument doesn’t hold.
Terminating an app because of a NULL dereference is the other footgun. Sadly, we still have “panic” in modern languages like Go and Rust.
For instance, what if your error handling code throws?
One spectacular example of panic saving the day was the Apollo 11 1202 error. It was basically a watchdog timer that fired if the landing computer was starved of CPU time. A misconfiguration caused the first moon landing to hit CPU overload. When that happened, the computer displayed 1202 and rebooted, then ran things in priority order. Apollo 11 landed in the middle of a tight 'panic, reboot, adjust thrusters' loop. Thanks to core memory, most of the heap survived the reboot, so the reboot step was fast enough for it to mostly land without human interaction. The pilot had to make ~1 adjustment to get it to actually hit the landing site, but the panic-ridden computer handled the rest of the descent.
https://dpma.de/english/our_office/publications/ingeniouswom...
Panic is fine, but fundamentally a whole-program crash is undesirable in a really wide range of use-cases. I don't want other SSH connections to crash (or the server to stop accepting valid logins until someone restarts sshd) if one person tries to log in with a credential that triggers a panicking bug in crypt(). Same for crashes in request handlers and so on.
The language/runtime itself doesn't (and probably cannot, at least if it's a general purpose language) distinguish between local and global error conditions, so aborts end up being either unnecessarily damaging, or languages like Go end up normalizing panic "localization"/handling in dangerous ways. Isolation using threads isn't robust enough. Isolation using processes is far too fiddly, slow, and opaque.
The better solution is required error handling. If user code must engage with the possibility of an error state in order to access a result, that's a huge step forward. They can do the error handling wrong, but if you make most people have to do it at all most of the problems will go away.
There are other good solutions too, like Erlang's process boundaries, but those tend to have drawbacks that make them inapplicable to a lot of programming areas. Tagged unions, on the other hand, are only inapplicable in C library code because of language defects and terrible community conventions around ABIs.
"I forgot to check for an error" should not be implicated in bugs in 2026; we have had a large menu of possible alternatives (checked exceptions, unchecked exceptions, tagged unions, etc.) for decades. But we're still over here writing C (and Go), languages pathologically incapable of providing this ability.
This is also why you can upgrade SSH over SSH without having the connection drop mid-stream.
Again: You cannot avoid "I forgot to check for an error". What happens if you get a sigpipe on stdout when printing the "I crashed" line? If Linux overcommits memory? If you turn off overcommit, port all software to check for + cleanly handle NULL from malloc (a substantial effort!), but then get a machine check exception from a double bit ECC error during an infallible read?
It has to: if it did 100% of the post-accept(2) work in a subprocess it would be pretty easy to remotely forkbomb/PID-exhaust a system by spamming invalid logins.
> You cannot avoid "I forgot to check for an error"
Yes you can. Checked exceptions and tagged unions both force calling code to be written with error-awareness.
In SSH's case, "I forgot to check" might lead to crashes (which might knock over sshd so new connections are impossible), or it might lead to false-positives via comparing NULL == NULL.
I'd much rather that boneheaded API users have to write code like
At least that way some keystrokes are required to do the stupid thing. People still will (https://www.schneier.com/blog/archives/2008/05/random_number...), but the harder we can make it, the better.This contains a false snuck premise - that one and only one thing was the footgun. This is categorically not true. An obtuse, poorly-understood, purpose-built text-based DSL is absolutely a footgun and if it was replaced with a JSON and inactive accounts had been indicated by `"active": false` instead of a freaking asterisk then this would absolutely would not have happened.
> For historical reasons, we could’ve ended up JSON as “hash”: “” just the same.
That's dishonest, because it intentionally conceals the fact that that's a
vanishingly* unlikely scenario.People have brains that, factually, are far more likely to see a single context-less character "" and not realize the implications of it than to see `"hash": ""` and think "oh, that's normal".
This is just straight-up psychologically true.
> The /etc/shadow format lets you skip values completely, like “::”
Totally irrelevant.
> so your argument doesn’t hold.
False `::` is both (1) still harder to understand than `"active": false` and (2) even though the syntax existed it wasn't used here because everyone thought they understood the system.
Don't make excuses for this insane text syntax. It is, factually, harder to understand than a decently-written JSON format.
We have seriously overused strings and we're now paying the price.
I can't even get an obvious UX improvement through PAM because they like it broken.
https://github.com/linux-pam/linux-pam/issues/778
Honestly I've been shaking my fist at these since 0 becomes True/Success in Bash and the opposite everywhere else
A function is expected to produce a result - so the question is "what was the result?". !0 (something) is the logical expected default return.
I've never found it to be contradictory.
"False pretends to have an error" is where it gets weird. It's not that they got booleans backwards, it's that they're stretching a tenuous analogy a bit too far.
E.g. in Pascal you can can define a record and then defined file of record to have typed I/O. I.e. it comes with built-in serialization format supported by compiler. Not a good one - but perhaps a step in a good direction.
OTOH C people starving for serialization often believe that `struct` byte layout made by a specific version of a compiler is a serialization format - a view not endorsed by the standard.
Or, for backwards compatibility, start non-hash strings with a character which can't appear in a hash, such as '@'. Your examples would then become '@no_password' and '@error_while_hashing'.
In a sense, they should've changed the parsing logic to return a struct with a field 'password_hash: Option<String>', and defined the grammar of the field to be: "a hash string, or '*' for no login". The bug is, in part, caused by the haphazard way they don't change any formal grammar or contract and just relied on the fact that '' happens to* never be a valid hash.
It's the difference between "happens to work" and "defined to work by construction". The former approach breaks once a hash function is invented where '*' is a legal hash (such as this new musl crypt() hash function where all strings longer than 256 bytes hash to '*'). The latter keeps working regardless because the asterisk is treated as "no password" already at the parser stage.
https://git.musl-libc.org/cgit/musl/tree/src/crypt/crypt_des...
(hey, this workaround is wrong if the salt is "x"!)
but they forgot to work around it here:
https://git.musl-libc.org/cgit/musl/tree/src/crypt/crypt_sha...
The vulnerable path in musl is only a little bit vulnerable, because "*" is not actually a sha256 encrypted configuration and should not reach this code: musl's crypt(..., "*") will not call that code! Turns out that sshd detects that the password hash is too short and does not forward the salt!
https://github.com/openssh/openssh-portable/blob/master/auth...
Look, no else statement! If that if doesn't get taken, then salt remains NULL. And the sshd code is really quite messy and doesn't keep track of the fact that this code path is inappropriate for validating an existing password hash.
A workaround in the sha256 path in musl wouldn't even have prevented the bug!
I'm going to give some credit to musl for unwisely using "*" and some credit to OpenSSH for having nonsensical control and data flow.
When the salt is "x", it'll return "*" and still not match, won't it? The code is correct as long as the second argument is the hash that's being checked. And like you say, the sha512 path isn't relevant since you only get there if the salt started with "$".
> Turns out that sshd detects that the password hash is too short and does not forward the salt!
I think you're misunderstanding the intent of that check. It was added by the 2016 change linked in the writeup[1] and causes OpenSSH's xcrypt() to select and use a salt from a different line of /etc/shadow to prevent timing attacks. It is correct for a crypt() implementation that returns NULL on error.
[1] https://lists.mindrot.org/pipermail/openssh-commits/2016-Jul...
I think you're right. That's what I get for doing a very very quick read.
> I think you're misunderstanding the intent of that check. It was added by the 2016 change linked in the writeup[1] and causes OpenSSH's xcrypt() to select and use a salt from a different line of /etc/shadow to prevent timing attacks. It is correct for a crypt() implementation that returns NULL on error.
I don't care what the intent of the check is. If you're doing a fake password hash to obscure timing, remember that it's fake and don't let the user log in! Don't deliberately pass nonsense into the live check and rely on the nonsense check failing to prevent the fake check from accidentally working. And don't have a code path that thinks it's checking the real password but passes in the wrong second parameter to crypt().
Or, better yet, have an entirely separate function fake_check_password() that does the fake check, returns void, and cannot possibly be confused with the actual password check code.
I think this is the crux. Instead of doing the right thing, they tried to codegolf to get the desired behavior and it got real subtle, real fast.
this is a little flip: maintaining a security-critical program naturally pushes one to minimize code changes because a 30-line change is usually easier to review than a 100-line change. Except when it's not.That would be a much better approach. I see your point now.
Don't return empty string/asterisk/anything else. Instead, return a failure value.
When absolutely needed, choose the magic string with more caution:
One string showing the SHA-256 of a non-existing hash? It's a good idea for it to be a valid hash. Now we have a new risk: if it's broken once, it's broken for all. So it's a good idea for it to be a random one for each machine and/or account. Even this might have problems I can't think of, but it seems to be as secure as an account with a real password.
If this is not a good decision for some reason, choose something unmistakable: "--ACCOUNT-HAS-NO-PASSWORD--". This is much less likely to be mistakenly used in a different context, is much easier to look up in a code base than "*", and is self-documenting.
My AI usage involves telling it to do things and setting up lint rules to catch all the bad programming it’s learnt.
One potential point of disagreement: it's easy to implement structural prohibition of magic strings in low-level languages too if you consider Rust a low-level language (which I've heard before, even though I didn't really consider it to be one for our application of it).
I'm not a Rust expert but do run a few thousand lines of it in prod for some perf/correctness critical code, and I wish it didn't have the reputation of being so difficult to comprehend. There's so much resistance to introducing it or new languages in our large org, that I'm surprised we were able to introduce our thin slice at all. I am not sure I'd use it where you need easy async primitives like goroutines, but it prevents whole swathes of bugs that are repeated over and over again with devastating impact, like the OP's bug, via thoughtful language design.
Maybe this resistance is an artifact of where I work more than large organizations generally, though. Curious about others' experiences. I'm not really an evangelist for the language but would really like to work in a more polyglot environment one day.
I fail to see how the attack complexity of this vulnerability is high, as all it requires is attempting an ssh login with a password longer than 256 characters to an affected alpine image with port 22 open. Am I in the wrong?
[0] https://nvd.nist.gov/vuln/detail/cve-2026-123158
I think for any one of those bugs there might simply be half a dozen footguns or bugs that once it's spelled out you wonder how anyone let it slide.
I think that's the case where LLMs might indentify most.
It's challenging though as the current AI audits produce so many false positives it's hard to know when it's being pedantic or when it's genuinely caught a human being a bone head
It is not really that hard to find in the wild if you had a device built this way, when overflowing buffers on embedded devices is a standard attack vector.
It might be hard to root cause, but it would not be that hard to find.
Each one relies on a perfectly valid argument - that * can't match because we all know it'll be rejected by the other thing.
Granted I mostly work with high level programming languages so I may be missing something.
But that's not a perfectly valid argument, and it never was.
That being said, OpenSSH did mismanage their attempts at handling a garbage return-value from crypt(), so there's definitely some blame to go around.
The whole thing exists because APIs that return a NULL value that "nobody checks", plus a string that you shouldn't use if you actually did check the null value.
In languages with sum types, this cannot happen.
Either you get your hash, or no result at all. You couldn't get eg NULL and *.
But there has to be something in the payload field when it is not applicable. A null is perfectly fine.
The argument for changing that to "*", if I understand the above linked discussion correctly, was that some users of this api threw away errors and a dummy value would be harder to ignore. Maybe one could argue that these behaviours are less common in Rust or Go, but not because of a technical difference.
The world is not perfect and sometimes you really need to work around bad users. Even in Rust or Go. I am not convinced this was such a case. If you are convinced it was it would still have been better to select a less trivial dummy value (such as "*:ERROR:*").
Not in languages with ergonomic sum types. A `Result` or `Either` will have no payload field on the error type. The compiler won't let you dereference it.
I can think of two common patterns that I've seen frequently:
* Return NULL for errors (sometimes with a global errorno() function).
* Return an integer error with any return values as out parameters.
So if it is possible to improve, why doesn't anyone do it?
> * Return an integer error with any return values as out parameters.
You must first standarize all that (usually negative) integers, make docs, check for it, do not make mistakes in your if's or case's...
And errno is just that - integers with a meaning. errno is even thread local now and quite safe to use but people on internet scream loudly when someone try to use that not app global varible :) I guess their llms statistically have less new texts feeded in ;)
At some point it is just better to return pointer to struct with all info you can imagine.
And how you think other languages implemented their Error type returns ? Just a struct or union with tag. Because current CPU's operate on simple type registers there is no other sane way to do it.
the fault for sshhole isn't just "C bad", it's a series of poor choices in workarounds falling together to create a vulnerability.
being condescending about decisions made 50 years ago, before any of the better languages you're mad the developer didn't use even existed, is silly.
if you want to lay blame, lay it at the feet of more recent devs that started returning values instead of NULL because people were using crypt incorrectly. likely sha512 returned a fake value instead of a real one because prior hashes couldn't fail. the interface should have been updated then, but instead they worked around it.
Because in fact all the changes were about changing the function signatures. The function signature is enforced by the linker. The semantic of strings that go through isn't.
Returning 2 values is possible in C (via pointers input parameters). The main issue here is that libc is a strong API contract based on C function signatures, that nobody can change, only augment with new symbols. Such strong API compatibility constraints happen in any programming language, and workaround to fit new behaviours in old APIs happens everywhere.
But also, its not like there were many alternative languages when the POSIX standards were designed so it's fair that it works like this.
- sum/union types
- multiple return values/tuple types
- exceptions
means that most ways of returning an error from a function are not available in C. Because of this, many C APIs, including `crypt` in this case, resort to in-band error signalling by using a sentinel value (like NULL, a negative integer, "*" etc.).
Most other languages lack one or two of the above, but very few lack all three, and so most have developed a different culture around error reporting (Result in Rust, multiple return values in Go, exceptions in Java, Python etc.).
And really often when your "in-band error signalling" in Rust is more expensive you realise (perhaps to your horror) that your C wouldn't be correct, so you just dodged a bullet. For example Option<OwnedFd> is, like OwnedFd, a 32-bit value, Unix has always reserved -1 to mean invalid, so Rust can treat that as a niche. But the Windows equivalent Option<OwnedHandle> can't do this because, well, cue Raymond Chen: https://devblogs.microsoft.com/oldnewthing/20040302-00/?p=40...
But aside from NonZero- and NonNull-based layout optimizations, I don't think Rust is doing anything special to obtain that machine code - it's just relying on LLVM to optimize things.
But sure, for the most part LLVM will do The Right Thing™ here without much help and Rust is just giving better ergonomics to the same backend.
I think with C23 you got half a banana now. Still can't do it because C23 didn't adopt proposals to allow returns of untagged/anonymous structs.
If you could then you could write
However, I also think "weakly typed" is not a well-defined concept, everyone seems to apply it differently. In particular, it seems to mostly mean "playing it loose with types in a way that I don't like".
Implicit conversion could be seen as weak typing, but it is mostly limited to specific arithmetic operations in C. I would see implicit conversions as problematic if they allowed operations that make no sense. Decay of arrays and function to pointer conversion also do not seem problematic to me for this reason.
Void pointers are also weakly typed, but they are specifically designed for this. So if you do not want this behavior, one would simply not use them. It is a bit like using "unsafe" in Rust.
There are certainly stronger typed languages, but on the full range C is more on the strongly than weakly typed side in my opinion. A language that allows arbitrary implicit conversions between pointers, would be weakly typed, or which allows calling functions with wrongly typed arguments (which was true in the past for C a long time ago)
And it's very evident why the difference matters. People gravitate to what's readily available and ergonomic. That's why every C API under the sun (most definitely including libc itself) does janky stuff like returning -1 or "*" on error instead of building their own bespoke Result-style abstraction. A programming language is not just the things that are in some way possible within it, but more importantly the things that are easy in it.
A language should encourage writing robust/secure code, not just make it possible.
In any case, I do not think the issue here is caused by not having a pre-defined result type. Checking for null would have been fine. The issue was that to accommodate old and broken code a compromise was made. But as long such compromises are done even where fixing such old code would be quite easy, it is clear that changing to another language is completely unrealistic. So what are we even talking about? The real issue is that nobody wants to spend money or effort maintaining the old code. No language can fix this.
In any case, you can not change the API of an existing function. But returning NULL for error has required by POSIX would have been just fine.
While not mandatory, a person would probably use it instead of the * hack.
Simply because the calling side wouldn't have an option to forget the check like with the NULL case
- result.unwrap() will panic if it failed
- result.ok() throws away the error but gives you a None value instead (still can’t use it as if it’s a success)
- matching on the Ok variant and doing nothing on the Err variant, you only get to use the success value inside the Ok arm
Etc etc.
The problem is when the compiler says “NULL is a valid instance of T”, which go lets you do, and rust doesn’t (without using the unsafe keyword and intentionally casting a null pointer, which you don’t do by accident.) Go lets you write code like `var hash string; hash, _ = hashfn()` and it compiles and now you have null instead of a valid hash.
This by the way is Tony Hoare's famous "Billion Dollar Mistake"
When later it turned out that musl's procrustian stack size made Python not really runnable under it, it cemented my resolve to not go near musl.
And now we have this.
And why because some dude was complaining sshd or whatever was crashing because it didn't handle nulls? Just handle the null my guy you already know why your bullshit is crashing.
This was not security testing as such - mostly latency, performance and hardcore network "can we break it" and "does it comply" stuff, but also a lot of cable and module / line card pulls, pluggable and fiber breaks, random reboots and pre-empting a lot of possible operator inventiveness. At one point we considered SEU bit flip testing on rented cyclotron time.
And always we battled against the top brass urging us to speed up the process and don't bother with "really basic" testing, because we observed software quality going down over the years as the push for release and time to market stared to dominate.
So we always started with sanity testing - and that included oversized password and CLI input strings. Always. And we caught things more than once, but never like this.
We were right after all.
Exhilarating, isn't it?
It's about maintaining solid engineering practices and far above proving who's right or wrong.
Integration of different software is the stage most likely to introduce uncaught bugs. That is what happened here: musl intentionally broke the documented API of a function used by OpenSSH. Then Alpine integrated this incorrect implementation with OpenSSH.
I was able to reproduce this on Alpine 3.23.7 with OpenSSH_10.2p1.
Luckily PermitRootLogin [0] hasn't allowed logins with passwords for root since 2015 [1]. I'm guessing that's at least part of why this went undiscovered for so long.
0: https://man.openbsd.org/sshd_config#PermitRootLogin
1: https://cvsweb.openbsd.org/diff/src/usr.bin/ssh/sshd_config?...
Am I doing anything wrong here?
OpenSSH_10.3p1, OpenSSL 3.5.9 29 Sep 2026
PRETTY_NAME="Alpine Linux v3.24"
But, got it to work on 3.15 - which requires changing the ! to a * in the shadow file in addition to other steps (making another user with a hash, setting PermitRootLogin to yes, etc.).
https://pkgs.alpinelinux.org/package/v3.16/main/x86_64/opens... Build time 2026-11-05 06:53:46
on 3.21 with the previous version ...
Didn't notice that the openssh-server-10.3_p1-r2 was released today.
I only ever see Alpine Linux used for containers which don't need accounts with interactive user logins, which is not to say it doesn't exist, just that it doesn't seem like Alpine comes with all preconditions met out of the box
> accounts that should never take a password at all (root, your key-only deploy user, ldap users, the default user on alpine's aws and azure images) take any password longer than 256 characters.
But yeah, Alpine for persistant installs isn't unheard of - just not as popular as the container use case. They offer official VM, Xen, Pi, and bare metal images users deploy as well. I actually had a fleet of VMs to update!
Edit: The Alpine notice made it clearer to me that at least one other account has to have a password, it's just the target account that doesn't. So if you truly have no users with passwords in your container deployment then those passwordless users aren't vulnerable to this bug, but any of the image types which have a user will likely have root exposed.
> The attacker needs the name of an account whose password field is * (an account with no password set, which is normal for key-only accounts), and needs the machine to have at least one other account with a $5$ or $6$ password hash.
https://alpinelinux.org/posts/2026-11-05-openssh-password-au...
Also didn't have much luck on Alpine 3.20.10 with OpenSSH_9.7p1 which I've freshly installed from an official ISO.
Last two entries in /etc/shadow:
SSHing into test@ with one 'wrong' password then one 'long' password doesn't work. I have UsePAM no and PasswordAuthentication yes in my sshd_config.I tried adding PermitRootLogin yes and PasswordAuthentication yes. Tried adding another user with a valid password.
Verified shadow entry has a *. Can't login as root or a test user. Tried another incorrect password first as well.
edit: can reproduce after `git revert 384e25e6bd1f2e7b30095900e48b570d46953510` in aports and rebuilding openssh without the fix. It also accepts the password on the 2nd+ attempt, NOT the first.
Microsoft has historically been one of the leaders on the wrong side of this one, but they have also demonstrated how effective it is to call out badly behaved software and make it annoy the users with UAC. When Vista came out, so many applications needed elevated privileges to run because they were badly designed to write user data in system locations, but now their application was more annoying to use and came with a scary dialog about it making changes to your computer that other better-designed software didn't get.
More often than not if a program didn't actually intend to change system settings it was just a matter of badly chosen default paths which are pretty trivial to fix, but so many vendors just would not do it despite years of complaints from NT admins wanting to not have to spend hours with Process Explorer figuring out specific permissions to grant limited users enough access. Once their failures were put front and center, calling their application out as the one getting things wrong every time the user ran it, they suddenly had motivation to fix it.
I suspect that Microsoft has the same history: the entire Windows platform was built with single-user PCs in mind, mostly only dealing with trusted input. The users were also expected to be tech-illiterate for the most part and unable to troubleshoot deep technical issues.
More recently I'm still working on video transmission, but on the production side this time. I also originally adopted the mentality that I should crash early if some unexpected situation was encountered, but I ended up walking that back a lot, because having a critical piece of live media production reboot mid FIFA worldcup final is generally Not Great. Degraded functionality is usually preferable. I still crash early in dev builds, but the production code just logs an error and continues if at all possible.
That said, I definitely agree with you for something like password handling. In this case, I do think it's warranted to crash the program without survivors if something is amiss.
It's hard to define what's security oriented and what is not. For example, is this HN web app security oriented? Not in the way of sshd but it must keep our accounts data separated and it must not let people exfiltrate data from the db, where for example it is storing the emails of some of us.
It's not great, but these days a trickster with AI will find some way to start broadcasting porn to a billion people because of the flaw, so may end up being the better option.
Story time?
100%
Go ahead and program a flight controller this way, I'm sure nothing bad will happen.
Is there a test or check I can use to determine that software is 'good'?
Oh there isn't? Good/bad is subjective? We need to scape goating hacks as bad software, as if anyone is capable of perfect code. Perfect code would require knowing the system inside and out, the stack up and down - no one knows all that, no one is perfect. Sufficiently advanced AI though can rip through all of that, find the weakest spot and zero day it, bringing the entire house of cards down.
Software you thought was good can still be hacked. There is no way to create provably unhackable software.
Which means all software is packed with vulnerabilities just waiting to be discovered, no matter how good you think it is.
Hey, but at least it can’t touch the System32 folder.
Great job guys.
Objective-C ahem nil messaging...
Sadly, we will never learn this because failing fast runs counter to intuition, and a class of person will never override it.
- an account with /etc/shadow hash set to '*'
- (note root won't work by default because of PermitRootLogin prohibit-password default)
- attempt twice, first attempt fails second long password is accepted (that took me a while to understand)
In practice passwd -l seems to lock with a ! prefix, and new account without passwords are created with a single ! which is also not affected, I'm not sure how big the impact actually will be in practice and don't understand the recommendation to upgrade urgently right now... Just don't use `*`?
EDIT: ah, some default container images apparently have a default alpine user with that hash, so I guess that's bad for them. This makes a bit more sense now.
Do you have an example? Looking at Alpine's advisory, I can only see that their cloud images (i.e. what you use to run a VM, not a container) have a default alpine user, no container images. And even with those, cloud images with cloud-init are unaffected with default config due to `PasswordAuthentication no` being inserted by cloud-init into sshd_config. So that leaves... Cloud images that use Tiny Cloud for provisioning. That's an instance bootstrapper authored by Alpine from the looks of it so I'm not sure how many people would prefer that over cloud-init (which a lot of VM/bare metal management software may already make easy to use) or just a non-cloud image.
Well but, alas, a non-zero number of user probably do use it, though and "upgrade X" is a clearer and better instruction to communicate than "change this setting where maybe you don't even know what it means and it might break your workflow, but saves you an upgrade".
A great example is Jedi Academy, a 2003 Quake 3 engine game whose multiplayer is still actively played today (its open-source engine port, openjk, is even in the Debian repos btw). Luigi had two classic scripts for it on his site: Jedi Academy <= 1.011 in-game server buffer-overflow (jamsgbof)¹, which crashed the server just by pushing a string of as past the 1024-byte console buffer, and Quake 3 engine in-game players kicking (q3msgboom)², where a wall of as hit the engine's 1022-char truncation limit and the leftover bytes confused the parser into disconnecting everyone on the server.
Kind of wild that 20 years later, the exact same payload goes from crashing Quake 3 matches from the tilde console to getting root on some SSH servers.
[1] https://web.archive.org/web/20101227181948/http://aluigi.alt... [2] https://web.archive.org/web/20060522175643/http://aluigi.alt...
Were those pictures with links generated using a publicly available tool that we could use, or were it custom (vibe)coded specifically to create this page?
Signatures/verification (typically) only provide a boolean guard on what you want to be controlled behavior.
To properly protect against cla controlled behavior, the check must be incorporated into the EXECUTION of the behavior (with a safe fail).
This is hard and I think a lot of pentesters will continue to have lucrative work hunting issues like this down for a long time.
Fundamentally, the problem is the "type" of the verification and the execution.
Are these people mad? Why on Earth would anybody indicate error like this?
[0]: Be they human or LLM. Or an entity with any proportion of the two. I do not judge nor discriminate.
I mean I understand that in “*sshole” it’s referencing the * that is central to the issue and putting it so it looks like a censored version of “asshole” while also referencing that it’s an ssh hole. And the domain is more simply ssh hole instead of sshole. Is a way maybe it does makes sense to do both. But it did seem at least at first strange to me that the domain is one thing and the name is a slightly different but almost same thing. Like there was a typo when registering the domain, or like the author changed their mind about the name slightly after registering the domain.
However, if someone already developed a strong AI-slop-tingle it will still be triggered by some telltale signs and let their subconsciousness know that they can't possible enjoy this.
This thread highlights why I often say that at some point in the future, the obsession with whether something was written by AI or not will be classified as a mental disorder.
It is tiring to virtually every day see this discussion play out in some HN thread or other. It's also just noise. I don't think your comment, nor the thread it inspired, added anything of value to the discussion.
gregor@krekrm3 ~ % ssh root@192.168.1.1 root@192.168.1.1's password: Permission denied, please try again. root@192.168.1.1's password: Permission denied, please try again. root@192.168.1.1's password: Connection closed by 192.168.1.1 port 22
Its not asking for trouble, it is there will be trouble, not if but when.
As for the people who say "well, I run SSH on port 942329053052". Well guess what, the bots make short work on that. We tried it at $work as an experiment, multiple bots found it in less than a couple of hours. That's why your SSH config must be hardened, preferably running on a hardened OS too.
Did any of those bots brute force a password? Because a 20 character random password is comparable in brute force complexity to an SSH key. The experiment you ran was just as valid for SSH keys on that port, and had no bearing on password vs key.
The point is they found the security by obscurity so-called "hidden" port in less than a couple of hours.
At that point, what they did is irrelevant. The point is that security by obscurity does not work in 2026.
> Because a 20 character random password is comparable in brute force complexity to an SSH key.
You can't be serious with your comment.
No its not.
And even then, SSH key is asymmetric, you're not sending the private key to the server. But with a password you are, which leaves you vulnerable to many attack vectors that simply do not exist with keys.
I feel the opposite, note how most all AI vulns discovered use source code level analysis, empirically keeping source code hidden is more secure.
kerchoff’s principle is misconstrued by a voyeuristic internet that wants to see source code
(almost) No company actually has secret source code, only source code not publicly viewable. And the LLMs crawling those don't belong to people who make public disclosures.
Using LLMs is a great way to empower the governments behind them (US and PRC) to cultivate a garden of undisclosed bugs in your closed commercial codebases.
> You can't be serious with your comment. \n\n No its not.
If you differ in opinion, maybe provide evidence? Especially for a claim like long randomly generated value 1 being absurdly ("can't be serious"-level) weak as compared to long randomly generated value 2
If you mean it's log(62^20,2)≈119 bits of entropy instead of 128+ and so it's theoretically weaker when we build a Dyson sphere around the sun to brute force that key (after somehow obtaining the shadow file, since an online attack would be way too slow), then sure
In other words, conceptually, at the end if the day, isn’t a key just another name for a password, really?
SSH certificates are cool too.
but I guess that the hash is always the same unlike with pub priv key mechanism.
Kind of irrelevant what gets sent to the server and what happens on the server.
Password is already easily defeatable long before it even leaves your computer, thanks to keyloggers and/or password manager breaches and/or old-school shoulder surfing (assisted in the 2026 era by covert filming and AI).
All of which are not vulnerabilities with hardware-backed keys which you don't even need a Yubikey for these days since most modern computers and smartphones have secure enclaves and/or TPMs. (Yubikeys are still great though).
Keys are usually at least 128 bits of entropy, and it's easy to take them up to 256.
Yes, both are a shared secret, but password auth - even with challenge-response - is weaker than key auth exactly because key auth provides more guess resistance.
The best password in the world will still be defeated by a simple keylogger or breach of your password manager.
Worth noting that if you are running a remotely recent version of SSH it has everything you need built-in. No need for Fail2Ban on SSH these days if your config is fully modernised.
What cpu and especially what disk load are you talking about? Also not running out of sockets, not sure if that's RAM-dependent (the box has 2GB)
Well, it's a heck of a lot weaker in this instance (at least when `PasswordAuthentication no`)
Even 5+ years ago, hardware-backed SSH keys were a thing (Yubikey). The only difference today is that its 100% easier to do keys properly because of the widespread commoditisation of secure enclaves in desktops and smartphones to support FIDO, its basically free of charge now because you're not obliged to buy a Yubikey.
Why not ?
Even if you don't want to go all the way to SSH certificates, SSH public-key authentication is laughably easy to do.
I also don't buy the argument "but its on a ULA address". Attacker's old favourite these days is finding a way in (North–South) and then using that as a stepping stone (East–West). Also the general concept of "lower security is OK because its on the LAN" is so 1997.
No excuse for doing that these days.
You can use a Yubikey or you can use your computer's secure enclave or TPM with FIDO at no extra cost.
A password is never, ever "strictly better". Seriously, you're just making shit up now.
I was already pointing out that if you have secure enclave for your private key you should use key-based authentication. That’s what I do on iOS. Otherwise a password is better than your private key in plain text on disk. So much malware steal your private SSH keys (along with your other saved passwords and browser cookies); no malware that I know of performs social engineering and tricks you into giving up your SSH passwords.
It's not just the parent commenter. I'm sure somebody out there is going to remove their account password instead of editing their sshd_config in an attempt to "patch" this vulnerability, actually exacerbating the problem.
You > This particular bug works even with password authentication turned off.
No. The bug is related to password authentication. If you do not have password authentication then you are not affected.
What freaks some people out is they chose parameters where they do have password authentication but they expect to always fail. That's a bad choice, like OK you did fit a cheap manual keyhole, but you threw the keys away thinking now nobody can open that - you realise somebody with a cheap jiggler can open it right ?
Does it ?
Somebody here has already said "You can set PasswordAuthentication=no or update to remediate"
We don't use musl at $work anyway, so I've no means to test.
As in: "Actually out there"
The hypothetical Alpine exists, yes, but what runs that like that?
Possibly public SFTP servers. I have allowed null passwords in the past for anonymous public SFTP and I run Alpine Linux exclusively for my servers both VM's and physical. The shadow package (PAM) did not get installed by default. I installed it but I had to specify it in the installation so there are likely people without it. There is probably a way to find them on shodan.io.
I will activate one for people to play around with for today:
There's no firewall and no password. Hack away HN. cd /pubget so*
Edit: Sure, a small minority presumably run sshd under these conditions. But a small minority times the number of Alpine containers running in the world gets to a large absolute number.
Edit 2: You really don't need to look very hard to find public container examples at million-download scale.
I'm guessing in practice relatively few people are actually exposed. But if they are, it's catastrophic.
It would be super helpful if you would answer a "what, specifically" question with a "this, specifically" itemized list or not answer at all.
This is not helpful.
[1] https://hachyderm.io/@damienmiller/117563026759098135
And like who even needed to plant back doors? If you're the NSA you already had an agent swarm in 1993, but it was made out of dudes from Berkeley and New Haven instead of racks in Prince William County and 印西 (and they were literal secret agents I guess, pretty metal). You can get back doors just by foraging, no need for agriculture.
If you think about it that’s the whole idea of neural nets
We aren’t smart enough to design the function we want by hand as programmers so we need to learn it through weights
This lesson just keeps recurring in many different ways
This isn't even just the software industry, humans are like this about everything. Imagine explaining to something rational that (a) humans didn't realise that they won't be able to competently operate a motor vehicle while watching TokTok dances, and (b) when we finally made that illegal because it's such an obviously bad idea they would keep doing it anyway.
This is why those "What if it builds SkyNet?" conspiracies were so stupid. Why on Earth would you build SkyNet when you can just invent Weeaboo and the idiot humans will die playing ?
There was even a contest for hiding/obfuscating malicious code, e.g. as an honest mistake: https://www.underhanded-c.org/ . And what do you know, the most recent, common approach shown on the page, is: improper comparison inputs that give unexpected comparison results, which is... well... this submission.
If I had a free afternoon, I'd ask a bunch of agents to track the people responsible and their other contributions (but usually the scheme is "contribute valuable patches to build reputation, and only inject an exploit when safe to do so").
> mostly the boring parts where it talks to libc, like the password code
Is clearly not human.
Edit: My bad as pointed below this is not in OpenSSH but in musl and it appears in the relase notes https://www.openssh.org/releasenotes.html#10.6p2. I had scrolled straight to the bottom of the website because the animations annoyed me. Still would like a CVE for proper tracking though!
Never enabled sshd there... except on my PostmarketOS PineBook Pro... oh shit!
Is it mainly that it is too big?
I feel like everybody still loves the unix philosophy, but the previous tools were too bad, systemd just filled user demand for easier tools.
The moment someone makes a pid 1 software that is good and still modular, open and pluggable I think that will be an instant success.
Check out dinit, it's basically the same declarative philosophy as systemd, but without the bloat, and with a much simpler config API. Or s6 if you specifically care about "modular", though unless you're an embedded dev, you probably don't actually want that.
But systemd is not a technical problem, it's Redhat (and IBM) having too much control over what happens to the init system.
And of course there is Pottering and his vile Microsoft stint, then another vile stint to enforce age verification on Linux.
Putting services behind ssh forwarding was the way to secure something you're worried about rather than the other way around, but Wireguard actually makes a good argument nowadays. Iirc they did proofs for the cryptography and it has good design in terms of attack surface, and it's old/mature enough now that I'll say it's probably better than exposing SSH. On the other hand, when this thing running in your kernel gets compromised, you're slightly worse off than with someone logging into a passwordless service account as in the current bug
Its security model doesn’t rely on that fact of course, but it’s the cherry on top.
This might be the beginning of the reason we shouldn't be reading the code. It's like in chess engines where the players complain about how weird, alien and unintuitive the moves are. But 100 steps down the line the engine finds some edge case exploit that gives it enough advantage to win.
So we may be crossing a line, right now, where the non-intuitive alien code being generated by agents is seeing edge cases down the line that might allow exploits. And if we "correct" the agent, we may be leaving ourselves open to exploits we don't see and have difficulty understanding.
The point of my comment is, no amount of "increased understanding" puts a human on equal footing to a chess engine.
You're saying it as if humans can't write that too.
I am arguing that, much like the "best" chess move is unintelligible to best chess players, truly secure code will be unintelligible to humans.
So AFAIU, you are fine.
the sshd/musl part dates to the 2016 CVE-2016-6210 fix, so other * accounts (key-only users, LDAP users) have been open for 10 years.
the website is amazing, I just think these details would clarify things.
ETA: I could trigger it. You need to enable passwords and add a second user with a valid password, then git@ works. Obviously this still means the default forgejo is safe as you say.
https://max.levch.in/post/724289457144070144/shamir-secret-s...
[0] https://en.wikipedia.org/wiki/Eternal_September
Dropbear is unaffected.
> so musl (the libc under alpine) goes ok fine, crypt() just won't return NULL anymore. if it fails, it returns a single character instead.
Considering it's been accepted that crashing if malloc returns a NULL is your fault, did musl really feel the need to accommodate bad implementations like this or is this just AI speculation?
[1] https://git.musl-libc.org/cgit/musl/commit/?id=b3c4cc121f70f...
Maybe I should update my plea to fail loudly (https://alejo.ch/3gk) to list this as an example.
NetBSD and libxcrypt both have similar workarounds. NetBSD has a paragraph on the reasoning.
OpenBSD and glibc both have similar simple, stupid-free return value implementations. OpenBSD: GNU: The distro I use on the desktop uses libxcrypt, though. Probably more distros are. Its, NetBSD's and musl's workarounds probably make sense in some way if you have no clue what goes into your userland and are willing to write stupid software to accommodate for that.I don't know about this specific case but in general? Yes, totally. No one is going to fix all existing software that depends on libs behaving exactly as glibc, you have to accept some trade-offs. Either your new implementation stops being perfectly simple and clean, or no one uses it.
No idea if it's likely to happen though.
1. https://news.ycombinator.com/item?id=50383006
OpenBSD probably did (which doesn't use musl), RedHat probably did (or some other distro using glibc), but they all don't use musl.
This is just negligent. Not a fan of rewrite in rust, but I find it highly unlikely this would happen in that community.
[0] https://github.com/leonjza/sshhole-lab
Edit: It seems to work fine on mobile. I'm not sure why the scrolling behaviour was added on the desktop version of the site
"tl;dr" was a bit of a downgrade though, but that's fine
it's a nice way to promote the company behind it
It starts with a bold claim: "ssh accounts that should not accept a password will accept any password between 256 and 1024 characters". But then it keeps adding qualifiers:
- sshd compiled with musl
- not using PAM
- password logins enabled
- at least one account with '*' as the password "hash"
- at least one OTHER account with a valid password
- server is aligned with 2 ley lines
- hacker is wearing a blue t-shirt and green socks
Ok, the last two are an exaggeration, but still. The conditions are all ANDed, miss a single one and you're fine.
On top of that, as someone else in these comments reminded, the target account needs to have a valid login shell or other exploitable program assigned to it, and these passwordless accounts tend to have nologin or something similar.
To me this seems to fall on the "technically possible, practically very unlikely" bucket, kinda like the PHP vulnerability from the other day. Maybe this has gone unnoticed for so long because it just doesn't happen.
> - sshd compiled with musl
So alpine as well as a bunch of other embedded Linuxes. Huge install base.
> - not using PAM
What you get if you try to have a minimal setup. Default sshd config if you install from source.
> - password logins enabled
The default. Super common.
> - at least one account with '*' as the password "hash"
Almost every Linux system.
> - at least one OTHER account with a valid password
Almost every Linux system.
So sure, there are caveats, but they cover a massive install base.
This is a super bad bug
I'll concede that embedded linux might be my blindspot here. Maybe cheap web hosting, too, but that seems much less likely to be using musl.
This vulnerability clearly is not one that affects everyone like the PHP vulnerability did. But it's exposure is still significant and very easy to exploit. As easy as it gets.
https://openwrt.org/docs/guide-user/services/ssh/transfer_dr...
I haven't thought it through, but I suspect in an ideal world, sshd shouldn't have accepted the user password regardless of the outcome of crypt() if the user did not exist and it only pretended to go through the motions to avoid a timing attack.
Did look into their patch yesterday(?) which seemed to mitigate this issue which they (rightfully IMHO) consider caused by musl. But it seems their fix is to do a sanity check on the supposed hashed password char length, and thus still theoretically still allowing the possibility of those "!!!INVALID_PASSWORD!!!" suggestions to fall through....
On top of the embedded linux blindspot, this statement seems unfounded.
> if one IS running sshd, how often is it with unchanged defaults?
By definition, I would imagine the answer to this is most of the time.
> Almost every Linux system.
But not, notably, on Alpine, which uses '!' instead.
Edit: or maybe yes on Alpine? I checked my desktop and it has zero *s in shadow, but Alpine's post seems to say they do use * sometimes
From man 5 shadow:
> A password field which starts with an exclamation mark means that the password is locked.
> If the password field contains some string that is not a valid result of crypt(3), for instance ! or *, the user will not be able to use a unix password to log in (but the user may log in the system by other means).
- blue t-shirts and green socks have been added to the default loadout for new hackers, since it's optimal according to AI and costs nothing to the attacker in terms of compute
- there are so many ley line "maps" out there that basically any server placement counts as aligned with at least two, so this one's free also
Disabling sshd password logins is security 101, and the default for some cloud provided images these days. If password logins haven't been disabled, what other doors have been left open anyway?
> The default. Super common.
This always seems bonkers to me. I think SSHD should give you ONE password free connection of setup to copy your key over and then it just shuts the door behind it.
It might actually a good idea, but I would likely avoid it since my built-in Bayesian filter would just mark it as spam.
Not sure whether the PHP vulnerability actually had been exploited, but I'm pretty sure there are literally thousands of machines out there affected by this ssh issue right now as we speak.
This isn't going to end well.
---
Addendum (which may or may not be related to the above) --
Everyone has heard of the tale of the boy who cried wolf. The main moral of the story is not to troll your fellow townsfolks so they don't distrust you. But everyone with a sane mind gets that.
The (IMHO) under-appreciated lesson is actually: don't be those villagers. Believe stories you are given, even if you might look like a fool for believing in them. Don't assume what happens is the same as what happened in the past. Look at the facts yourself. Trust and verify.
Sure there are native Alpine installs too, but that is far less common of a use case.
You'd think they'd all do the smart thing and exec a shell into the container's namespace. You'd be wrong.
OpenWRT also doesn't use PAM by default apparently, and there is a risk of people using password login, which you should never do past initial setup.
But outside specific niches, musl is rare, with good reason. Performance is a joke compared to glibc, especially the allocator. The only reason to choose musl in my opinion is for memory constrained devices (OpenWRT etc) or for building static binaries where performance doesn't matter but portability does. I consider the "small container" use case questionable: performance will suffer, and you could share large base layers instead which would amortize the cost over many different images.
Had to quickly scan the rest of the homepage of HN to confirm.
You had me there :D
LLMs will find it.
They work endlessly at finding the 'and' scenarios.
This should have been closed up a long time ago.
What? Pretty sure Cloudflare detected widespread attacks almost immediately after it was published. We are probably lucky that not much more happened because of responsible disclosure and the large media attention led to widespread admin awareness.
The good news is that considering Alpine is open-source, you can be the one to make the judgement call that it is worth fixing and go out there and fix it ;)
expect more of these
At this point I feel embarrassed in front of the AIs.
Can we have a “don’t treat me like five/toddler” / “idgaf about the explanation” quick link to a section for adults who need to patch systems?
(And if your reaction is “just scroll down, stupid”, I’m happy to report that with the scroll jacking, I scroll not enough / past the critical sections nine out of ten times.)
Edit: You may want to read https://sshhole.com/#exposure.
Corollary: “Nobody looks”
The proof of this corollary is left as an exercise for the reader.
You can use \ to escape formatting.
never changed i guess
We need to re-think EVERY assumption about security.
I also found scroll-triggered animations hard to read at first, but these days a lot of sites use them, so I've gotten used to them.
[0] https://news.ycombinator.com/item?id=50377221
Can we please now normalize AI usage in development? Just some other day people asked for "NO AI used" badges for software as measure for quality, and praised i.e. NetBSD for being AI-free. Yes, we come from a phase of AI slop, but it has become obvious that there are plenty of legit usecases where AI beats human-only effort nowadays.
- I found a way to get AI to help me while I made this
- I asked AI to make this
The people you're objecting to are complaining about the latter, this is an example of the former.
Just do HITL (human in the loop). Use an LLM to help you author, but steer the AI in whatever direction you want. It's like having a slightly delusional copy of yourself to pair program with. If you take the time to provide an LLM with some additional skills files and the like, with you in the pilot seat, you can get solid results (it doesn't look as cool as a hands free agent setup but the quality far surpasses anything they can do solo).
1. It is already normalized outside of a tiny, but very vocal bubble that lives mostly on Bluesky (sometimes also on Reddit/HN/Mastodon).
2. I don't think that even the most staunch AI hater would object to using AI for security research as long as all the artifacts (code, docs) that were produced based on the research are human-written. "Yes, there is a bug in my program, but I refuse to fix it, because I know that you found it using AI" would be a pretty weird position. Not sure, how would you defend it.
good fucking god. if I get this correctly... you can insta-own any aws/azure instance that was set up with alpine as long as it has ssh enabled, which it probably has.
Really: from the XZ backdoor in SSH (which went live on some distros) to this...
What's you rationale for calling port-knocking "security through obscurity" and not "defense in depth"?
Put it this way: if I advertize that my server is using port-knocking, that the sequence changes every 30 seconds (like TOTP) and that as soon one sequence is used, it's unusable (so you've only got one attempt to SSH in every 30 seconds, even if you know the sequence)...
Where's the security through obscurity?
But really: even if had musl and PasswordAuthentication allowed (I don't have neither), how would an attacker gets through the port knocking?
More fundamentally the question is: in which way is your system that does not use port knocking more secure than the one who do?
(remember: we're talking about remote exploits by simply having exposed TCP ports open or SSH listening)
Wanna have a talk too about the "solutions" proposed by "security experts" here "because SSH is too old" and that happen to be crufts on top of SSH that had remote root exploitable backdoors? (sorry, holes)
- It affects Alpine's openssh-server (built without PAM), or openssh-server-pam with UsePAM no, when PasswordAuthentication is enabled. Alpine says every release they checked is affected, 3.16 through 3.24 and edge.
- Fixed openssh packages are out for all of those, including the older 3.16-3.20 releases. Upgrade and restart sshd (rc-service sshd restart). If you can't upgrade yet, set PasswordAuthentication no.
- To check a box: apk info -v openssh-server for the version, and sshd -T | grep -iE 'usepam|passwordauthentication' for the effective config.
- Containers built FROM alpine that run sshd are easy to miss in an inventory. So are the official tiny-cloud images (3.18.11 through 3.24.3), which ship an alpine user with no password and passwordless doas, so a bypass there means root.
- After patching, look in /var/log/messages for sshd lines saying "Accepted password for" an account that has no password.
alpine stocks are crashing?
musl stocks are crashing?
glibc stonks are up
alternative implementations in general slightly down
Anticapitalism.