| ▲ | FiloSottile 5 days ago |
| Hah, I wrote the crypto/rsa comments. We take Hyrum's Law (and backwards compatibility [1]) extremely seriously in Go. Here are a couple more examples: - We randomly read an extra byte from random streams in various GenerateKey functions (which are not marked like the ones in OP) with MaybeReadByte [2] to avoid having our algorithm locked in - Just yesterday someone reported that a private ECDSA key with a nil public key used to work, and now it doesn't, so we probably have to make it work again [3] - Iterating over a map uses a randomized order to avoid exposing the internals - The output of rand.Rand is considered part of the compatibility promise, so we had to go to great lengths to improve it [4] - We discuss all the time what commitments to make in docs and what behaviors to disclaim, knowing we can never change something documented and probably something that's not explicitly documented as "this may change" [6] [1]: https://go.dev/doc/go1compat [2]: https://pkg.go.dev/crypto/internal/randutil#MaybeReadByte [3]: https://go.dev/issue/70468 [4]: https://go.dev/blog/randv2 [5]: https://go.dev/blog/chacha8rand [6]: https://go-review.googlesource.com/c/go/+/598336/comment/5d6... |
|
| ▲ | mjw_byrne 5 days ago | parent | next [-] |
| The map iteration order change helps to avoid breaking changes in future, by preventing reliance on any specific ordering, but when the change was made it was breaking for anything that was relying on the previous ordering behaviour. IMO this is a worthwhile tradeoff. I use Go a lot and love the strong backwards compatibility, but I would happily accept a (slightly) higher rate of breaking changes if it meant greater freedom for the Go devs to improve performance, add features etc. Based on the kind of hell users of other ecosystems seem willing to tolerate (cough Python cough), I believe I am not alone in this viewpoint. |
| |
| ▲ | wild_egg 5 days ago | parent | next [-] | | Data point of one, but I've been using Go since 2012 and would drop it instantly if any of the backwards compatibility guarantees were relaxed. Having bugs imposed on you from outside your project is a waste of time to deal with and there are dozens of other languages you can pick from if you enjoy that time sink. Most of them give you greater capabilities as the balance. Go's stability is a core feature and compensates for the lack of other niceties. Adding features isn't a good reason to break things. I can go use something else if I want to make that trade. | | |
| ▲ | otterley 5 days ago | parent [-] | | Respectfully, I don’t think you would just pack up and leave. The cost of switching to an entirely different language—which might have even worse backwards compatibility issues—is significantly higher than fixing bugs you inadvertently introduced due to prior invalid assumptions. I’d call your bluff. | | |
| ▲ | wild_egg 4 days ago | parent | next [-] | | That's a bit bold when you know nothing about me, but sure. I exist in a polyglot environment and we use Go for things that we expect to sit and do their job for years without modification. Nothing more annoying with these than needing to update a runtime to patch a CVE and suddenly needing to invest two weeks to handle all the breaking changes. Go lets us take 5 minutes to bump the version number in the Dockerfile and CI configs and move on to more important work. I'm not suggesting we'd go rewrite all of those if Go relaxed its guarantees but we'd stop picking it to write new things in and it would slowly disappear as we decommission the existing services over the years. | | |
| ▲ | otterley 3 days ago | parent [-] | | Every language and its environment has issues. Switching always introduces a new set of problems, some of which could be worse, and many of which you won't have anticipated when you encounter them. |
| |
| ▲ | hn34381 4 days ago | parent | prev [-] | | Also, there is a time and a place for things. Breaking API changes in a minor version update sucks and is often an unexpected time sink, and often mandatory because it has some security patch, critical bug fix, or something. Breaking API changes in a major version update is expected, can be planned for, and often can be delayed if one chooses. |
|
| |
| ▲ | dsymonds 4 days ago | parent | prev | next [-] | | The map iteration order was always "random", but imperfectly so prior to Go 1.3. From memory, it picked a random initial bucket, but then always iterated over that bucket in order, so small maps (e.g. only a handful of elements) actually got deterministic iteration order. We fixed that in Go 1.3, but it broke a huge number of tests across Google that had inadvertently depended on that quirk; I spent quite a few weeks fixing tests before we could roll out Go 1.3 inside Google. I imagine there was quite a few broken tests on the outside too, but the benefit was deemed big enough to tolerate that. | |
| ▲ | ljm 4 days ago | parent | prev | next [-] | | I'd consider stuff like that part of the opinion the language has. Go's opinion is that backwards compatibility at all reasonable cost is a priority. When it comes to ecosystems, the opinions have trade-offs. I would say that Go's approach to dependencies, modules and workspaces is one of those. As a language it mostly stays out of your way, but correcting imports because it pulled in the wrong version, or dealing with go.mod, go.work and replace directives in a monorepo, gets old pretty fast (to the extent it's easier to just have a monorepo-wide go.mod with literally every dependency in it). At least it's an improvement over having to use a fairly specific directory structure though. | |
| ▲ | jerf 5 days ago | parent | prev | next [-] | | Breaking iteration order was also well established as a valid move. Several other languages had already made a similar change, much later in their own lifecycle than Go did. That helps a lot, because it shows it is largely just an annoyance, mostly affecting tests. | |
| ▲ | hinkley 4 days ago | parent | prev [-] | | Java 5 was a fun upgrade for a lot of people because it caused JUnit tests to run in a different order. Due to hashtable changes altering the iteration of the reflected function names. Don’t couple your tests, kids! |
|
|
| ▲ | unscaled 5 days ago | parent | prev | next [-] |
| > We randomly read an extra byte from random streams in various GenerateKey functions (which are not marked like the ones in OP) with MaybeReadByte [2] to avoid having our algorithm locked in You don't seem to do that in ed25519. Back before ed25519.NewKeyFromSeed() existed, that was the only way to derive a public Ed25519 key from a private key, and I'm pretty sure I've written code that relied on that (it's easy to remember, since I wasn't very happy about it, but this was all I could do). The documentation of ed25519.GenerateKey mentions that the output is deterministic, so kudos for that. It seems you've really done a great job with investigating and maintaining ossified behavior in the Go cryptography APIs and preventing new ones from happening. |
|
| ▲ | mkesper 5 days ago | parent | prev | next [-] |
| The nil key case really makes me wonder how sane it is to support these cases. You will be forced to lug this broken behavior with you forever, like the infamous A20 line (https://en.wikipedia.org/wiki/A20_line). |
| |
| ▲ | FiloSottile 5 days ago | parent [-] | | > You will be forced to lug this broken behavior with you forever Yep, welcome to my life. | | |
| ▲ | atsjie 5 days ago | parent | next [-] | | Wouldn't that broken behaviour be a potential security issue by itself? I do remember Go making backwards incompatible changes in some rare scenarios like that. (and technically the loopvar fix was a big backwards incompatible change; granted that was done with a lot of consideration) | |
| ▲ | whizzter 5 days ago | parent | prev [-] | | Wouldn't a nil ECDSA key be a security risk? | | |
| ▲ | unscaled 5 days ago | parent [-] | | If a private key is available, the public key can be derived from the private key using scalar multiplication. This is how ecdsa.GenerateKey works by itself - it first generates a private key from the provided random byte stream and then derives a public key from that private key. I don't see how this can be a security risk, but allowing a public key that has a curve but a nil value is definitely a messy API. |
|
|
|
|
| ▲ | boloust 5 days ago | parent | prev | next [-] |
| Ironically, I once wrote a load balancer in Go that relied on the randomized map iteration ordering. |
| |
|
| ▲ | abtinf 4 days ago | parent | prev | next [-] |
| This is one of the least appreciated aspects of Go. Code I wrote 12 years ago still just works. |
|
| ▲ | gnfargbl 5 days ago | parent | prev [-] |
| As a user of your code this is true, and I'm very grateful indeed that you take this approach. I would add as a slight caveat that to benefit from this policy, users absolutely must read the release notes on major go versions before upgrading. We recently didn't, and we were burnt somewhat by the change to disallow negative serial numbers in the x509 parser without enabling the new feature flag. Completely our fault and not yours, but I add the caveat nevertheless. |
| |
| ▲ | FiloSottile 5 days ago | parent [-] | | We have gotten a liiiiittle more liberal ever since we introduced the new GODEBUG feature flag mechanism. I've been meaning to write a "how to safely update Go" post for a while, because the GODEBUG mechanism is very powerful but not well-known and we could build a bit of tooling around it. In short, you can upgrade your toolchain without changing the go.mod version, and these things will keep working like they did, and set a metric every time the behavior would have changed, but didn't. (Here's where we could build a bit of tooling to check that metric in prod/tests/CLIs more easily.) Then you can update the go.mod version, which updates the default set of GODEBUGs, and if anything breaks, try reverting GODEBUGs one by one. | | |
| ▲ | gnfargbl 5 days ago | parent [-] | | That sounds good. Breaking changes in major version updates is a completely normal thing in most software and we usually check for it. Ironically the only reason we weren't previously bothering in go is that the maintainers were historically so hyper-focused on absolute backwards compatibility that there were never any breaking changes! |
|
|
