Remix.run Logo
thayne 4 days ago

IME, one of the places hyrums law shows up the most is in tests.

I've seen tests of various types (unit, integration end-to-end) break because they made assumptions about behaviors that weren't garanteed, and supposedly backwards compatible updates broke them. Here are some examples of things that have broken tests:

- an update resulted in a change in the order of elements in a Hashmap or set.

- a change in an error message (or other user-facing message), changed

- a change in how leap days are handled for datetime arithmetic

- change in the format of locale-specific datetimes

- the timezone offset for a given area

- removal of internal-only APIs that were accessed using reflection

- something performed faster, which revealed race conditions in the testing code

- changing the precise representation of some data format. To give a specific example, changing a single byte when gzip compressing a file, that has no impact on the compressed content.