| ▲ | bri3d 7 hours ago |
| Yes, "floating point accumulation doesn't commute" is a mantra everyone should have in their head, and when I first read this article, I was jumping at the bit to dismiss it out of hand for that reason. But, what got me about this is that: * every other Apple device delivered the same results * Apple's own LLM silently failed on this device to me that behavior suggests an unexpected failure rather than a fundamental issue; it seems Bad (TM) that Apple would ship devices where their own LLM didn't work. |
|
| ▲ | sva_ 5 hours ago | parent | next [-] |
| > floating point accumulation doesn't commute It is commutative (except for NaN). It isn't associative though. |
| |
| ▲ | ekelsen 3 hours ago | parent [-] | | I think it commutes even when one or both inputs are NaN? The output is always NaN. | | |
| ▲ | addaon 3 hours ago | parent [-] | | NaNs are distinguishable. /Which/ NaN you get doesn't commute. | | |
| ▲ | ekelsen 3 hours ago | parent [-] | | I guess at the bit level, but not at the level of computation? Anything that relies on bit patterns of nans behaving in a certain way (like how they propagate) is in dangerous territory. | | |
| ▲ | addaon 2 hours ago | parent [-] | | > Anything that relies on bit patterns of nans behaving in a certain way (like how they propagate) is in dangerous territory. Why? This is well specified by IEEE 754. Many runtimes (e.g. for Javascript) use NaN boxing. Treating floats as a semi-arbitrary selection of rational numbers plus a handful of special values is /more/ correct than treating them as real numbers, but treating them as actually specified does give more flexibility and power. | | |
| ▲ | ekelsen an hour ago | parent [-] | | Can you show me where in the ieee spec this is guaranteed? My understanding is the exact opposite - that it allows implementations to return any NaN value at all. It need not be any that were inputs. It may be that JavaScript relies on it and that has become more binding than the actual spec, but I don't think the spec actually guarantees this. Edit: actually it turns out nan-boxing does not involve arithmetic, which is why it works. I think my original point stands, if you are doing something that relies on how bit values of NaNs are propagated during arithmetic, you are on shaky ground. |
|
|
|
|
|
|
| ▲ | danpalmer 6 hours ago | parent | prev | next [-] |
| FYI, the saying is "champing at the bit", it comes from horses being restrained. |
| |
| ▲ | mylifeandtimes 4 hours ago | parent | next [-] | | hey, I appreciate your love of language and sharing with us. I'm wondering if we couldn't re-think "bit" to the computer science usage instead of the thing that goes in the horse's mouth, and what it would mean for an AI agent to "champ at the bit"? What new sayings will we want? | | | |
| ▲ | odo1242 4 hours ago | parent | prev [-] | | chomping at the bit | | |
| ▲ | danpalmer 4 hours ago | parent [-] | | Actually it was originally "champing" – to grind or gnash teeth. The "chomping" (to bite) alternative cropped up more recently as people misheard and misunderstood, but it's generally accepted as an alternative now. | | |
| ▲ | kortilla 3 hours ago | parent [-] | | It’s actually accepted as the primary now and telling people about “champing” is just seen as archaic. | | |
| ▲ | danpalmer 3 hours ago | parent [-] | | Do you have a source on this, or a definition for what it means to be "primary" here? All I can find is sources confirming that "champing" is the original and more technically correct, but that "chomping" is an accepted variant. |
|
|
|
|
|
| ▲ | BeetleB 4 hours ago | parent | prev [-] |
| As a sister comment said, floating point computations are commutative, but not associative. a * b = b * a for all "normal" floating point numbers. |
