I never understood why fixed precision, and integer math isn't more popular. In engineering, we used fixed point all the time, it ran on much simpler hardware and the error is mathematically easy to model. IEEE 754 floats are not only suspect when it comes to theory, but are often outperformed with integers smaller than the mantissa (so less than 24 bits of int can beat a 32 bit float), when it comes to things like loss of precision.

▲

AlotOfReading an hour ago | parent [-]

I recommend pretty much everyone avoid fixed point and other float alternatives, barring exceptional cases after you've done your own numerical analysis, or you lack floating point hardware (rare these days).

Yes, fixed point can use simpler hardware. That's also a completely irrelevant consideration for software. The vast majority of processors are optimized for floats now and some operations (e.g. division) are actually faster.

The precision argument also falls apart. Any float with mantissa >= X+Y can get exactly the same results as a QX.Y fixed point. The float will actually perform better across the same range because you have to round it to perform like the fixed point. That means more precision, lower error, automatic normalization, better overflow behavior, a larger working range, etc. And it'll probably be just as fast, unless you're bottlenecked on memory bandwidth of inputs (unlikely). When you inevitably want an exp() or another special function, it's a heck of a lot easier to call libm than implement your own and it will perform better.

Floats are also much easier to get right for your coworkers that aren't numerical analysts.

▲

torginus an hour ago | parent | next [-]

I didn't recommend fixed point for simpler HW - I recommended it for better precision (if you know what you are doing). First, a point I didn't make, is that if you have 32 bits of fixed, you get way more precision than with a 32 bit float. But I can think of a pretty common case where a 24 bit int would win against a 32bit float: convolution filters. If you have a filter whose inputs are supposed to sum up to 1 (which is the most common case), integer computations mean that, even with internal overflows, the end result will be correct. In contrast, with floats, you can lose precision. If you apply said operation 10000x recursively (say, you are 'stepping' a simulation), those errors can add up bigtime.

> Floats are also much easier to get right for your coworkers that aren't numerical analysts.

That one is true, however, when you have people, such as EEs who really care about precision, and know the theory behind it, then floats are often not the obvious choice. It has other advantages, like your calculation running the exact same regardless of CPU and/or compiler, which I'm sure a lot of analysts care about. Afaik finance people don't even use floats for things like account balances, because you can't represent something like 0.1$ exactly.

Fixed point has basically no language support, and is very hard to get right, but sometimes you need to do that.

Do you have any subject matter expertise in quantization errors? Like doing simulations or DSP work? Not trying to be antagonistic, just figure out where you're coming form.

▲

Asraelite 33 minutes ago | parent | prev | next [-]

> The vast majority of processors are optimized for floats now and some operations (e.g. division) are actually faster.

This seems backwards. Hardware is optimized for floats because people use floats. If people used fixed point, hardware would become optimized for that instead.

Given an equal number of transistors, I'm pretty sure fixed point would be a lot faster on equally optimized hardware for almost all operations.

▲

hilariously an hour ago | parent | prev | next [-]

fixed-point provides uniform precision, exact integer-scaled arithmetic, is deterministic whereas floating point is more convenient but its not a panacea

	▲	AlotOfReading an hour ago \| parent \| next [-]
		As I said, floats can provide results that are no worse than a specified fixed point type. So if you want uniform absolute precision, just round down to the required precision. Floating point is generally deterministic in practice with a fairly minor amount of effort, the major remaining issue being library rounding. I actually wrote a library that guarantees this for arbitrary code, with some small, obvious caveats like standard library precision. And the conference talks linked above note, the standard library issues are an increasingly solved problem for modern toolchains. The remaining cases are mostly things you won't do in fixed point. Let me know if you're aware of anyone computing erfc in fixed point for determinism though. I'm not saying there aren't any situations where other systems are justified, but you probably won't know if you fall into any of them without the kind of numerical analysis that most codebases will never receive.
	▲	mananaysiempre 21 minutes ago \| parent \| prev [-]
		Fixed-point arithmetic provides uniform absolute precision; floating-point arithmetic provides almost-uniform relative precision, as used by scientists and engineers for literally centuries (“significant digits” except the binary version is more uniform than the pencil-and-paper decimal one). Hardware floating point on CPUs (including SIMD units) is almost always IEEE 754 compliant these days (excepting only IBM’s weird fantasy land), and there the rules for the non-YOLO operations (+, -, , /, sqrt, fma) are completely unambiguous and deterministic: treating the inputs as exact, compute the mathematically exact result, then either return it as is if it is exactly representable as a floating-point number, or if it’s not then round it to one that is according to the current rounding mode. Things that can mess this up: - GPUs just do whatever they feel like will make them look faster on benchmarks, don’t count on anything. - Transcendental functions (exp, sin, etc.) are really* hard (multiple literal PhDs) to implement according to the rules I’ve just described (“correct rounding”), so you’ve only been able to get such implementations like this year, and I believe no stock libm has completely switched so bring your own if you need them. - Decimal-to-binary and binary-to-decimal conversions, by contrast, are not that hard to implement according to the rules in principle (it’s making them fast that’s difficult), yet Microsoft couldn’t get it right for literal decades, so if you need Windows then double-check CRT versions and bring in well-known open-source conversion code as necessary. - Denormal inputs or outputs are very slow in some implementations, leading to a hardware option to flush them to zero. Either make sure to not produce them or keep an eye on the option. - The precise bit pattern of the NaNs you get for invalid inputs may differ across platforms. Either make sure to not produce them (you really shouldn’t) or canonicalize upon de/serialization. - Sometimes compilers will try to HALP by performing e.g. single-precision math in double-precision accumulators and only rerounding upon store to memory; by fusing * followed by + (two roundings) to hardware fma (only one); by reassociating; etc. Take care to prohibit your compiler from doing these shenanigans (no -ffast-math or -funsafe-math-optimizations ever, in your code or in any dependencies, and God help you if you’re on MSVC). - Most shamefully, the 8087 (despite spawning the entire IEEE standard in the first place) tried to HALP by using 80-bit registers, so if you need x86-32 then be especially careful with compiler settings (I seem to remember the HALP mode might even be ABI-mandated on some 32-bit platforms so you’ll need to violate that). The concept of floating point is solid, the IEEE standard is stellar, but the superstructure around it is just—not, requiring an unnecessary amount of vigilance to just make it work as designed.

▲

an hour ago | parent | prev [-]

[deleted]