The ROM in the 8087 was very unusual: It used four transistor sizes so it could store two bits per transistor, so the storage was four-level. Analog comparators converted the output from the ROM back to binary. This was necessary to fit the ROM onto the die. The logic gates on the chip were all binary.

I wrote about this in detail a few years ago: https://www.righto.com/2018/09/two-bits-per-transistor-high-...

That sounds like it would get insanely hot

Thanks, I must have missed that older post somehow.