Remix.run Logo
gopalv 2 hours ago

> For double/bigint joins that leads to observable differences between joins and plain comparisons, which is very bad.

This was one of the bigger hidden performance issues when I was working on Hive - the default coercion goes to Double, which has a bad hash code implementation [1] & causes joins to cluster & chain, which caused every miss on the hashtable to probe that many away from the original index.

The hashCode itself was smeared to make values near Machine epsilon to hash to the same hash bucket so that .equals could do its join, but all of this really messed up the folks who needed 22 digit numeric keys (eventually Decimal implementation handled it by adding a big fixed integer).

Databases and Double join keys was one of the red-flags in a SQL query, mostly if you see it someone messed up something.

[1] - https://issues.apache.org/jira/browse/HADOOP-12217