Scaling and benchmarking a critical message bus using a new indexing strategy

This is pretty funny to see, because every financial firm interview process I've seen involves some variant of solving a bunch of stuff with min heaps.

Never before has a set of engineers been more primed to solve a problem

▲

soltanov 3 hours ago | parent | prev [-]

Linear scans break at scale. Partitioning by prefix and using pooled 1024-entry blocks is the right move to prevent 2 GB worst-case index bloat.

	▲	hansvm 2 hours ago \| parent [-]
		Yes, they do, but the extent to which that matters varies greatly. Accuracy/time tradeoffs exist (and are moderately common at $WORK right now). If you provably can't do better than a linear scan (and benefit from the increased accuracy from doing so at a business level), you might as well lean into it and choose a dead-simple, CPU-friendly algorithm for your problem.