Do cpus really track that much about branches? I know JIT does but where does a cpu find the needed memory to store those counters - and them how does reading those not result in a different miss because the cpu can't speculate until it does the if prediction?

last time I checked a cpu documentation they had a simple rule that branches are always taken, that would be easy for the compiler to code order first. However I don't recall which CPU that was. Still this whole thing feels like a citation needed with me being suspicious it is false. CPU designers know this matters and sometimes compilers have information they don't that users care about: they document how it works. (This is the CPU not the instruction set)

Maybe I'm misinterpreting you, but I think you are vastly underestimating both the accuracy and benefit of branch prediction. Yes, CPU's track that much about branches. Your suspicion that this is false is misguided. Here's Dan Luu's 2017 overview: https://danluu.com/branch-prediction/.

Modern cores can have more sram for branch prediction than they do for L1I$.

> last time I checked a cpu documentation they had a simple rule that branches are always taken, that would be easy for the compiler to code order first.

Was that in the 80s? Modern performant CPUs all use dynamic branch prediction.

I’m not really sure I understand the “where does it get the memory” point. Yes, this requires some amount of memory per tracked branch. This memory is hardwired into the CPU, just like all the other memory a CPU needs to do its job (registers, L1 cache, TLB, various in-flight instruction state, etc.)

All of the side channel attacks for CPUs has been from in depth use of branch prediction techniques that make up a significant part of every modern processor’s performance. It’s one of the main things we can do aside from just clocking them higher.