over the 15 million lines of code I maintain there are a lot of branches. the cpu can track the most common ones but as soon as the code spills out of cache where does that memory come from?

It doesn’t track all of them, it’s a cache. The ones that were hit long enough ago get evicted, and then if you ever hit them again the processor will indeed have to fall back to a naive static prediction strategy with a high miss probability.