> So if I’m reading this right, ISA can make a difference, but it’s incremental compared to the many architectural decisions and trade offs that go into a particular design.

This is true, but only in the sense that is very rarely correct to say “Factor Y can’t possibly make a difference.”