The one thing that makes a modern computer faster than an 80s computer is cache. Without cache, your computer has to go to the memory bus to fetch every instruction and memory read or write, and your system will wait to get the bytes back before it takes any action. You end up at the performance level of an 80s computer.

So you replace the CPU with a faster one with built-in cache. CPU ends up with its own private copy of the RAM and ROM sitting in its cache. But that's not the end.

Computers have a memory map, memory bank switching, memory-mapped IO, and other things to consider. The CPU with its cache has to be kept in sync with the actual memory map of the system. Both the CPU and any memory mapping hardware need to be kept in sync with each other. Memory-mapped IO reads and writes need to go to the actual memory bus at native bus speed.

Then you're left with the issue of other devices that need to access the RAM. This requires cache flushing for writes, and cache invalidation for reads.