This is true, but the encoding space reserved for future extensions that is interpreted as NOP should be many times smaller than the space for encodings that generate the invalid instruction exception.

The reason is that the number of useful instructions that are only performance hints or checks for some exceptional conditions, so that if they are ignored that does not have bad consequences, is very limited.

For the vast majority of instruction set extensions, not executing the new instructions completely changes the behavior of the program, which is not acceptable, so the execution of such programs must be prevented on older CPUs.

Regarding the order of prefixes, Intel has made mistakes in not specifying it initially in 8086 and in allowing redundant prefixes. The latter has been partially corrected in later CPUs by imposing a limit for the instruction length.

Because of this lack of specification, the various compilers and assemblers have generated any instruction formats that were accepted by an 8088, so it became impossible to tighten the specification.

However, what is really weird is why Intel and AMD have continued to accept incrorrect instruction encodings even after later extensions of the ISA that clearly specified only a certain encoding to be valid, but in reality the CPUs also accept other encodings and now there are programs that use those alternative encodings that were supposed to be invalid.

The prefix structure has been enforced starting with the VEX prefixes (which is a lot later than it should have; AMD made a mistake not enforcing more rules around the REX prefix). The legacy prefixes are of course an unfixable mess because of legacy.