There was a trend in 80s synthesizers along these lines: Yamaha DX7, Ensoniq Mirage, etc.

They were famously hard to program. The DX7 in particular is known for basically being a preset machine because almost no one could figure out how to build patches with it.

Muscle memory is really important and it's hard for users to build a mental model of the internal architecture if the external architecture doesn't reflect it at all.