I've thought the same, but as I really started to play and compose some music, it became clear that we almost exclusively use 5-7 note systems within a part of a music piece, and accidentals give great hints about what's going on musically. There is no need to use whole 12 semitones on the staff all the time

The problem with linnstrument and similar things is exactly equality of any note, it's not easy to think like "ok, here we move to another key, so we have to add one sharp here" on a controller like that. For me, even on a guitar I have to "see" the notes to play well and understand what I'm playing, so I had to learn "white keys" on the fretboard, and on top of that, learn octave numbers for each note, only then it really works for me and it's still hard compared to the keyboard where you see all sharps and flats and can treat them as you like. That's more important than using the same muscle memory imo