This mirrors my experience.

I'd feel bad calling it 'half-baked' though, seeing as they've published some incredibly in-depth posts about how they develop and test the vim mode. (At the time of writing at least, they had a Neovim core contributor on the team.)

So it does surprise me they haven't made switching between panels more seamless. I've had to add a bunch of extra custom keybinds, and tbh I'm still getting tripped up. Feels like I'm constantly reminding myself to switch between different sets of muscle memory.