I'm a security eng and I've worked for both a FAANG and TS government contractor. Neither of them bothered with either of the of the stupid suggestions. IP restrictions prevent roaming, the point of working remotely via SSH. passwords are equally defeated by using an ssh agent, something I'd suggest everyone use. Then on top of that there's no reasonable threat model where something would be given unrestricted access to user env, but also be untrusted. If it can read from ~/.ssh neither IP protection nor keyfile password protection will protect you from maleficence.

The only reasonable response from a security perspective is don't use grok, then use it sandboxed. Trying to claim it's the users fault for not using password protection and IP restrictions is completely nonsensical. Same energy as telling someone their computer is more secure when it's off.