You can get such disposable shell in k8s with a command like:

kubectl run -i --tty --rm debug --image=busybox --restart=Never -- sh

Doesn't use ssh though.