Is it also a risk to the humans automating it?

If AI works out as the frontier labs hope, humans will not be involved in automating it -- the improvement will also be automated.

So, of course not. Those humans won't exist.