> Migrating a test suite is exactly the kind of work that LLM's excel at beause it's extremely easy to verify

I don't think that's true. All you see is all the test pass. You don't know that the tests still cover everything that they used to.

LLMs excel where there is an excellent test suite, and you ask them to modify the thing that the test suite tests, and forbid them from changing the tests.