I have also tried developing games with them. I find them very weak at these things.

Often they can't figure out how to test the thing, so "does this attempted solution even work at all?" is its guess from reading the source code, and sometimes I've even caught them writing "tests" which are a regex on the source code, not functionality.

They know almost nothing about "is this solving the right problem?", they're trained to assume the user is right, not to frame-challenge.

My general experience both in game and non-game projects is that it will be lazy by default rather than solving things correctly. Sometimes I spot this from their responses, other times I only notice with manual testing.