Either the current test suite covers the apis and expected behavior or it does not.

These big ports we are hearing about are of projects with hundreds (prob thousands) of unit tests. Not only are the models typically trained with the original code bases in their dataset, but the test suites themselves have project architecture encoded in them..