shocking. the code it generated, while technically working, was entirely garbage. i used it for code review and it flagged twenty issues, sol checked the review and found 75% of them were hallucinations. sol was much closer to reality. i no longer trust benchmarks at all because of it.