I recently just started using Claude/ChatGPT/China models for some PS3 homebrew work.

Every model seemingly falls flat in this scope of programming. The PS3 is very complex and the tooling is fairly undocumented in a lot of instances. It doesn't surprise me most of these AI PR's are nonsense.

If anyone else has attempted writing PS3 homebrew apps using AI and has refined their tooling/systems/automation please let me know how you got the agents to work for you (:

I like to send Claude Code or Codex on max settings off to try a problem in parallel while I work on it.

In a complex codebase it’s funny how often they’ll come back with gigantic commits that just make everything worse or accomplish the goal but have 1000 lines of unnecessary complexity.

Every time they present it with a confident summary. I can see how a junior or just lazy dev would think this is their ticket to becoming a contributor to a repo with some big thing to put on their resume.

I've been working on a project myself over the last few weeks where the documentation is quite minimal. To no surprise the LLMs fell flat at being able to generate any sort of meaningful code. However, I realized that if I focused first on building out documentation and coding tools (linters, parsers, formatters, etc...), LLMs can do a decent job at solving fundamental problems.

Similar for me, but regarding the Classic Macintosh APIs. The difference is that there are plenty of books, and some source code available… just not enough to stop Codex from writing subtly wrong gibberish.

I get the impression that the “10x velocity!!!!” claims still only reflect which areas have a sufficient corpus to learn from, rather than any inductive reasoning.

I have had the same result when trying similar things for graphics on modern consoles. I hear so much great stuff about AI coding but in my niche it just seems to fall flat. Even just rubber ducking around graphics and performance they sound like a beginner who has read a lot of good blogs but with no practical experience.

Why would you expect the LLMs to work for PS3 development? How much PS3 code do you think there is in the training set?

You do realize that’s actually how they work, right? They don’t understand or reason about anything, your prompt and other input is just about trying to guide where the pachinko balls fall in the output.