Does the Unreal API change a bit over versions? I've noticed when asking to do a simple telnet server in Rust it was hallucinating like crazy but when I went to the documentation it was clear the api was changing a lot from version to version. I don't think they do well with API churn. That's my hypothesis anyway.

▲

toshinoriyagi 4 days ago | parent | next [-]

I think the big thing with Unreal is the vast majority of games are closed source. It's already only used for games, as opposed to asking questions about general-purpose programming, but there is also less training data.

▲

mac-mc 4 days ago | parent | prev | next [-]

You see this dynamic even with swift which has a corpus of OSS source code out there, but not nearly as much as js or python and so has always been behind those languages.

▲

Tiktaalik 4 days ago | parent | prev [-]

There would be significant changes from 4 to 5, but sadly I haven’t had any improvement if clarifying version.

	▲	frankc 4 days ago \| parent [-]
		Clarifying can help but ultimately it was trained on older versions. When you are working with a changing api, it's really important that the llm can see examples of the new api and new api docs. Adding context7 as a tool is hugely helpful here. Include in your rules or prompt to consult context7 for docs. https://github.com/upstash/context7