This would be expected. The corner cases people faced with PHP throughout the decades have been well documented on the internet for eons.

Swift, not so much. It's relatively new. Looking at AI's abilities like an engineer's career span scaled about 10-20x of time makes it make a bit more sense.

It's going to be worse at newer/niche things, intuitively - which is only going to get worse as it "learns" from garbage outputted by other LLMs moving forward.

▲

ChrisMarshallNY 5 hours ago | parent | next [-]

Also, I suspect most "production" Swift –the type of stuff written by seasoned experts– (I just had to add em-dashes ;) is behind closed-source walls.

	▲	vinnymac 5 hours ago \| parent [-]
		No doubt in my mind, a future Apple model will be the best to use for this purpose. They likely have more swift to train on than anyone else, and would benefit directly from more quality apps, rather than the slop flowing into the App Store (>1k app submissions per hour; they claim)

▲

red75prime 5 hours ago | parent | prev [-]

> which is only going to get worse as it "learns" from garbage outputted by other LLMs moving forward

You seem to assume that autoregressive pretraining (and unfiltered behavior cloning, maybe) are the only ways to improve LLM performance.