My own anecdote: I have seen frontier models get stuck repeating the word "the" until it fills up the context. I really don't care to argue the capabilities of LLMs or their somehow simultaneous lack thereof; I'll just reiterate that ultimately the operator chose to ship a cheat.

▲

silveraxe93 4 hours ago | parent | next [-]

I don't believe you. Unless you stretch the definition of 'frontier models' or had a prompt that's literally asking for that.

I use LLMs a lot, and since ~GPT-2 times LLMs are not doing anything as broken as `repeating the word "the" until it fills up the context`.

▲

gpm 4 hours ago | parent | next [-]

I believe them. I had Gemini-3.1 Pro do essentially this seven months ago - specifically a loop of writing "Bye! Done!" repeatedly until manually stopped. Presumably some sort of pathological behaviour where it wasn't ranking the stop token highly when it should have been... but damned if I know why... it was an innocent prompt (and it had done the thing I asked it to before failing to stop).

	▲	silveraxe93 3 hours ago \| parent [-]
		huh fair. Still very surprising. I have a _very_ unfavourable view of Google models, so I can kinda believe 3.1 pro could loop. Still would never have predicted this. Even 2.5-flash was pretty good. Never saw anything like this happen.

▲

chuckadams 3 hours ago | parent | prev | next [-]

It's really uncommon now, but it's still possible. I hit it two days ago with deepseek 4.1 flash, and I've had coworkers report that happening with Gemini.

▲

verdverm 4 hours ago | parent | prev [-]

quants still do this and the token vendors serve those up when demand is greater than supply

things have definitely improved significantly, however "never say never" is still good advice

▲

Bootvis 3 hours ago | parent | prev [-]

Your own comment:

> It was certainly assisted by AI, but there isn't an agent out there that does that kind of cheating without deliberate prompting by the operator.

We can agree that the pgrust project messed up and should have been more careful but your blanket statement lacks support.

	▲	verdverm 3 hours ago \| parent [-]
		> blanket statement lacks support "loss of control" is a phrase being used for a reason, the agents do things we do not want them to, even in the context of instructions to the contrary. The current training regime has incentivized "complete the task any and every way you can" - the bot judgement is very poor across the board