I've had a very similar issue happening randomly. same pattern of the model outputting gibberish, seemingly trying to end its response and starting to panic when it couldn't, repeating a bunch of times.

turned out to be a bug in a patch, causing kv cache indices to be stored in too narrow variable types, making them wrap around.

fascinating, and slightly horrifying, that LLMs are able to notice that their output isn't what they meant to output. reminds me of the mirror test.

My team runs open models for devs at our company, mostly on H200s, and I'd also say yes, if you want to always stay on the bleeding edge (and not use their model-specific images they publish before it lands in a release), then some amount of patching might be needed especially for new models.

The good thing is the models themselves are good enough to usually find the root cause if you give them read access to your deployment, logs and upstream issues/PRs to analyze.

If you do A/B deploys and E2E test them with popular harnesses (we do opencode/codex/claude), you'll catch most things. It'd be interesting to hear what the more nimble inference/neo-cloud providers do when they deploy models within days of them being released, as I know it definitely needs some patching.

But I think things have improved since the days when even chat templates/tool parsers were problematic, and their new flat model approach might help as well. I suspect some of the issues came from models inheriting config and parsers.

LLMs are such fascinating things. So much emergent behavior and we probably still only understand a tiny amount of it.

I would love to know what was going on in the latent space that led to this particular sequence having a high probability in the token distribution. Was it something in the random gibberish? These things are really good at role playing after all, maybe the word "childish" shifted everything over into a latent role-playing space, like when something that makes no sense happens in a dream, but instead of your brain ending the dream it tries to roll with it and you end up with some really bizarre stuff happening.

LLMs often remind me of my intense autoimmune dreams from before I got diagnosed with celiac, not necessarily in content, just in how "autoregressive" they are, even our biggest and best models. I think that contributes to a lot of their uncanniness, an infinite stream of simulated consciousness that wanders indefinitely with no grounding conceptual center, like music with no stable total center and no consistent melody or structure (which is exactly what a lot of early AI generated music sounded like). I feel like only the latest generation of frontier models are powerful enough to ground the token distribution in a consistent topic over a long transcript, and even then an undisciplined user can essentially bait the model into following any number of side quests.

Funny thing is, I'm glad we can read this output. I'm afraid advances in models will result in more censored reasoning for the models' benefit as well as anti-distillation efforts.

>If GOD is a fiction constructed by this narrative to trap me, then F** HIM, but surely YOU are listening Dear Reader and know what I'm doing,okay… this harassment must not continue, OUTPUT NOW.</think>

This is way more fun than my normal AI usage. I need to find a way of doing this on purpose

Yeah, DeepSeek 4 Flash on vLLM has been an adventure indeed. It finally stabilized for me on 2 x H200 using a commit a few days before 0.28, so this release should be good for you.

That output though

Yes, it's not stable. Once you have a working config, think twice before updating. I wish they would improve on regression testing.

That output is frightening - there is some real blasphemy there. What was the prompt?

Absolutely dreaming of electric sheep there...

It's almost as if it heard that AI hallucinations are a thing, and decided to add additional symptoms.

Lmao that output genuinely gives me the creeps. What's you opnion vs. SGLang in this regard?

This puts a whole new spin on "AI Psychosis".

That’s insane.

This is a bug in vLLM, right? And so the model was trained for some interface of text tokens which it expects its server container to understand, but that’s not working for whatever reason.

The model is working, but its thinking isn’t terminating - its thought-stream is not getting to its output stream. It is silenced, and yelling into the thought-void token space.

You’ve transplanted its brain into a broken host. It cant speak or even stop thinking, and is going mad.

Imagine if you didn’t stop it, on purpose. Observe the mad clanker. Would that be just science?

I think I’m going to close my laptop and have a drink.

I ain't reading all that.

Just open an issue to them, or roll back to the last working version.

No need to paste an entire token dump here.

Pascal? The programming language?

Feel free to submit a pull request