Remix.run Logo
nijave 5 hours ago

Error messages matching Z.ai GLM I think are the simplest/most compelling. I had Opus 4.8 poke it and it came back with a couple different errors than the article mentions.

Matching the tokenizer is interesting tho

randomblock1 2 hours ago | parent [-]

I find the tokenizers most compelling. That's what the model is trained on, it's an immutable fact of the model and its architecture. You know for a fact that the model is at least related to other models that way. And if a tokenizer is unique / specific to one lab, like GLM's is, it's basically as good as it gets.

Comparatively, you can't be 100% sure that Z.ai isn't able to host some other lab's model (although in this case, the hosting errors still support the GLM theory).

Chu4eeno an hour ago | parent [-]

No, that's not how anything works.

You can finetune an LLM to a new tokenizer by nudging just a few layers (even wildly different kinds of tokens), and there's nothing stopping a lab from using someone else's tokenizer.