Remix.run Logo
Chu4eeno an hour ago

No, that's not how anything works.

You can finetune an LLM to a new tokenizer by nudging just a few layers (even wildly different kinds of tokens), and there's nothing stopping a lab from using someone else's tokenizer.