Do you know where <||> syntax is coming from? I find it very odd that existing markup wasn't used and there is very little info online about it.

▲

jesdo 7 hours ago | parent | next [-]

To the LLM, the <|...|> tags are single, special tokens. They are rendered as <|...|> tags for readability, but to the model the tags are single tokens.

I do not know why these tokens are visualized as <|...|>, but I speculate that this way they are easily recognizable as special-purpose tokens, and difficult to confuse with other types of existing markup.

▲

zahlman 7 hours ago | parent [-]

So to be clear, you can't prompt-inject these?

... How did they get into the training data?

▲

mmastrac 7 hours ago | parent | next [-]

They are a special token that you can't just type out in text. It _renders_ as text, but cannot be input into the system as text. The model understands the difference between the textual toolcall token and the "special" one that's really just a spicy number.

▲

zahlman 6 hours ago | parent | next [-]

But I mean, how do you train the model to understand such a token, when it can't be found by scraping a web page?

	▲	mmastrac 6 hours ago \| parent \| next [-]
		(this is just made up tokenization but it's close enough to the real world) If the user types <tool_call>, you get < (token id = 32), tool_call (41), > (102). It never decodes to <tool_call>, which is spicy number 12345, for example. The tool call tokens are _always_ composed of multiple real tokens, so you can never get them from normal text. Seeing <tool_call> is a bit misleading because that's just a debug rendering if anything. The front-end sees a special number and knows it's a toolcall. The chat template uses that text form because it's easier than writing the spicy numbers there, but it also has a special parsing mode for those.
	▲	Muromec 5 hours ago \| parent \| prev \| next [-]
		The same way it understands what all the other tokens mean -- we throw them at the wall and beat it with a stick until it behaves.
	▲	tingletech 6 hours ago \| parent \| prev [-]
		They are specified in the token map as special tokens. Presumably they get trained to use them when they get RL'ed to use tools?

▲

pyrolistical 6 hours ago | parent | prev [-]

Why can’t user request be spicy numbers so prompt injection is impossible

▲

Muromec 5 hours ago | parent | prev [-]

It can happen that you would be able to inject them if there is a bug, same way sql injections work.

You have all the same pieces -- untrusted input on the left and the thing that accepts a stream of tokens on the right. In the middle could be anything, especially when vibecoded.

▲

gpugreg 4 hours ago | parent | prev | next [-]

The syntax was specifically chosen to avoid confusion with other tag formats. XML could show up in any user- or LLM-generated text and would mess up the message format. Some variations even used unicode symbols like "｜" or "〈" to decrease the chance of confusion.

Also, back then, some people believed that it could prevent prompt injection, but history has shown that it was a futile attempt.

▲

t_mahmood 7 hours ago | parent | prev | next [-]

Correct me if I wrong, I saw this when llama.cpp or ollama initializes, it is used as token separator of some sort.

	▲	cowlevel 7 hours ago \| parent [-]
		GPT-2 used <\|endoftext\|>

▲

embedding-shape 6 hours ago | parent | prev [-]

The first place I think I encountered it, was GPT-2's tokenizer, back when OpenAI did actual open source stuff, maybe around 2020?

I'm guessing as with most stuff in the LLM ecosystem, someone did it, no one re-evaluated why and just copied it wholesale, and here we are. The most important point, that this sequence of text never occurs in "real text", is the main point of the specific character chosen, and seems to still work good for that purpose.

The original usage is only "<|endoftext|>" I think, which was the EOS token for GPT-2, then more and more special tokens were added as various delimiters to new model releases.