So to be clear, you can't prompt-inject these?

... How did they get into the training data?

mmastrac 3 hours ago | parent | next [-]

They are a special token that you can't just type out in text. It _renders_ as text, but cannot be input into the system as text. The model understands the difference between the textual toolcall token and the "special" one that's really just a spicy number.

▲

zahlman 3 hours ago | parent | next [-]

But I mean, how do you train the model to understand such a token, when it can't be found by scraping a web page?

	▲	mmastrac 3 hours ago \| parent \| next [-]
		(this is just made up tokenization but it's close enough to the real world) If the user types <tool_call>, you get < (token id = 32), tool_call (41), > (102). It never decodes to <tool_call>, which is spicy number 12345, for example. The tool call tokens are _always_ composed of multiple real tokens, so you can never get them from normal text. Seeing <tool_call> is a bit misleading because that's just a debug rendering if anything. The front-end sees a special number and knows it's a toolcall. The chat template uses that text form because it's easier than writing the spicy numbers there, but it also has a special parsing mode for those.
	▲	Muromec an hour ago \| parent \| prev \| next [-]
		The same way it understands what all the other tokens mean -- we throw them at the wall and beat it with a stick until it behaves.
	▲	tingletech 2 hours ago \| parent \| prev [-]
		They are specified in the token map as special tokens. Presumably they get trained to use them when they get RL'ed to use tools?

▲

pyrolistical 3 hours ago | parent | prev [-]

Why can’t user request be spicy numbers so prompt injection is impossible

▲

Muromec an hour ago | parent | prev [-]

It can happen that you would be able to inject them if there is a bug, same way sql injections work.

You have all the same pieces -- untrusted input on the left and the thing that accepts a stream of tokens on the right. In the middle could be anything, especially when vibecoded.