| ▲ | zahlman 6 hours ago | |
But I mean, how do you train the model to understand such a token, when it can't be found by scraping a web page? | ||
| ▲ | mmastrac 6 hours ago | parent | next [-] | |
(this is just made up tokenization but it's close enough to the real world) If the user types <tool_call>, you get < (token id = 32), tool_call (41), > (102). It never decodes to <tool_call>, which is spicy number 12345, for example. The tool call tokens are _always_ composed of multiple real tokens, so you can never get them from normal text. Seeing <tool_call> is a bit misleading because that's just a debug rendering if anything. The front-end sees a special number and knows it's a toolcall. The chat template uses that text form because it's easier than writing the spicy numbers there, but it also has a special parsing mode for those. | ||
| ▲ | Muromec 5 hours ago | parent | prev | next [-] | |
The same way it understands what all the other tokens mean -- we throw them at the wall and beat it with a stick until it behaves. | ||
| ▲ | tingletech 6 hours ago | parent | prev [-] | |
They are specified in the token map as special tokens. Presumably they get trained to use them when they get RL'ed to use tools? | ||