Remix.run Logo
laurentiurad 4 hours ago

You can paste text and immediately see how it's split into tokens, along with token IDs, counts, and other details. It's useful for understanding why prompts cost what they do, why context limits behave the way they do, and why models sometimes split words in unexpected places.

I made it because I kept explaining tokenization to friends and realized there wasn't a simple, interactive tool that focused on learning rather than documentation.

I'd love feedback on:

Features that would make it more useful UI improvements Bugs or edge cases you've found

Hope it's useful!

hk__2 2 hours ago | parent [-]

> I made it

*I asked a LLM to make it

JojiDoesCode 2 hours ago | parent [-]

Honestly this person's whole github pages screams tech grifter