Remix.run Logo
michaelmior 10 hours ago

This depends on what your workflow is. There are use cases for tokenization that don't always involve immediately feeding the text into a model.