| ▲ | thraxil a day ago | |
Not every application uses LLMs the same way. For some use-cases, price per 1M tokens is absolutely meaningful. Eg, we do a lot of pretty basic classification/entity-extraction/summarization type work on large inputs (100k tokens per request being very common). It's pretty easy stuff; Gemini 2.0 Flash was perfectly adequate, quite fast, and cost $0.10 per 1M tokens (and even less when we could make use of the batch API). Every newer more powerful model obviously can handle the same work but costs significantly more. When we're deciding what model to use, price per 1M tokens is definitely a meaningful metric. | ||