| ▲ | swe_dima 5 hours ago |
| It's scary relying on Google's models. I have a very price sensitive workload that used to run on flash 2.5 lite - it's deprecated now. The replacement 3.1 flash lite is a lot more expensive, but now also has a sunset date. 3.5 flash lite is even more expensive. So the price is rising and you have no choice but to keep paying more and more. |
|
| ▲ | rayboy1995 5 hours ago | parent | next [-] |
| I moved directly from 2.5 flash lite to deepseek v4 flash, its already cheaper and if your prompt caching is good you can save so much more money. |
| |
| ▲ | binary132 4 hours ago | parent [-] | | could you explain how to optimize prompt caching or point to a doc about it? | | |
| ▲ | NeutralForest 4 hours ago | parent | next [-] | | Anything Sam Rose is worth reading: https://ngrok.com/blog/prompt-caching but the implementation will be up to your provider and harness, for deepseek, they expose some numbers: https://api-docs.deepseek.com/guides/kv_cache/ and Anthropic has a list of actions invalidating your cache: https://platform.claude.com/docs/en/build-with-claude/prompt... Basically, you avoid anything dynamic: model change, tool change, etc it's also important that your system prompt or main prompt doesn't have non-static data like the date/time/place or someone's name (the person you interact with in a chatbot for example). That should be left to tool call or search. | | |
| ▲ | samwho 3 hours ago | parent | next [-] | | Sam Rose here. Thank you <3 | | | |
| ▲ | arjie 4 hours ago | parent | prev [-] | | I just put the varying parameters in a trailer prompt and have them change every time. It doesn’t matter because the cache is prefix keyed. You lose caching for the last 20 tokens or so but that’s not a big deal. Moving it to a tool call makes it too slow (needs full roundtrip). If you’re constructing the prompt you don’t have to jam everything together you can arrange it appropriately. | | |
| |
| ▲ | apwheele 3 hours ago | parent | prev | next [-] | | Not an open source, but I discuss it in my book with examples for OpenAI/Anthropic/Gemini, https://crimede-coder.com/blogposts/2026/LLMsForMortals. All of the models, you need to have a consistent input to get the cache hit. So if you are chatting with a document, and change the system prompt, it will be a cache miss, even if the rest of the items are all the same. If you even pass in the document in not the same order as the prompts, it will be a cache miss. Or if you add tool calls or structured outputs, it will be a cache miss. (Since those generally go at the beginning of the prompt call, not at the end.) Most of the time when reading documents from URLs directly it will never cache. (Need to typically pass in the bytes directly, or use the provider document store index.) Gemini has a 4096 minimum token size with the 3 version models before even getting a cache hit. OpenAI it is lower (1024), and is automatic, but only happens in increments of 124. Anthropic can also get cache hits at 1024 tokens, but you need to explicit ask for it (and pay extra). Caching by default typically lives for 5 minutes since the last cache hit across providers. But some of them you can ask for longer. AWS for Anthropic models can be tricky with multiple endpoint routing, so can get cache misses if it happens to route to a different endpoint. | |
| ▲ | 4 hours ago | parent | prev [-] | | [deleted] |
|
|
|
| ▲ | ElFitz 4 hours ago | parent | prev | next [-] |
| That’s part of why, since Firebase, I’ve tried to never depend on Google products for business, especially not GCP. Features stay in Beta for ages, whatever that actually means, and released ones get deprecated things fast. Where some of the competitions treats deprecating entire services as "let’s not put it on your frontpage, put deprecation notices all over the doc, and politely ask new users not to start new project with them". |
|
| ▲ | hagen8 5 hours ago | parent | prev | next [-] |
| Just switch the model, its not that much effort tbh. And u can also get a cheaper model than 2.5 lite for the same intelligence |
| |
| ▲ | tacoooooooo 4 hours ago | parent | next [-] | | its not always that simple. dropping in a new model is trivial, but highly specific workflows may rely on specific _invisible_ aspects of a model. when that model gets deprecated, the workflow needs to be rebuilt/re-tuned to work with a different model. google's inability or unwillingness to provide stable timelines for model deprecation makes it risky to build complex workflows using their models | | |
| ▲ | aitchnyu 2 hours ago | parent | next [-] | | Load-bearing (whoops) quirks were noticeable months back, but haven't most flagship models become predictable and reliable? | |
| ▲ | written-beyond 4 hours ago | parent | prev [-] | | 100% agreed in the same boat right now. Feeling really screwed over by Google rn |
| |
| ▲ | ActivePattern 3 hours ago | parent | prev | next [-] | | You would be surprised how much of a difference the model makes for certain niche tasks. For my use case, `gemini-3.1-flash-lite` is ~20% higher accuracy than the next best model of comparable cost (considering both proprietary and open-weight alternatives) | | |
| ▲ | wasfgwp 2 hours ago | parent [-] | | Well it is a bit surprising that 3.1 flash-lite could be better than deepseek-v4-pro (cheaper output and way cheaper cache so might cost less for quite a few use cases). They are not anywhere close according to pretty much every benchmark (even v4-flash is considerably ahead and its way cheaper than flash-lite). Maybe tuning prompts/tools/etc. might be useful? | | |
| ▲ | swe_dima an hour ago | parent [-] | | Gemini flash lite family of models currently has the best ratio for price/speed/intelligence for understanding images, no real alternative AFAIK |
|
| |
| ▲ | deaux 2 hours ago | parent | prev [-] | | "Intelligence" being what, math? Coding? Unfortunately there's a billion use cases for LLMs whose performance is not at all captured by the popular benchmarks they're all trying to maxx. | | |
| ▲ | whimsicalism 2 hours ago | parent [-] | | if you are relying on a model for a business process, it should be simple enough to benchmark on that process |
|
|
|
| ▲ | Cyclone_ 4 hours ago | parent | prev | next [-] |
| They know that there's big enterprises that will have a strong preference to work with another big enterprise instead of relying on a younger company. At least that's why I think they believe they can do this sort of thing and get away with it. |
|
| ▲ | raducu 4 hours ago | parent | prev | next [-] |
| > So the price is rising and you have no choice but to keep paying more and more. I presume you can't use deepseek? |
| |
| ▲ | bradfa 4 hours ago | parent | next [-] | | There are plenty of 3rd party providers hosting deepseek models, if you don't want to use the 1st party API. 3rd party providers are generally slightly more expensive, but still quite cheap compared to other models of similar vintage and size. | |
| ▲ | karolist 3 hours ago | parent | prev [-] | | sadly it's not multimodal |
|
|
| ▲ | 5701652400 4 hours ago | parent | prev | next [-] |
| same here. our production workloads was on Gemini for 2 years. seeing Google unilaterally dropping perfectly fine models and charing you 50x more for worse results is not good. we are switching to Deepseek. |
|
| ▲ | h2aichat 3 hours ago | parent | prev | next [-] |
| Opencode Go is just the same. Each month I will I can do less. Dont ask me why? |
|
| ▲ | thinkingtoilet an hour ago | parent | prev | next [-] |
| All models are increasing in price. Everything up to now has been subsidized by investors, private and public. |
|
| ▲ | pdntspa 3 hours ago | parent | prev | next [-] |
| I'm running price-sensitive data extraction workloads on flash 2.5 and its still the king when it comes to accuracy + cost, all the gemini 3 variants perform a bit worse and cost a lot more. Low-key freaking out, ngl |
|
| ▲ | superkuh 4 hours ago | parent | prev | next [-] |
| I felt the same way about openai's text-davinci-002 and code-davinci-002 (gpt-3.5). They were amazing completion models and openai basically dumped them with no equal cost or equal performance replacement. Instead all their models are opaque with no ability to work in completion mode where one actually controls the text input to the model. These days no company even has completion models where one controls the text input fully. Worthless. |
|
| ▲ | zuzululu 4 hours ago | parent | prev | next [-] |
| same I just switched to OpenAI after using flash 2.5 lite for almost everything at our company. We spent thousands just to build this workflow now Google says screw off |
|
| ▲ | viccis 4 hours ago | parent | prev | next [-] |
| >So the price is rising and you have no choice but to keep paying more and more. You can also just write code like you did a year or two ago. |
| |
| ▲ | anthonypasq 4 hours ago | parent [-] | | [flagged] | | |
| ▲ | Mistletoe 3 hours ago | parent | next [-] | | Can you give examples of other things they can do that would be worth paying for? | | |
| ▲ | 2 hours ago | parent | next [-] | | [deleted] | |
| ▲ | Cyclone_ 3 hours ago | parent | prev | next [-] | | We use it for sentiment analysis of medical data. | | | |
| ▲ | STRiDEX 3 hours ago | parent | prev [-] | | classifying things, grouping things. We use it to help group issues at Sentry. |
| |
| ▲ | viccis 2 hours ago | parent | prev [-] | | Mostly because the person I was replying to has commented about using it to write code. If you're using it for other purposes, then I give you permission to ignore my comment; there's no reason to descend into name calling. | | |
|
|
|
| ▲ | stiltzkin 4 hours ago | parent | prev [-] |
| [dead] |
