Remix.run Logo
_aavaa_ 4 hours ago

Their plans are still worth it if you use their models. You can see how many tokens you can except to get based on plan here: https://docs.z.ai/devpack/overview#estimated-token-allowance

The max plan will provide ~1,100 USD of GLM-5.3 or ~260 USD of GLM-5.3-flash per month for 168 USD. I can personally attest to these numbers through omp (~97% cache hit rate).

Unless you are able to highly parallelize (your work, you won't be able to hit your hourly or weekly quota using the flash model simply because it's so slow.

They give you ~3x more flash tokens, which maybe comes out to ~2x more actual work after accounting for the extra thinking it does to achieve the same result. The mental model, for not getting angry, is 5.3 is fast mode by default, and you can disable fast mode for 2x the work output at 1/3-1/10th the speed.

They're serving me 5.3 at ~40 tok/s and 5.3-flash at 30 tok/s (according to omp).

Schlagbohrer 4 hours ago | parent [-]

That table assumes cache hit rate of 95% or better. Am I understanding this correctly that people really are doing such repetitive prompts (compared to each other, across the concurrent user base at that time) that only 5% or less need actually be computed by the intended LLM?

That is shocking. Is it per-token I wonder?

workbreak 4 hours ago | parent | next [-]

Every tool call is essentially entire prompt so far sent again with the response and that's why cache rates are so high for agentic workloads. This really bites when using expensive models since most models are 1/10 for cached input.

_aavaa_ 4 hours ago | parent | prev [-]

If you are using their coding plan for coding, then yes you can easily hit such cache rates, with a good harness.

I’m getting 97%.