| ▲ | declanjackson a day ago |
| Hi there - AA employee here. This is due to 3.8 Flash having a much lower cache hit rate compared to Gemini 4 Hydrogen |
|
| ▲ | gruez a day ago | parent | next [-] |
| Why would cache rates differ? Does hydrogen have a longer cache period? |
|
| ▲ | nightshift1 a day ago | parent | prev [-] |
| sorry if this is a dumb question, but how is this possible? what can cause the cache hit rate to change just by changing the model? |
| |
| ▲ | declanjackson a day ago | parent | next [-] | | This is based on what we measure from the API - we see different endpoints depending on how they are served can have different cache hit rates (cache hit rate is defined as the % of cacheable tokens that actually come back as cache hits) | |
| ▲ | scott_weber 18 hours ago | parent | prev [-] | | One possibility (just a guess): new arch produces a more compact cache, it becomes cost effective to hold more cache for longer. |
|
