| ▲ | throwdbaaway 12 hours ago | |
That's exactly what I said. They do care when FLOPs are involved. Restoring an old session with 900k tokens will require a lot of FLOPs to reprocess the 900k token. Meanwhile, they don't really care if you use hundreds of millions of cached input tokens, which doesn't consume any FLOP. | ||