Predictive Speculative KV Replication for Bursty LLM Inference

Remix clone Hacker News

new | show | ask | jobs Github

Predictive Speculative KV Replication for Bursty LLM Inference(jwlabs.vercel.app)

21 points by shreybirmiwal 5 hours ago | 2 comments

https://github.com/jwlaboratory/bite-the-bullet

msp26 an hour ago | parent [-]

the hand drawn diagrams and highlights are charming

	▲	shreybirmiwal a minute ago \| parent [-]
		thank you so much