You could always stream from SSD storage. Especially effective if you get a cheap old-gen HEDT with lots of PCIe slots to add NVMe storage to and reasonable overall PCIe bandwidth.

That nearly certainly boots you to secs-per-tok land (as opposed to tok/s). Plausible if you are willing to wait hours to days for responses for simple testing, but not (debatably) "usable".