| ▲ | pancomplex a day ago | |
Agreed this would be super cool and I do see that coming in the future. But Jev-level latency just isn't there yet with full images. | ||
| ▲ | fzysingularity a day ago | parent [-] | |
We just did exactly this - added TypeSafe-compatible API (incl. websocket support) for various VLMs. Latency right now is <250ms, but will be able to get it to <150ms (p95). Take a look at a snake demo with streaming image inputs: https://x.com/spillai/status/2103630735425089957 | ||