We actually deployed working speech to speech inference that builds on top of vLLM as the backbone. The main thing was to support the "Talker" module, which is currently not supported on the qwen3-omni branch for vLLM.

Check it out here: https://models.hathora.dev/model/qwen3-omni

▲

sosodev 3 days ago | parent | next [-]

Is your work open source?

▲

red2awn 3 days ago | parent | prev [-]

Nice work. Are you working on streaming input/output?

▲

AndreSlavescu 3 days ago | parent [-]

Yeah, that's something we currently support. Feel free to try the platform out! No cost to you for now, you just need a valid email to sign up on the platform.

	▲	valleyer 2 days ago \| parent [-]
		I tried this out, and it's not passing the record (n.) vs. record (v.) test mentioned elsewhere in this thread. (I can ask it to repeat one, and it often repeats the other.) Am I not enabling the speech-to-speech-ness somehow?