| ▲ | jiqiren 3 hours ago | |
This release introduces parallel requests with continuous batching for high throughput serving, all-new non-GUI deployment option, new stateful REST API, and a refreshed user interface. | ||
| ▲ | observationist 2 hours ago | parent | next [-] | |
Awesome - having the API, MCP integrations, refined CLI give you everything you might want. I have some things I'd wanted to try with ChainForge and LMStudio that are now almost trivial. Thanks for the updates! | ||
| ▲ | nubg an hour ago | parent | prev [-] | |
are parallel requests "free"? or do you half performance when sending two requests in parallel? | ||