| ▲ | keeganpoppen 2 hours ago | |
uhhh i cast doubt on multi-language support as affecting latency. model size, maybe, but what is the mechanism for making latency worse? i think of model latency as O(log(model size))… but i am open to being wrong / that being a not-good mental model / educated guess. | ||
| ▲ | kergonath 37 minutes ago | parent | next [-] | |
Even model size, it’s modest. There is a lot of machinery that is going to be common for all languages. You don’t multiply model size by 2 when you double the number of supported languages. | ||
| ▲ | make3 2 hours ago | parent | prev [-] | |
model size directly affects latency | ||