Remix.run Logo
keeganpoppen 2 hours ago

uhhh i cast doubt on multi-language support as affecting latency. model size, maybe, but what is the mechanism for making latency worse? i think of model latency as O(log(model size))… but i am open to being wrong / that being a not-good mental model / educated guess.

kergonath 37 minutes ago | parent | next [-]

Even model size, it’s modest. There is a lot of machinery that is going to be common for all languages. You don’t multiply model size by 2 when you double the number of supported languages.

make3 2 hours ago | parent | prev [-]

model size directly affects latency