| ▲ | miroljub 7 hours ago | |
This subsidized inference is just a marketing ploy to increase prices and profit. If common people can have a DIY setup with an open source model cheaper than those behemoths with a scale advantage, it's clear that we have been played. Time to either self host a Chinese open source model or to just pay the cheap Chinese providers. | ||
| ▲ | gibsonsmog 5 hours ago | parent [-] | |
Yeah, local is clearly the future. Even beyond the cheap Chinese models you can install the apfel[1] stuff if you're on a mac and want a quick available onboard cli option. And I'm sure people will adapt the Flash-MoE[2] integration to be even better soon as well. [1] https://apfel.franzai.com/ [2] https://github.com/danveloper/flash-moe | ||