| ▲ | cj00 2 hours ago | |
Invariably when local AI comes up, people try to see how it can be used for frontier-level AI tasks like asking it to one shot a web app from scratch and then they get disappointed when they discover it's not performing at the same level as Opus. But they're ignoring that a local LLM can still be very useful for any kind of repetitive task, things like categorization or generating embeddings, translation or summarization tasks. I'm self-hosting Karakeep and use a local LLM for the AI tasks like embedding. And once you've got that local LLM running, you can start thinking about integrations with your own custom built apps. | ||
| ▲ | sachaa 2 hours ago | parent [-] | |
Also, they are thinking of one single LLM model at a time. What if we have a network of dynamically loaded micro-models loaded on demand based on the user intent? Think of a global DNS of micro-LLMs. We need to decentralize AI. I started building this here https://modelxare.com | ||