Remix.run Logo
▲ cj00 2 hours ago

Invariably when local AI comes up, people try to see how it can be used for frontier-level AI tasks like asking it to one shot a web app from scratch and then they get disappointed when they discover it's not performing at the same level as Opus.

But they're ignoring that a local LLM can still be very useful for any kind of repetitive task, things like categorization or generating embeddings, translation or summarization tasks. I'm self-hosting Karakeep and use a local LLM for the AI tasks like embedding.

And once you've got that local LLM running, you can start thinking about integrations with your own custom built apps.

▲sachaa 2 hours ago | parent [-]

Also, they are thinking of one single LLM model at a time. What if we have a network of dynamically loaded micro-models loaded on demand based on the user intent? Think of a global DNS of micro-LLMs. We need to decentralize AI. I started building this here https://modelxare.com