| ▲ | lxgr 12 hours ago | ||||||||||||||||
This should in theory be solveable by using a custom frontend and only using the various backend APIs as stateless inference providers, but everything I've tested falls flat on a few aspects: Chat history RAG and web search, and to a lesser extent tool use. Yes, all of these are theoretically possible (the APIs now all support web search, as far as I know, there are RAG APIs too, and tool use has been supported for a while), but the various "chat" models just seem to be much better at using their first-party tools than any third-party harness, which makes sense that this is what they've been trained on. | |||||||||||||||||
| ▲ | peteforde 12 hours ago | parent [-] | ||||||||||||||||
I've had friends suggest a custom frontend several times, but unless that frontend starts off by faithfully downloading and recreating my entire chat history... now I just have two problems. | |||||||||||||||||
| |||||||||||||||||