| ▲ | Aurornis 2 days ago | |
> I added -c 4096 to cut down the context size That’s a pretty big caveat. In my experience, using a small context size is only okay for very short answers and questions. The output looks coherent until you try to use it for anything, then it turns into the classic LLM babble that looks like words are being put into a coherent order but the sum total of the output is just rambling. | ||