| ▲ | calgoo 3 hours ago | |
So in my limited experience: The smaller the model, the bigger the harness. The biggest issue becomes the context window. For big models you can kind of just give it bash access and let it run... while with the smaller ones you need to fully manage the context in each LLM call. If you can ask the model for a specific function; with a spec design (typed languages help too) then the small models are great! I have had good progress with generating small python modules for example, but you need verification rounds to catch issues. So test driven design + a good spec sheet + a very detailed todo.md (or even better if its todo.json because then the LLM does not need to manage it, you do from the harness) is your best bet for small models. | ||