| ▲ | perlgeek 18 hours ago | |
Because training a model is expensive, takes a lot of time, and new models need to be evaluated. But you are right: the trend to represent some helpers compactly so that they don't eat up much of your context window, that's all a workaround for a very real limitation: that fully-trained LLMs cannot meaningfully learn from new context and new data. It's a bit like writing super-compact HOWTOs for all the tasks that employees ought to be able to do, instead of properly training new employees. There's a place for that, but it only gets you so far. | ||