Remix.run Logo
▲ sho_hn an hour ago

I've tried here and there, but even the top-end models always tend toward shallow generalist takes. I've not been able to get more than basic primers out of LLMs, and nothing close to the ability of a professional author to stay on course with an idea, detail level, specificity etc.

People in the early days used to often whine that LLMs just regurgitate text snippets (unfounded of course), but I think the way we currently train and RLHF them actually seems to largely make them unable to reproduce the knowledge they have been trained on, since they seem to just always want to please the mean with their output. I'm oversimplifying the mechanisms, but you get my drift.