Makes me wonder whether it's a fundamental limitation of the technology or more like missing a good set of tasks in the training pipeline

Might be that the vast majority of their training data is poor writing.