They won’t make punctuation mistakes though.
Wouldn't they do exactly that if they were trained on enough text with punctuation mistakes?