Tampering with pretraining is as simple as uploading a few pages to github. See pliny