| ▲ | orbital-decay 13 hours ago | |
Entirely possible, it always leaked but it's particularly bad in most recent models. Actually almost all issues with creative writing in LLMs are artifacts of either instruction tuning, alignment training, or CoT RL and seeding (newer models have CoT data even in pretraining). | ||