| ▲ | anonymous_sorry 2 hours ago | ||||||||||||||||
It is a rather odd control. Is it just to confirm that Claude isn't reading the sequence off disk? | |||||||||||||||||
| ▲ | gilleain an hour ago | parent [-] | ||||||||||||||||
Perhaps. They explain why: > Shuffling keeps the letters and destroys their order, so the difference in accuracy measures what the model knows about the sequence itself. So by analogy, if you know a word in English starts with 'Q', you could reasonably predict the next letter would be 'u'. By shuffling, you might encounter the letter Q later on, and not successfully predict a 'u'. Put another way, the shuffled sequences can only use frequency analysis for prediction, while the unshuffled ones have local and global order signals. | |||||||||||||||||
| |||||||||||||||||