Remix.run Logo
cantor_S_drug 6 days ago

There was a writer who in order to get ideas to write about used to cut up words from newspaper headlines and then rearrange them.

In one rearrangement, he got "Son sues father for xyz". That headline came true 2 years later.

lblume 6 days ago | parent [-]

How is this relevant? Modern LLMs don't rearrange words at all in a meaningful sense of the word, and are certainly better than just using random chance to sample tokens.

cantor_S_drug 6 days ago | parent [-]

Indeed it is exactly that process. They cutup words. Then categorize them based on some metric of nearness (not random). Then link them up. Obviously this process is much more sophisticated than what I have described here.