Remix.run Logo
ThePhysicist 9 hours ago

I didn't think LinkedIn could get much worse than it was but now 90 % of the posts in my timeline are AI slop full of the usual LLM writing patterns. I'm getting a little paranoid already, but the style really is quite tiring, maybe because it uses hooks and bait so much and buries the lede at the end of the text, as if it was trained to keep human attention for as long as possible instead of just conveying the required information.

I recently tried writing a paper with Claude and it was an absolute disaster, I spent hours (days?) instructing it about writing style and pointing out anti patterns to avoid, but I couldn't get it to even produce simple sentences, it would always add unnecessary lead up sentences, put the most important information at the end of the sentence, use the typical "It's A, not B!" or "B, not A!" type sentences. In the end I gave up and edited everything manually. Makes me wonder how AI can be so smart that it poses a human-level extinction threat but can't seem to even write a simple paper based on facts and information you spoon feed it. I now think it's an intelligence illusion due to the training data and optimization process being hidden from us, essentially it keeps working better and better because we invested massively in optimization of specific use cases like coding, where users contributed billions of training samples that are part of the LLM model. The same is true for text-based workflows and others, the sampling density of the training space is getting much better due to the massive use of AI everywhere so the models extrapolate better between the different instances, but I'd wager they would still miserably fail to generalize to things that are outside of the most common training use cases now. That's why I am also very skeptical about recursive self improvement of these systems, look at what happens when you let agents work recursively / in a loop now, they just keep piling more garbage onto garbage and choke on their own output. I have observed it in my paper writing as well, you feed input into the AI system, the system produces output, the next paper iteration works on that output but the AI doesn't differentiate properly between it's own output and your original input, that pollutes the next output which is then used as input again, eventually the system just churns on its own hallucinated/fabricated outputs until the result is complete garbage that no amount of steering will fix. The same is true for most vibe coded software I built with AI, it holds together decently initially, but the more AI code and decisions accumulate the more the system operates on its own outputs and keeps piling more output on it. More than anything we really need a way to keep system data accurately tagged, i.e. clearly mark human input from AI output and keep AIs churning on output data that it produced itself but treats as input.