| ▲ | jerf 4 days ago | |
As powerful as they are, this is something that I don't think we can trust LLMs with. With the architecture of an LLM, and the fact that at the core there is no such thing as an "out of band" with them no matter how hard you try to put one in, it's intrinsically an arms race, and in the scamming arms race, the scammer side has a loooooot of resources. I've written before about this: [1] You need to think of the scammers as perhaps not hiring PhDs at scale, but making up for it in the ability to just try every possible permutation you can think of and thus making up for the lack of PhDs by leveraging the ability to evolve attacks against the system, and having resources and motivation roughly comparable to at least a company the size and sophistication of Google to do so. They don't need to derive from first mathematical principles a way to figure out how to fool LLMs at a deep neural level... they just need to try a lot of things and then continue in the direction of what works. And they have a track record of good success at fooling full-on human intelligences too, which does not bode well for creating AIs with current technologies that can win against such swarm evolution. I make no strong claims about what future AI architectures may be able to do in this domain, or whether we'll ever create AIs that can defeat the scamming ecosystem in toto (even when the scamming ecosystem has full access to the very same AIs, which makes for a rather hard problem). I'm just saying that LLMs don't strike me as being able to deal with this without some sort of upgrade that will make them not described by "LLM" anymore but as some fundamentally new architecture. (You can of course adjoin them to existing mechanisms like blocklists for sites, but a careful reading of the article will reveal that the authors were already accounting for that.) | ||