I could be wrong, but I just don’t see how trying to “detect” LLM generated texts is ever going to work. The only thing that makes any sense if you truly want to have confidence a human wrote it is some type of “proof of work“ system. I think there’s a lot of interesting ways to approach the proof of work problem with different pros and cons, but that is where our energy should be focused if we seriously want to solve this problem.

▲

IshKebab a day ago | parent | next [-]

> I just don’t see how trying to “detect” LLM generated texts is ever going to work

He literally demonstrated a working system in this post. Do you mean you'll never get to 100% accuracy? Clearly, but you don't need that.

▲

40four 9 hours ago | parent [-]

I just mean no matter how hard anyone tries, I don’t see how useful these systems would be in practice. Sure they demonstrated a “working” system. Plenty companies sell products that “work” to one extent or another.

But how useful is it really to get a result of “This is 80% likely chance of being LLM generated”? Or 75%, or 95%? What if the text is a mix of human written text and LLM text? How would you even begin to test that?

I suppose a text that is half human half LLM would theoretically score in the 50% range, but do you see the problem? You can slap a confidence % score on a test run, but interpreting the results leads to a whole other can of worms.

Point is there are so many variables, and it’s not clear that the result from any of the systems is even valid or applicable to help you make a decision in a real life situation.

	▲	IshKebab 7 hours ago \| parent [-]
		Depends on the application. I would love to have that percentage next to HN submissions so I don't waste time reading (or starting to read) obvious slop. Doesn't really matter if I occasionally skip something that isn't actually slop.

▲

alok-g 19 hours ago | parent | prev | next [-]

>> don’t see how trying to “detect” LLM generated texts is ever going to work

... if the assumption is LLMs are being optimized to evade such detection.

PS: I didn't read the actual article.

I think it is instruction-tuning that is having LLMs write differently from humans, and this is not being optimized away.

▲

jaco6 a day ago | parent | prev [-]

[dead]