I’m surprised at the current sentiment in the comments. Pangram is amazing and has really interesting engineering too. I would have guessed that reliably identifying LLM generated text was not possible without watermarks.

▲

akersten a day ago | parent | next [-]

> I would have guessed that reliably identifying LLM generated text was not possible

It depends what you mean by "reliably." If you mean, "we should be comfortable relying on this kind of tool at scale to identify and punish students, professionals, and writers who may have used AI," absolutely not.

If you take "reliably" to mean "1 in 200 false positive rate" as they disclose on their front page, absolutely that is possible (they are doing it today!). If you think there are more than 200 assignments turned in over a given year at university, you probably do not consider a tool like this fit for purpose. It's an open question whether those procuring said tool are aware of this

Unfortunately their marketing is really insisting on the former, and trying to push it into the zeitgeist that detection of AI-generated or edited text is reliable-type-1 now and long-term. They fail to make it clear that this is merely a tool that strongly suggests text follows patterns known to us at the present time of known LLMs. However, that fingerprint will drift over time, as LLMs get better, human writing style evolves, and the line between human and "smart autocorrect" becomes even blurrier (does speech-to-text push the model into "AI assisted" mode, because it tidied up your punctuation, for example?)

"What color are your bits" is good reading today as it was 20 years ago: https://ansuz.sooke.bc.ca/entry/23

▲

NobodyNada a day ago | parent [-]

> If you take "reliably" to mean "1 in 200 false positive rate" as they disclose on their front page

Pangram claims a 1 in 10,000 false positive rate (rate at which human-authored texts are incorrectly classified as AI-generated). 1 in 200 sounds like the false negative rate (rate at which AI-generated texts are classified as human-authored), or perhaps a rate for a specific category of text.

	▲	akersten a day ago \| parent [-]
		Good point of precision, I read their site too quickly. I don't think my argument materially changes with that number instead however. There were over 40,000 students enrolled in my university alone. Generously assuming they only turn in one assignment per year, having 4 of them go through the "computer says you cheated, and as you can see, it's 99.9% accurate" gauntlet is not a price we should be willing to pay for... the marginal benefit of this tool over more classical ways to proctor and assess pupils

▲

nikcub a day ago | parent | prev [-]

they also publish great tech reports. their founder is so confident in their model that he's regularly on social media offering bounties for false positives

https://www.pangram.com/blog/pangram-4-technical

https://pangram-public.s3.us-east-1.amazonaws.com/pdf/pangra...

https://www.pangram.com/blog/introducing-pangram-image-detec...