> I would have guessed that reliably identifying LLM generated text was not possible

It depends what you mean by "reliably." If you mean, "we should be comfortable relying on this kind of tool at scale to identify and punish students, professionals, and writers who may have used AI," absolutely not.

If you take "reliably" to mean "1 in 200 false positive rate" as they disclose on their front page, absolutely that is possible (they are doing it today!). If you think there are more than 200 assignments turned in over a given year at university, you probably do not consider a tool like this fit for purpose. It's an open question whether those procuring said tool are aware of this

Unfortunately their marketing is really insisting on the former, and trying to push it into the zeitgeist that detection of AI-generated or edited text is reliable-type-1 now and long-term. They fail to make it clear that this is merely a tool that strongly suggests text follows patterns known to us at the present time of known LLMs. However, that fingerprint will drift over time, as LLMs get better, human writing style evolves, and the line between human and "smart autocorrect" becomes even blurrier (does speech-to-text push the model into "AI assisted" mode, because it tidied up your punctuation, for example?)

"What color are your bits" is good reading today as it was 20 years ago: https://ansuz.sooke.bc.ca/entry/23

> If you take "reliably" to mean "1 in 200 false positive rate" as they disclose on their front page

Pangram claims a 1 in 10,000 false positive rate (rate at which human-authored texts are incorrectly classified as AI-generated). 1 in 200 sounds like the false negative rate (rate at which AI-generated texts are classified as human-authored), or perhaps a rate for a specific category of text.