Have you tried putting known human writing into pangram? I have. I've gotten 100% AI with multiple samples of my own human writing. It has also given me 50% on things I know were 100% AI written (from my prompts).

Pangram and everything like it is useless. The results are random on known samples.

▲

MostlyStable 7 minutes ago | parent | next [-]

Pangram specifically (as opposed to most other detectors) publish internal audits, and seem to welcome external audits [0]. I'm not saying that you are necessarily wrong, just that in my opinion they have earned a higher bar of criticism than random one off anecdote.

[0] https://xcancel.com/JohnHolbein1/status/2059648132250570975#...

	▲	jedberg 3 minutes ago \| parent [-]
		That's a fair criticism, I certainly didn't run a full benchmark. Just a few of my own pieces of writing. I also did it a few months ago, maybe it's gotten better since.

▲

no_multitudes 11 minutes ago | parent | prev [-]

That's interesting! I have tried to get false positives from pangram and failed, so I trusted it a bit more than any of the others, although I generally just rely on my own intuition. I am curious what your false positive samples looked like, if you're willing to share.

(I'm less interested in false negatives; I have successfully produced those myself.)

	▲	jedberg 3 minutes ago \| parent [-]
		I'll try to pull them up for you, I'd have to go back and find them on my computer.