My PhD is on auditability of AI systems, and we are exactly working on the engineering of ZKP in this context. I can confirm through our industry partner (AI Assurance and auditing), that black box auditing is indeed a huge problem for them, and snarks have shown their potential in this context. It's great to see Anthropic confirming our early results.

▲

ngruhn 9 hours ago | parent | next [-]

I'm blind to the problem being solved here. Can you motivate it? Why do I need more than normal HTTPS on the token API?

▲

fscaramuzza 8 hours ago | parent [-]

HTTPS makes sure you’re communicating on a secure channel with the API endpoint. ZKPs guarantees that the computation happened correctly behind the endpoint.

The work from Anthropic isn't a new idea, but they made the process more efficient, as inference verification is one of the biggest bottlenecks in the AI lifecycle. Beyond that, there's a lot more that can be proven, like showing a model was trained on a specific dataset with certain properties.

From an auditing perspective, this used to be handled either by showing the data (which is pretty rare), checking samples (which has been shown to be easily hackable), or similar workarounds. For inference, an auditor might want to interact with a model on demand and have absolute certainty that the model they're talking to is actually the one declared, and not a substitute.

	▲	grey-area 5 hours ago \| parent \| next [-]
		I’m not sure what you mean by correctly here - do you mean with the model expected? That’s not related to correctness. Pleased to hear somebody cares about this. What is it a bottleneck for though? Are people actually doing this right now in non-academic settings? I had the impression this is not in fact usable with larger models.
	▲	ngruhn 6 hours ago \| parent \| prev [-]
		> guarantees that the computation happened correctly behind the endpoint but why do I care about that? can you some example "user stories"?

▲

brcmthrowaway 8 hours ago | parent | prev [-]

So, you got beaten to the punch?

	▲	fscaramuzza 8 hours ago \| parent [-]
		Well luckily not really, their work is more foundational-y, the idea is not new, but they made it more efficient and practical. On our side, we work more on the integration of these protocol in production systems, but we are pretty much on the frontier and is not very common to see such big companies experimenting with this technology. Microsoft already tried something [1] but I'm not sure if it ever went further than a research prototype. [1] https://arxiv.org/pdf/2402.02675