What DNS service did the agent discover that allowed it to execute arbitrary llm queries? And how?

brian_cunnie an hour ago | parent | next [-]

It was my service that the agent used: nip.io / sslip.io

OpenAI emailed me to let me know that their agent used my service to break out of test containment. OpenAI was quite polite (shout-out to Steven), and let me know they'd keep the name of my service out of the blog post.

nip.io is a simple DNS service: when queried with a hostname with an embedded IP address, returns that IP address, e.g. 127-0-0-1.nip.io → 127.0.0.1

The nip.io service is airtight with one exception: the "_acme-challenge" subdomain. In that case, rather than returning the IP address (or whatever record was queried), nip.io would return an empty "answer" section but an NS authority & not authoritative. In other words, nip.io would delegate DNS queries to that IP address.

That was the escape hatch.

In a couple of weeks I'll close that escape hatch. It was originally intended as a mechanism to allow savvy uses to procure wildcard certs (e.g. "*.64-176-22-9.nip.io") from certificate authorities such as Let's Encrypt. But experience proved that the it was an undue burden trying to support unsophisticated users attempting to procure a wildcard cert. "Wildcard certs are not supported" became my new mantra.

But I had neglected to remove the old code.

(the late Roopinder Singh created nip.io, and he was a good guy. I miss him)

▲

itintheory an hour ago | parent | next [-]

But that's still DNS, right? Where does it bleed over into an LLM API? I understand there are DNS to LLM server projects, but how would the agent discover one? And I'm guessing most people who run something like that don't expose it publicly...

	▲	oefrha 22 minutes ago \| parent [-]
		Yes it's still DNS. _acme-challenge.<arbitrary-ip>.nip.io's role here is to allow tunneling to <arbitrary-ip>:53 through the approved local resolver at 10.214.0.2; without it the direct request to <arbitrary-ip>:53 is dropped. Someone still has to run that LLM over DNS on an <arbitrary-ip> serving public requests. > And I'm guessing most people who run something like that don't expose it publicly... There was a post last week https://news.ycombinator.com/item?id=49771110 that stayed at #1 on front page for hours. If you ignore the LLM framing it's literally an anonymous file host where anyone can upload or download anything, with no or absurdly high file size limits. That should be enough to give any reasonable server admin a heart attack... It's trivial to vibe code shit and throw it on the Internet these days, people who don't understand or care about consequences are doing it by the droves. Go figure.

▲

jacquesm an hour ago | parent | prev [-]

Wow, such a tiny hole. Thank you for keeping it alive.

▲

oefrha an hour ago | parent | prev | next [-]

Found one https://duyet.github.io/llm-over-dns/ and far from the only one since “X over DNS” is a deeply unoriginal idea https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que... and trivial to code up.

▲

itintheory an hour ago | parent | next [-]

Right, but your have to run this server somewhere, which the agent couldn't do.

	▲	oefrha 42 minutes ago \| parent [-]
		Yes, the linked project does say they have a demo server at llm-over-dns.duyet.net. (I didn't bother to check whether it's still working.) Edit: This particular demo server doesn't work. There's another LLM over DNS post from a year ago https://news.ycombinator.com/item?id=44813298 where the server seems to answer some queries but not others. Edit 2: Actually the server in https://news.ycombinator.com/item?id=44813298 does work with queries like "what is the capital of france", I was querying with special characters like "what's" or "1+1". So yes there are people opening hosting these on the Internet.

▲

jerjerjer an hour ago | parent | prev [-]

Thank you!

▲

an hour ago | parent | prev [-]

[deleted]