Remix.run Logo
aflukasz 6 hours ago

AI bots (or clients claiming to be one) appear quite fast on new sites, at least that's what I saw recently in few places. They probably monitor Certificate Transparency logs - you won't hide by avoiding linking. Unless you are ok with staying in the shadow of naked http.

KetoManx64 4 hours ago | parent [-]

Get a wildcard cert and use it behind a reverse proxy.

RIMR 4 hours ago | parent [-]

Okay, but then what? Host your sites on something other than 'www' or '*', exclude them from search engines, and never link to them? Then, the few people who do resolve these subdomains, you just gotta hope they don't do it using a DNS server owned by a company with an AI product (like Google, Microsoft, or Amazon)?

I really don't know how you're supposed to shield your content from AI without also shielding it from humanity.

throwaway81523 2 hours ago | parent [-]

Don't have any index pages or heavy cross-linking between pages.

petcat 35 minutes ago | parent [-]

None of that matters. AI bots can still figure out how to navigate the website.