Interestingly, there is llms.txt (like robots.txt) which is a new emerging standard to aid AI in crawling and gathering data from your website.