| ▲ | alightsoul 17 minutes ago | |
I have been wanting do do this. The biggest source of domains is certificate transparency logs. Also ICANN zone files. According to some scientific papers these cover 88% of all registered domains. You could crawl dns for CNAME records with all ipv4 IPs by distributing requests across dozens of DNS servers, the internet archive or the common crawl but doing it for the internet archive is a dick move without giving them money There's about 200 million active domains currently. That's about 66% of all businesses worldwide of which there are around 300 million. Around 100 to 150 million have active webpages | ||