> they would likely obey robots.txt

If only... Despite providing a useful service, they are not as nice towards site owners as one would hope.

Internet Archive says:

> We see the future of web archiving relying less on robots.txt file declarations geared toward search engines

https://blog.archive.org/2017/04/17/robots-txt-meant-for-sea...

They are not alone in that. The "Archiveteam", a different organization, not to be confused with archive.org, also doesn't respect robots.txt according to their wiki: https://wiki.archiveteam.org/index.php?title=Robots.txt

I think it is safe to say that there is little consideration for site owners from the largest archiving organizations today. Whether there should be is a different debate.