Dan Scott

Dan Scott at

One other problem... A robots.txt "Disallow:" line will prevent archive.org from saving a given URL, per https://archive.org/about/faqs.php#14, which says the Internet Archive "does respect robots.txt instructions, and even does so retroactively".


Claes Wallin (韋嘉誠) likes this.

retroactively is really disturbing.

Bradley M. Kuhn at 2015-07-06T22:01:35Z

Richard Fontana, Dan Scott likes this.