![Dan Scott](https://identi.ca/uploads/dbs/2020/11/28/qPXk3A_thumb.jpg)
Dan Scott at
One other problem... A robots.txt "Disallow:" line will prevent archive.org from saving a given URL, per https://archive.org/about/faqs.php#14, which says the Internet Archive "does respect robots.txt instructions, and even does so retroactively".
Claes Wallin (韋嘉誠) likes this.
![](https://identi.ca/uploads/bkuhn/2019/10/14/fqc_Cg.png)
retroactively is really disturbing.
Bradley M. Kuhn at 2015-07-06T22:01:35Z
Richard Fontana, Dan Scott likes this.