Most sites will be crawled during our regular archiving efforts as long as they are fairly well-linked from other sources. Please check the Wayback Machine for your site to make sure we don’t already have it archived.
Sites in the Wayback Machine may have been crawled by the Internet Archive, Alexa Internet, or another partner who donates content for preservation. There are two ways to make sure crawlers can find your site.
- Get your site listed in major directories, like dmoz.org, or encourage other, popular sites to link to you. To add your site to the Open Directory, go to http://www.dmoz.org/, find the category your site should be listed in, and click the “Add URL” link at the top of the page. We highly recommend reading their submission guidelines first at http://dmoz.org/add.html. This method will help your site be discovered by everyone, not just us!
- Submit the URL of your site to Alexa Internet to be crawled. Visit this page to tell them to crawl you. Sites crawled via this method will not appear in the Wayback Machine for at least 6 months, but this is an easy way to make sure your site gets noticed.
In all cases, ensure that your site’s robots.txt rules and in-page META robots directives allow crawlers to capture your site.
UPDATE 2/7/2011: At some point in the past couple of months, Alexa removed the ability to submit your site for crawling (option 2 above). At the moment, we do not have an alternative method of submission. As always, the best way to make sure we find your site is to make sure lots of people link to it. We’ll notify users on this page if another solution becomes available. Thanks!
- How can I find all the pages from one site?
- How can I view a page without the Wayback code in it?
- Can I get a copy of this web page?
- The page I want redirects now – how can I see the old versions?
- How should I report issues?
- Can I get just one page archived?
- What’s the difference between the classic Wayback Machine and the new version?
- What is the Wayback Machine?
- What are the known issues with the new version?
- How can I have my site removed from the Wayback Machine?
- My site’s not archived! How can I add it?