Much of the content and links displayed on a webpage may not actually be visible to a search engines, e.g. Flash based content, content generated through javascript, content displayed as images etc. The Spider Simulator displays the contents of a webpage much like a search engine would see it. It also displays the hyperlinks that will be crawled by a search engine when it visits the particular webpage.