A noun and a verb, Search Engines have spiders crawl through all the linked pages of a website to gather information to include the site in their Natural Listings and also use to determine their ranking on various Search Terms. Understanding how these spiders work is crucial to SEO, as both keyword usage and technical factors can play a part in how a spider sees your site.