How the Internet Happened: From Netscape to the iPhone

by Brian McCullough

To this day, a “search engine” is actually a database of website copies. The search engine sends out “spiders,” which are computer programs that go out onto the web and find new web pages. The spiders locate the pages and then copy some or all of the code into the search engine’s own database. When a user searches a search engine, they’re not actually searching the web itself, but are instead querying the database of copied webpages the search engine has compiled. The accuracy and comprehensiveness of this database varied from search engine to search engine and the results therefore varied depending on the weight each search engine gave to various factors in the database. Search engine A might list a certain webpage as the number-one result for windsurfing because the word “windsurfing” was prominent in the title of the webpage. But search engine B might list a completely different page as the first result because the word “windsurfing” showed up the greatest number of times in the body of the page.