A Web crawler system and method for quickly fetching and analyzing Web pages on the World Wide Web includes a hash table stored in random access memory (RAM) and a sequential Web information disk file. For every Web page known to the system, the Web crawler system stores an entry in the sequential disk...http://www.google.de/patents/US5974455?utm_source=gb-gplus-sharePatent US5974455 - System for adding new entry to web page table upon receiving web page including link to another web page not having corresponding entry in web page table
