A master repository service maintains a directory of web servers and the most recent times that their web contents were modified, and provides this information to web crawlers to increase their efficiency. The master repository service receives web content update reports from a plurality of web servers,...http://www.google.de/patents/US6418452?utm_source=gb-gplus-sharePatent US6418452 - Network repository service directory for efficient web crawling
