Survey of High-performance Web Crawler
高性能网络爬虫:研究综述

Web Crawlers,one of basic components of Search Engine,are programs to download resources from Internet.We illuminated the work theory of the Web Crawlers,and its development,and how to design a high-performance,scalable,distributed Web crawler,including the faced key problem.