%0 Journal Article
%T Efficiency bottlenecks analysis and solution of Web crawler<br>网络爬虫效率瓶颈的分析与解决方案
%A YIN Jiang
%A YIN Zhi-ben
%A HUANG Hong
%A <br>尹江
%A 尹治本
%A 黄洪
%J 计算机应用
%D 2008
%I 
%X The efficiency of a web crawler determines the quality of services a web searching system offers to its users. How to design a more efficient and faster web crawler is becoming a hot issue in the research of web crawler. In order to raise the crawling efficiency of a web crawler, the crawling strategy needs to be reformed. Besides, the design of the web crawler system has to be optimized and its structure also needs to be improved to eliminate bottlenecks. In this paper, an improved scheme of designing a general web crawler was presented through analyzing crawler's structure, application environment and user requirement, and the preferable testing result has proven better efficiency it has.
%K crawl strategy
%K socket
%K multi-threaded
%K web crawler<br>爬行策略
%K 套接字
%K 多线程
%K 网络爬虫
%U http://www.alljournals.cn/get_abstract_url.aspx?pcid=5B3AB970F71A803DEACDC0559115BFCF0A068CD97DD29835&cid=8240383F08CE46C8B05036380D75B607&jid=831E194C147C78FAAFCC50BC7ADD1732&aid=F56B677BE1958C05A561DEAD4DB7FE6E&yid=67289AFF6305E306&vid=D3E34374A0D77D7F&iid=94C357A881DFC066&sid=5348E91DA94080DE&eid=76C32027E03E49D7&journal_id=1001-9081&journal_name=计算机应用&referenced_num=0&reference_num=8