An Algorithm for Detecting Duplicated Chinese Web News Based on Suffix Tree
基于后缀树的中文新闻重复网页识别算法

In view of the shortcomings of traditional methods detecting duplicated Chinese Web news,this paper proposes the suffix tree as the basic data structure,and then designs an algorithm for detecting duplicated Chinese Web news according to the title and published date of news.This algorithm is based on Ukkonen algorithm and Matching Statistics algorithm and optimizes the realization.Experiments show that the proposed algorithm is an effective solution to detect the duplicated Chinese Web news,and it can be generalized to measure the similarity of strings.