AUTOMATIC SEGMENTATION OF CHINESE USING OVERLAYING AMBIGUITY EXAMINING METHOD AND STATISTICS LANGUAGE MODEL
利用覆盖歧义检测法和统计语言模型进行汉语自动分词

In this paper, the question of Chinese automatic segmentation is discussed using overlaying ambiguity examining method and statistics language model. The multi-time iterative method is applied to train language model, which can produce a better model. The process of training language model is described in detail. The result shows that the perplexity of language model is reduced. The accuracy of segmentation changes with different language model and the reason is analyzed.