|
|
- 2015
基于不一致的汉语句法树库潜在错误查找
|
Abstract:
摘要： 语料库是自然语言处理NLP(natural language processing)的基础,其标注质量影响着基于有指导机器学习方法的NLP系统的性能。针对汉语句法树库,提出了一种基于不一致查找树库潜在标注错误的方法,该方法主要从两方面进行不一致检测:一是从类似短语内部构成并结合可疑度来检测不一致;二是从标注大纲入手,检测词性、短语等各类标记符号与大纲定义不符合的情况。实验结果表明,在查找到的不一致现象中,存在一定数量的语料库标注错误。
Abstract: Corpora are fundamental to natural language processing(NLP) and corpus annotation quality influences the performance of the systems based on supervised machine learning approaches. Aiming at Chinese treebank, an approach was proposed to find potential errors based on inconsistencies. Inconsistencies were detected with two strategies: one uses similar internal structure and suspicious degree, the other uses the annotation guideline to check those annotations, which don't meet the definitions of the guideline. Experimental results show that there are some annotation errors in the inconsistencies
| [1] | 刘江, 郑家恒, 张虎.中文文本语料库分词一致性检验技术的初探[J]. 计算机应用研究, 2005, 22(8):52-54. LIU Jiang, ZHENG Jiaheng, ZHANG Hu. Studies on the consistency o f word-segmented Chinese corpus[J]. Application Research of Computers, 2005, 22(8):52-54. |
| [2] | KULICK S, BIES A. Treebank analysis and search using an extracted tree grammar[C]// Proceedings of the 8th International Workshop on Treebanks and Linguistic Theories, Milan: [s.n.], 2009:1-12. |
| [3] | DICKINSON M. Similarity and dissimilarity in treebank grammars[C]// Proceedings of the 18th International Congress of Linguists. Seoul:[s.n.], 2008: 1597-1611. |
| [4] | KULICK S, BIES A, MOTT J. Using derivation trees for treebank error detection[C]// Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies.Stroudsburg, PA, USA: Association for Computational Linguistics, 2011: 693-698. |
| [5] | 苗玺, 郑家恒. 中文语料库分词不一致的分类处理研究[J]. 山西大学学报:自然科学版, 2006,29(1):23-25. MIAO Xi, ZHENG Jiaheng. Classified study on inconsistency of segment for Chinese corpus[J]. Journal of Shanxi University: Natural Science Edition, 2006, 29(1):23-25. |
| [6] | 张虎, 郑家恒, 刘江. 语料库词性标注一致性检查方法研究[J]. 中文信息学报, 2004, 18(5):11-16. ZHANG Hu, ZHENG Jiaheng, LIU Jiang. The inspecting method study on consistence of part of speech tagging of corpus[J]. Journal of Chinese Information Processing, 2004, 18(5):11-16. |
| [7] | VOLOKH A, NEUMANN G. Automatic detection and correction of errors in dependency tree-banks[C]// Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA, USA: Association for Computational Linguistics, 2011: 346-350. |
| [8] | GARDENT C, NARAYAN S. Error mining on dependency trees[C]// Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA, USA: Association for Computational Linguistics, 2012: 592-600. |
| [9] | 魏莉, 谭红叶, 郑家恒, 等. 汉语句法树库一致性检验方法研究[J]. 广西师范大学学报: 自然科学版, 2010(001):139-142. WEI Li, TAN Hongye, ZHENG Jiaheng,et al. Study of keeping consistency of Chinese corpus of complete parsing[J]. Journal of Guangxi Normal University: Natural Science Edition, 2010(001):139-142. |
