|
基于残差注意力网络的医疗命名实体识别方法
|
Abstract:
针对临床医疗记录中的复杂语义实体和长短距离依赖关系识别准确率低的难题,文章提出了一种双向语义与残差注意力网络的医疗文本命名实体识别方法。利用BERT-wwm预训练模型捕捉语义特征,结合双向门控循环单元BiGRU用于处理复杂长程语义关联;增加残差连接的注意力Attention结构,保障专注于关键信息的同时,不会丢失捕捉到的整体序列特征;条件随机场CRF负责最后的序列标注预测,对前序多层神经网络抽取的特征序列进行最优路径解码。实验结果表明,通过本方法能够有效提升命名实体识别的准确率。
Aiming at the challenge of low recognition accuracy for complex semantic entities and long- and short-range dependencies in clinical medical records, this paper proposes a medical text named entity recognition method that integrates bidirectional semantics with a residual attention network. The method leverages the BERT-wwm pre-trained model to capture semantic features and combines it with a Bidirectional Gated Recurrent Unit (BiGRU) to handle complex long-range semantic associations. An Attention mechanism with residual connections is added to ensure focus on key information while preserving the overall sequence characteristics captured. A Conditional Random Field (CRF) is responsible for the final sequence labeling prediction, performing optimal path decoding on the feature sequences extracted by the preceding multi-layer neural networks. Experimental results demonstrate that this approach can effectively improve the accuracy of named entity recognition.
[1] | 杜晋华, 尹浩, 冯嵩. 中文电子病历命名实体识别的研究与进展[J]. 电子学报, 2022, 50(12): 3030-3053. |
[2] | 赵铁军, 许木璠, 陈安东. 自然语言处理研究综述[J/OL]. 新疆师范大学学报(哲学社会科学版), 1-23. https://doi.org/10.14100/j.cnki.65-1039/g4.20230804.001, 2024-10-15. |
[3] | Radford, A., Narasimhan, K., Salimans, T., et al. (2018) Improving Language Understanding by Generative Pre-Training. Preprint, OpenAI. |
[4] | 杨飞洪, 王序文, 李姣. 基于BERT-TextCNN模型的临床试验筛选短文本分类方法[J]. 中华医学图书情报杂志, 2021, 30(1): 54-59. |
[5] | Li, Y., Ding, Z., Ma, Z., Wu, Y., Wang, Y., Zhang, R., et al. (2023) Few-Shot Relation Classification Based on the BERT Model, Hybrid Attention and Fusion Networks. Applied Intelligence, 53, 21448-21464. https://doi.org/10.1007/s10489-023-04634-0 |
[6] | Radford, A., Wu, J., Child, R., et al. (2019) Language Models Are Unsupervised Multitask Learners. Preprint, OpenAI. |
[7] | Zhu, E., Sheng, Q., Yang, H., Liu, Y., Cai, T. and Li, J. (2023) A Unified Framework of Medical Information Annotation and Extraction for Chinese Clinical Text. Artificial Intelligence in Medicine, 142, Article 102573. https://doi.org/10.1016/j.artmed.2023.102573 |
[8] | Liu, C. and Yang, S. (2022) Using Text Mining to Establish Knowledge Graph from Accident/incident Reports in Risk Assessment. Expert Systems with Applications, 207, 117991. https://doi.org/10.1016/j.eswa.2022.117991 |
[9] | 郁圣卫, 卢奇, 陈文亮. 基于领域情感词典特征表示的细粒度意见挖掘[J]. 中文信息学报, 2019, 33(2): 112-121. |