|
基于Bert的层次多标签文本分类
|
Abstract:
层次多标签文本分类(Hierarchical Multi-label Text Classification, HMTC)是自然语言处理领域(Natural Language Processing, NLP)一项重要的任务。在其由浅至深的标签层次结构中,深层标签更能精确地代表文本所属的标签类别。然而,深层标签的样本实例较少且彼此之间语义接近,导致其难以被正确分类。针对上述的问题,文章提出了基于Bert的层次多标签文本分类方法,先利用Bert构建优越的文本表示,再以自上而下逐层的方式利用浅层级的标签信息引导深层级标签的分类,有效地提升了分类精度。实验结果表明所提模型与其它基线模型相比具有更好的分类性能。
Hierarchical Multi-label Text Classification (HMTC) is an important task in the field of natural language processing (NLP). In its shallow-to-deep label hierarchy, deep labels can more accurately represent the label categories to which the text belongs. However, there are fewer sample instances of deep labels and they are semantically close to each other, making it difficult to classify them correctly. To address the above problem, this article proposes a hierarchical multi-label text classification method based on Bert. First, it uses Bert to construct a superior text representation, and then uses the shallow-level label information to guide the classification of deep-level labels in a top-down layer-by-layer manner, effectively improving the classification accuracy. The experimental results show that the proposed model has better classification performance compared to other baseline models.
[1] | Aly, R., Remus, S. and Biemann, C. (2019) Hierarchical Multi-Label Classification of Text with Capsule Networks. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, Florence, July 2019, 323-330. https://doi.org/10.18653/v1/P19-2045 |
[2] | Wehrmann, J., Cerri, R. and Barros, R.C. (2018) Hierarchical Multi-Label Classification Networks. Proceedings of the 35th International Conference on Machine Learning, Stock Holmsm?ssan, July 2018, 5225-5234. https://doi.org/10.1145/3019612.3019664 |
[3] | Huang, W., Chen, E., Liu, Q., et al. (2019) Hierarchical Multi-Label Text Classification: An Attention-Based Recurrent Network Approach. Proceedings of the 28th ACM International Conference on Information and Knowledge Management, New York, November 2019, 1051-1060. https://doi.org/10.1145/3357384.3357885 |
[4] | Zhang, X., Xu, J., Soh, C., et al. (2021) LA-HCN: Label-Based Attention for Hierarchical Multi-Label Text Classification Neural Network. Expert Systems with Applications, 187, Article ID: 115922. https://doi.org/10.1016/j.eswa.2021.115922 |
[5] | Devlin, J., Chang, M.W., Lee, K., et al. (2019) Bert: Pre-Training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, June 2019, 4171-4186. |