|
|
一种基于知识蒸馏与交叉注意力的轻量级多模态增强子识别方法
|
Abstract:
增强子是一类非编码元件,在基因转录调控中起关键作用。如果增强子出问题,会和很多疾病密切相关。传统实验方法在鉴定增强子时,成本高,时间也长。现有的计算方法在处理单细胞数据或者覆盖度较低的表观组学数据时,识别效果会明显变差。我们提出了一种用于低样本量表观基因组学增强子识别的知识蒸馏轻量化多模态交叉注意力模型(AttLight-Enhancer),这是一个多模态轻量化深度学习模型,用到了交叉注意力机制。它的目标是从低覆盖度多组学数据中识别增强子。我们还用了知识蒸馏的方法,把复杂教师模型里的知识迁移到轻量化学生网络中。我们在模拟低覆盖度数据和真实单细胞ATAC-seq数据上做了实验。模拟数据的覆盖度从10%一直降到1%。结果显示,当覆盖度只有1%时,AttLight-Enhancer的AUROC达到了0.865,比各个基线模型都好。经过知识蒸馏,学生模型的参数量只有教师模型的18.5%。通过注意力可视化分析,我们还看到了不同表观遗传特征的重要程度。其中H3K27ac最重要,然后是染色质可及性,最后是序列信息。
Enhancers are a class of non-coding elements that play a critical role in the transcriptional regulation of genes. Enhancer dysfunction is closely associated with numerous diseases. Traditional experimental methods for identifying enhancers are costly and time-consuming. Existing computational approaches exhibit significantly degraded performance when processing single-cell data or epigenomic data with low coverage. In this study, we propose AttLight-Enhancer, a lightweight multimodal deep learning model that employs a cross-attention mechanism. It is designed to identify enhancers from low-coverage multi-omics data. We also utilize knowledge distillation to transfer knowledge from a complex teacher model to a lightweight student network. We conduct experiments on both simulated low-coverage data and real single-cell ATAC-seq data, with the coverage of simulated data reduced from 10% to 1%. The results demonstrate that AttLight-Enhancer achieves an AUROC of 0.865 at only 1% coverage, outperforming all baseline models. After knowledge distillation, the number of parameters of the student model is merely 18.5% of that of the teacher model. Through attention visualization analysis, we further reveal the importance of different epigenetic features: H3K27ac is the most critical, followed by chromatin accessibility, and finally sequence information.
| [1] | Yang, J.H. and Hansen, A.S. (2024) Enhancer Selectivity in Space and Time: From Enhancer-Promoter Interactions to Promoter Activation. Nature Reviews Molecular Cell Biology, 25, 574-591. https://doi.org/10.1038/s41580-024-00710-6 |
| [2] | Panigrahi, A. and O’Malley, B.W. (2021) Mechanisms of Enhancer Action: The Known and the Unknown. Genome Biology, 22, Article No. 108. https://doi.org/10.1186/s13059-021-02322-1 |
| [3] | Vasileva, A.V., Gladkova, M.G., Ashniev, G.A., Osintseva, E.D., Orlov, A.V., Kravchuk, E.V., et al. (2024) Super-enhancers and Their Parts: From Prediction Efforts to Pathognomonic Status. International Journal of Molecular Sciences, 25, Article 3103. https://doi.org/10.3390/ijms25063103 |
| [4] | Shu, M., Hong, D., Lin, H., Zhang, J., Luo, Z., Du, Y., et al. (2022) Single-Cell Chromatin Accessibility Identifies Enhancer Networks Driving Gene Expression during Spinal Cord Development in Mouse. Developmental Cell, 57, 2761-2775.e6. https://doi.org/10.1016/j.devcel.2022.11.011 |
| [5] | Green, N.F.O., Sutton, G.J., Pérez-Burillo, J., Wang, J., Bagot, S., Danon, H.G., et al. (2025) Crispri Screening in Cultured Human Astrocytes Uncovers Distal Enhancers Controlling Genes Dysregulated in Alzheimer’s Disease. Nature Neuroscience, 29, 703-716. https://doi.org/10.1038/s41593-025-02154-3 |
| [6] | Yang, S., Ahmed, I., Li, Y., Bleaney, C.W. and Sharrocks, A.D. (2024) Massively Parallel Reporter Assays Identify Enhancer Elements in Oesophageal Adenocarcinoma. NAR Cancer, 6, zcae041. https://doi.org/10.1093/narcan/zcae041 |
| [7] | Wu, H., Zhang, J., Tan, L. and Xie, X.S. (2025) Single-Cell Micro-C Profiles 3D Genome Structures at High Resolution and Characterizes Multi-Enhancer Hubs. Nature Genetics, 57, 1777-1786. https://doi.org/10.1038/s41588-025-02247-6 |
| [8] | Denaud, S., Bardou, M., Papadopoulos, G., Grob, S., Di Stefano, M., Sabarís, G., et al. (2024) A PRE Loop at the Dac Locus Acts as a Topological Chromatin Structure That Restricts and Specifies Enhancer-Promoter Communication. Nature Structural & Molecular Biology, 31, 1942-1954. https://doi.org/10.1038/s41594-024-01375-7 |
| [9] | Wu, H., Liu, M., Zhang, P. and Zhang, H. (2023) iEnhancer-SKNN: A Stacking Ensemble Learning-Based Method for Enhancer Identification and Classification Using Sequence Information. Briefings in Functional Genomics, 22, 302-311. https://doi.org/10.1093/bfgp/elac057 |
| [10] | Khanal, J., Tayara, H. and Chong, K.T. (2020) Identifying Enhancers and Their Strength by the Integration of Word Embedding and Convolution Neural Network. IEEE Access, 8, 58369-58376. https://doi.org/10.1109/access.2020.2982666 |
| [11] | Niu, K., Luo, X., Zhang, S., Teng, Z., Zhang, T. and Zhao, Y. (2021) iEnhancer-EBLSTM: Identifying Enhancers and Strengths by Ensembles of Bidirectional Long Short-Term Memory. Frontiers in Genetics, 12, Article 665498. https://doi.org/10.3389/fgene.2021.665498 |
| [12] | Wang, T. and Gao, M. (2025) Utilizing a Deep Learning Model Based on BERT for Identifying Enhancers and Their Strength. PLOS ONE, 20, e0320085. https://doi.org/10.1371/journal.pone.0320085 |
| [13] | Chen, S., Gan, M., Lv, H. and Jiang, R. (2021) DeepCAPE: A Deep Convolutional Neural Network for the Accurate Prediction of Enhancers. Genomics, Proteomics & Bioinformatics, 19, 565-577. https://doi.org/10.1016/j.gpb.2019.04.006 |
| [14] | Toneyan, S. and Koo, P.K. (2024) Interpreting Cis-Regulatory Interactions from Large-Scale Deep Neural Networks. Nature Genetics, 56, 2517-2527. https://doi.org/10.1038/s41588-024-01923-3 |
| [15] | Xiao, Z., Wang, L., Ding, Y. and Yu, L. (2022) iEnhancer-MRBF: Identifying Enhancers and Their Strength with a Multiple Laplacian-Regularized Radial Basis Function Network. Methods, 208, 1-8. https://doi.org/10.1016/j.ymeth.2022.10.001 |
| [16] | Basith, S., Hasan, M.M., Lee, G., Wei, L. and Manavalan, B. (2021) Integrative Machine Learning Framework for the Identification of Cell-Specific Enhancers from the Human Genome. Briefings in Bioinformatics, 22, bbab252. https://doi.org/10.1093/bib/bbab252 |
| [17] | Mu, X., Huang, Z., Chen, Q., Shi, B., Xu, L., Xu, Y., et al. (2024) DeepEnhancerPPO: An Interpretable Deep Learning Approach for Enhancer Classification. International Journal of Molecular Sciences, 25, Article 12942. https://doi.org/10.3390/ijms252312942 |
| [18] | Mehmood, F., Arshad, S. and Shoaib, M. (2024) ADH-Enhancer: An Attention-Based Deep Hybrid Framework for Enhancer Identification and Strength Prediction. Briefings in Bioinformatics, 25, bbae030. https://doi.org/10.1093/bib/bbae030 |
| [19] | Kosicki, M., Baltoumas, F.A., Kelman, G., Boverhof, J., Ong, Y., Cook, L.E., et al. (2024) VISTA Enhancer Browser: An Updated Database of Tissue-Specific Developmental Enhancers. Nucleic Acids Research, 53, D324-D330. https://doi.org/10.1093/nar/gkae940 |
| [20] | Yeo, I.S., Jeong, D., Lee, Y., Ahn, J., Bae, S. and Song, M. (2025) An Integrated Transcriptomic Approach for Identifying Enhancers in Limb Motor Pools. Scientific Reports, 15, Article No. 34813. https://doi.org/10.1038/s41598-025-18807-z |
| [21] | Zhang, Y., Zhang, P. and Wu, H. (2024) Enhancer-MDLF: A Novel Deep Learning Framework for Identifying Cell-Specific Enhancers. Briefings in Bioinformatics, 25, bbae083. https://doi.org/10.1093/bib/bbae083 |