Treatment-Resistant Depression (TRD) remains one of the most challenging subtypes of major depressive disorder, affecting approximately one-third of patients and leading to significant morbidity, healthcare costs, and reduced quality of life. Predicting TRD onset and progression is complex, as it requires integrating heterogeneous biomarkers spanning neuroimaging, genomics, and clinical history. This study presents an Enhanced Multimodal Transformer (EMT) designed to fuse functional magnetic resonance imaging (fMRI), single-nucleotide polymorphism (SNP) profiles, and structured clinical variables into a unified predictive framework. The architecture employs modality-specific encoders patch-based embeddings with positional encodings for fMRI, attention-weighted embeddings for SNP data, and normalized dense projections for clinical features-followed by the introduction of modality tokens to enable cross-modal information exchange within a Transformer encoder. To validate the architecture in a controlled setting, we generated a balanced, clinically inspired synthetic dataset with distinct activation patterns in brain regions (prefrontal cortex, amygdala, anterior cingulate), SNP distributions with predictive loci, and clinically relevant severity profiles. Model training achieved rapid convergence with early stopping at eight epochs. Evaluation demonstrated a perfect Area Under the ROC Curve (AUC = 1.00) and average precision of 1.00, indicating complete separation in probability space. However, accuracy at a fixed 0.5 decision threshold was limited (50%), reflecting probability compression into distinct but narrow ranges (≈0.32 for non-TRD, ≈0.41 for TRD). Feature analysis revealed that clinical severity, specific SNP clusters, and region-specific fMRI activations dominated predictive importance. These results provide a proof-of-concept that Transformer-based multimodal fusion can capture complex, cross-domain patterns in TRD, supporting its potential for precision psychiatry. Future work will extend to real-world datasets and incorporate probability calibration to improve threshold-based classification performance in clinical settings.
Cite this paper
Filippis, R. D. and Foysal, A. A. (2026). Enhanced Multimodal Transformer for Treatment-Resistant Depression Prediction Using Synthetic fMRI, Genomic, and Clinical Data. Open Access Library Journal, 13, e14447. doi: http://dx.doi.org/10.4236/oalib.1114447.
Brown, S., Rittenbach, K., Cheung, S., McKean, G., MacMaster, F.P. and Clement, F. (2019) Current and Common Definitions of Treatment-Resistant Depression: Findings from a Systematic Review and Qualitative Interviews. The Canadian Journal of Psychiatry, 64, 380-387. https://doi.org/10.1177/0706743719828965
Zhdanava, M., Pilon, D., Ghelerter, I., Chow, W., Joshi, K., Lefebvre, P., et al. (2021) The Prevalence and National Burden of Treatment-Resistant Depression and Major Depressive Disorder in the United States. The Journal of Clinical Psychiatry, 82, 20m13699. https://doi.org/10.4088/jcp.20m13699
Voineskos, D., Daskalakis, Z.J. and Blumberger, D.M. (2020) Management of Treatment-Resistant Depression: Challenges and Strategies. Neuropsychiatric Disease and Treatment, 16, 221-234. https://doi.org/10.2147/ndt.s198774
Berlim, M.T. and Turecki, G. (2007) What Is the Meaning of Treatment Resistant/refractory Major Depression (TRD)? A Systematic Review of Current Randomized Trials. European Neuropsychopharmacology, 17, 696-707. https://doi.org/10.1016/j.euroneuro.2007.03.009
de Sousa, R.T., Zanetti, M.V., Brunoni, A.R. and Machado-Vieira, R. (2015) Challenging Treatment-Resistant Major Depressive Disorder: A Roadmap for Improved Therapeutics. Current Neuropharmacology, 13, 616-635. https://doi.org/10.2174/1570159x13666150630173522
Denee, T., Kerr, C., Ming, T., Wood, R., Tritton, T., Middleton-Dalby, C., et al. (2021) Current Treatments Used in Clinical Practice for Major Depressive Disorder and Treatment Resistant Depression in England: A Retrospective Database Study. Journal of Psychiatric Research, 139, 172-178. https://doi.org/10.1016/j.jpsychires.2021.05.026
Sousa, R.D., Gouveia, M., Nunes da Silva, C., Rodrigues, A.M., Cardoso, G., Antunes, A.F., et al. (2022) Treatment-Resistant Depression and Major Depression with Suicide Risk—The Cost of Illness and Burden of Disease. Frontiers in Public Health, 10, Article 898491. https://doi.org/10.3389/fpubh.2022.898491
Johnston, K.M., Powell, L.C., Anderson, I.M., Szabo, S. and Cline, S. (2019) The Burden of Treatment-Resistant Depression: A Systematic Review of the Economic and Quality of Life Literature. Journal of Affective Disorders, 242, 195-210. https://doi.org/10.1016/j.jad.2018.06.045
Kubitz, N., Vossen, C., Papadimitropoulou, K. and Karabis, A. (2014) The Prevalence and Disease Burden of Treatment-Resistant Depression—A Systematic Review of the Literature. Value in Health, 17, A455-A456. https://doi.org/10.1016/j.jval.2014.08.1247
Trevino, K., McClintock, S.M., Fischer, N.M., Vora, A. and Husain, M.M. (2014) Defining Treatment-Resistant Depression: A Comprehensive Review of the Literature. Annals of Clinical Psychiatry, 26, 222-232. https://doi.org/10.1177/104012371402600310
Rymaszewska, J., Fila-Pawłowska, K. and Szcześniak, D. (2023) Chronic Mental Disorders: Limitations and Perspectives of Prediction, Prevention, Diagnosis, and Personalized Treatment in Psychiatry. In: Podbielska, H. and Kapalla, M., Eds., Predictive, Preventive, and Personalised Medicine: From Bench to Bedside, Springer, 261-282. https://doi.org/10.1007/978-3-031-34884-6_15
Omiyefa, S. (2025) Artificial Intelligence and Machine Learning in Precision Mental Health Diagnostics and Predictive Treatment Models. International Journal of Research Publication and Reviews, 6, 85-99. https://doi.org/10.55248/gengpi.6.0325.1107
Conway, C.R., George, M.S. and Sackeim, H.A. (2017) Toward an Evidence-Based, Operational Definition of Treatment-Resistant Depression: When Enough Is Enough. JAMA Psychiatry, 74, 9-10. https://doi.org/10.1001/jamapsychiatry.2016.2586
Perna, G., Spiti, A., Torti, T., Daccò, S. and Caldirola, D. (2024) Biomarker-Guided Tailored Therapy in Major Depression. In: Kim, Y.K., Ed., Recent Advances and Challenges in the Treatment of Major Depressive Disorder, Springer, 379-400. https://doi.org/10.1007/978-981-97-4402-2_19
de Sousa, R.D., Zagalo, D.M., Costa, T., de Almeida, J.M.C., Canhão, H. and Rodrigues, A. (2025) Exploring Depression in Adults over a Decade: A Review of Longitudinal Studies. BMC Psychiatry, 25, Article No. 378. https://doi.org/10.1186/s12888-025-06828-x
Buoli, M., Capuzzi, E., Caldiroli, A., Ceresa, A., Esposito, C.M., Posio, C., et al. (2022) Clinical and Biological Factors Are Associated with Treatment-Resistant Depression. Behavioral Sciences, 12, Article 34. https://doi.org/10.3390/bs12020034
Murphy, J.A., Sarris, J. and Byrne, G.J. (2017) A Review of the Conceptualisation and Risk Factors Associated with Treatment-Resistant Depression. Depression Research and Treatment, 2017, Article ID: 4176825. https://doi.org/10.1155/2017/4176825
Halaris, A., Sohl, E. and Whitham, E.A. (2021) Treatment-Resistant Depression Revisited: A Glimmer of Hope. Journal of Personalized Medicine, 11, Article 155. https://doi.org/10.3390/jpm11020155
Koski, L. and Paus, T. (2000) Functional Connectivity of the Anterior Cingulate Cortex within the Human Frontal Lobe: A Brain-Mapping Meta-Analysis. Experimental Brain Research, 133, 55-65. https://doi.org/10.1007/s002210000400
Marusak, H.A., Thomason, M.E., Peters, C., Zundel, C., Elrahal, F. and Rabinak, C.A. (2016) You Say ‘Prefrontal Cortex’ and I Say ‘Anterior Cingulate’: Meta-Analysis of Spatial Overlap in Amygdala-To-Prefrontal Connectivity and Internalizing Symptomology. Translational Psychiatry, 6, e944-e944. https://doi.org/10.1038/tp.2016.218
Rigucci, S., Serafini, G., Pompili, M., Kotzalidis, G.D. and Tatarelli, R. (2010) Anatomical and Functional Correlates in Major Depressive Disorder: The Contribution of Neuroimaging Studies. The World Journal of Biological Psychiatry, 11, 165-180. https://doi.org/10.3109/15622970903131571
Yücel, M., Wood, S.J., Fornito, A., Riffkin, J., Velakoulis, D. and Pantelis, C. (2003) Anterior Cingulate Dysfunction: Implications for Psychiatric Disorders? Journal of Psychiatry and Neuroscience, 28, 350-354. https://doi.org/10.1139/jpn.0342
Rodríguez-Cano, E., Sarró, S., Monté, G.C., Maristany, T., Salvador, R., McKenna, P.J., et al. (2014) Evidence for Structural and Functional Abnormality in the Subgenual Anterior Cingulate Cortex in Major Depressive Disorder. Psychological Medicine, 44, 3263-3273. https://doi.org/10.1017/s0033291714000841
Fountoulakis, K.N., Giannakopoulos, P., Kövari, E. and Bouras, C. (2008) Assessing the Role of Cingulate Cortex in Bipolar Disorder: Neuropathological, Structural and Functional Imaging Data. Brain Research Reviews, 59, 9-21. https://doi.org/10.1016/j.brainresrev.2008.04.005
Finlayson, T.L., Moyer, C.A. and Sonnad, S.S. (2004) Assessing Symptoms, Disease Severity, and Quality of Life in the Clinical Context: A Theoretical Framework. American Journal of Managed Care, 10, 336-344.
Crabtree, H.L., Gray, C.S., Hildreth, A.J., O’Connell, J.E. and Brown, J. (2000) The Comorbidity Symptom Scale: A Combined Disease Inventory and Assessment of Symptom Severity. Journal of the American Geriatrics Society, 48, 1674-1678. https://doi.org/10.1111/j.1532-5415.2000.tb03882.x
Sheshanarayana, R. and You, F. (2025) Molecular Representation Learning: Cross-Domain Foundations and Future Frontiers. Digital Discovery, 4, 2298-2335. https://doi.org/10.1039/d5dd00170f
Zheng, Y. (2015) Methodologies for Cross-Domain Data Fusion: An Overview. IEEE Transactions on Big Data, 1, 16-34. https://doi.org/10.1109/tbdata.2015.2465959
Chen, E. (2025) Unified AI Framework for Scientific Simulation: Multimodal Modeling and Cross-Domain Transfer. Journal of Computer Science and Software Applications, 5, No. 7.
Petch, J., Di, S. and Nelson, W. (2022) Opening the Black Box: The Promise and Limitations of Explainable Machine Learning in Cardiology. Canadian Journal of Cardiology, 38, 204-213. https://doi.org/10.1016/j.cjca.2021.09.004
Şahin, E., Arslan, N.N. and Özdemir, D. (2024) Unlocking the Black Box: An In-Depth Review on Interpretability, Explainability, and Reliability in Deep Learning. Neural Computing and Applications, 37, 859-965. https://doi.org/10.1007/s00521-024-10437-2
Azodi, C.B., Tang, J. and Shiu, S. (2020) Opening the Black Box: Interpretable Machine Learning for Geneticists. Trends in Genetics, 36, 442-455. https://doi.org/10.1016/j.tig.2020.03.005
Hassija, V., Chamola, V., Mahapatra, A., Singal, A., Goel, D., Huang, K., et al. (2023) Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence. Cognitive Computation, 16, 45-74. https://doi.org/10.1007/s12559-023-10179-8
AlSaad, R., Abd-alrazaq, A., Boughorbel, S., Ahmed, A., Renault, M., Damseh, R., et al. (2024) Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook. Journal of Medical Internet Research, 26, e59505. https://doi.org/10.2196/59505
Chen, Z., Xu, L., Zheng, H., Chen, L., Tolba, A., Zhao, L., et al. (2024) Evolution and Prospects of Foundation Models: From Large Language Models to Large Multimodal Models. Computers, Materials & Continua, 80, 1753-1808. https://doi.org/10.32604/cmc.2024.052618
Sagheer, S.V.M., K H, M., Ameer, P.M., Parayangat, M. and Abbas, M. (2025) Transformers for Multi-Modal Image Analysis in Healthcare. Computers, Materials & Continua, 84, 4259-4297. https://doi.org/10.32604/cmc.2025.063726
Xu, P., Zhu, X. and Clifton, D.A. (2023) Multimodal Learning with Transformers: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45, 12113-12132. https://doi.org/10.1109/tpami.2023.3275156
Ji, L., Hou, W., Zhou, H., Xiong, L., Liu, C., Yuan, Z., et al. (2025) EBMGP: A Deep Learning Model for Genomic Prediction Based on Elastic Net Feature Selection and Bidirectional Encoder Representations from Transformer’s Embedding and Multi-Head Attention Pooling. Theoretical and Applied Genetics, 138, Article No. 103. https://doi.org/10.1007/s00122-025-04894-z
Mahmud, S.M.H., Goh, K.O.M., Hosen, M.F., Nandi, D. and Shoombuatong, W. (2024) Deep-Wet: A Deep Learning-Based Approach for Predicting DNA-Binding Proteins Using Word Embedding Techniques with Weighted Features. Scientific Reports, 14, Article No. 2961. https://doi.org/10.1038/s41598-024-52653-9
Fan, Y. and Waldmann, P. (2024) Tabular Deep Learning: A Comparative Study Applied to Multi-Task Genome-Wide Prediction. BMC Bioinformatics, 25, Article No. 322. https://doi.org/10.1186/s12859-024-05940-1
Schuran, M., Goudey, B., Dite, G.S. and Makalic, E. (2025) A Survey on Deep Learning for Polygenic Risk Scores. Briefings in Bioinformatics, 26, bbaf373. https://doi.org/10.1093/bib/bbaf373
Mukherjee, S., McCaw, Z.R., Pei, J., Merkoulovitch, A., Soare, T., Tandon, R., et al. (2024) EmbedGEM: A Framework to Evaluate the Utility of Embeddings for Genetic Discovery. Bioinformatics Advances, 4, vbae135. https://doi.org/10.1093/bioadv/vbae135
Hadizadeh, A., Tarokh, M.J. and Ghazani, M.M. (2025) A Novel Transformer-Based Dual Attention Architecture for the Prediction of Financial Time Series. Journal of King Saud University Computer and Information Sciences, 37, Article No. 72. https://doi.org/10.1007/s44443-025-00045-y
Wang, Y., Wang, Z., Kang, X. and Luo, Y. (2022) A Novel Interpretable Model Ensemble Multivariate Fast Iterative Filtering and Temporal Fusion Transform for Carbon Price Forecasting. Energy Science & Engineering, 11, 1148-1179. https://doi.org/10.1002/ese3.1380
Zhou, Y., Wang, X., Zhang, M., Zhu, J., Zheng, R. and Wu, Q. (2019) MPCE: A Maximum Probability Based Cross Entropy Loss Function for Neural Network Classification. IEEE Access, 7, 146331-146341. https://doi.org/10.1109/access.2019.2946264
Ho, Y. and Wookey, S. (2020) The Real-World-Weight Cross-Entropy Loss Function: Modeling the Costs of Mislabeling. IEEE Access, 8, 4806-4813. https://doi.org/10.1109/access.2019.2962617
Palla, K., Hyland, S.L., Posner, K., Ghosh, P., Nair, B., Bristow, M., et al. (2022) Intraoperative Prediction of Postanaesthesia Care Unit Hypotension. British Journal of Anaesthesia, 128, 623-635. https://doi.org/10.1016/j.bja.2021.10.052
Purnomo, T.D. and Sutopo, J. (2024) Comparison of Pre-Trained Bert-Based Transformer Models for Regional Language Text Sentiment Analysis in Indonesia. International Journal Science and Technology, 3, 11-21. https://doi.org/10.56127/ijst.v3i3.1739
Homann, J., Osburg, T., Ohlei, O., Dobricic, V., Deecke, L., Bos, I., et al. (2022) Genome-wide Association Study of Alzheimer’s Disease Brain Imaging Biomarkers and Neuropsychological Phenotypes in the European Medical Information Framework for Alzheimer’s Disease Multimodal Biomarker Discovery Dataset. Frontiers in Aging Neuroscience, 14, Article 840651. https://doi.org/10.3389/fnagi.2022.840651
Taylor, H., Lewins, M., Foody, M.G.B., Gray, O., Bešević, J., Conroy, M.C., et al. (2025) UK Biobank—A Unique Resource for Discovery and Translation Research on Genetics and Neurologic Disease. Neurology Genetics, 11, e200226. https://doi.org/10.1212/nxg.0000000000200226
Ching, C.R.K., Kang, M.J.Y. and Thompson, P.M. (2024) Large-Scale Neuroimaging of Mental Illness. In: Paus, T., Brook, J.R., Keyes, K. and Pausova, Z., Eds., Principles and Advances in Population Neuroscience, Springer, 371-397. https://doi.org/10.1007/7854_2024_462
