全部 标题 作者
关键词 摘要

OALib Journal期刊
ISSN: 2333-9721
费用:99美元

查看量下载量

Reinforcement Learning-Based Personalized Depression Treatment Using Synthetic Data and Real-Time Decision Support

DOI: 10.4236/oalib.1113959, PP. 1-17

Subject Areas: Simulation/Analytical Evaluation of Communication Systems, Big Data Search and Mining, Applications of Communication Systems, Psychiatry & Psychology

Keywords: Reinforcement Learning, Depression, Synthetic Data, Real-Time Recommendation, Precision Psychiatry, Deep Q-Learning, Treatment Personalization, Patient Simulation

Full-Text   Cite this paper   Add to My Lib

Abstract

Depression treatment often involves a complex and lengthy trial-and-error process, where clinicians sequentially prescribe medications to identify the most effective treatment for each patient. This approach can lead to delayed recovery, unnecessary side effects, and increased healthcare costs. To address this challenge, we present a Reinforcement Learning (RL)-based framework designed to optimize antidepressant treatment strategies through dynamic, patient-specific decision-making. The proposed system leverages synthetically generated patient data to simulate real-world treatment scenarios while ensuring privacy and scalability. A synthetic depression patient simulator was developed to model daily symptom trajectories influenced by medication type, dosage, adherence, side effects, and stochastic life events. This simulated data allowed the training of a deep Q-learning agent within a custom-built reinforcement learning environment. The agent learned to recommend treatment adjustments or continuations based on temporal symptom patterns and treatment history. Key components of the framework include experience replay, target model updates, and an epsilon-greedy exploration strategy to balance exploration and exploitation during training. The system was evaluated using unseen synthetic patients to assess generalization performance. Comprehensive visual analyses were conducted to characterize the symptom distribution, medication assignment, agent reward dynamics, and real-time treatment recommendations. The real-time recommendation system demonstrated the ability to provide timely, personalized treatment suggestions, switching medications when appropriate and main-taining stability when patient symptoms improved. The model’s decision-making process is closely aligned with clinical reasoning, supporting its potential as a decision support tool in precision psychiatry. This study offers a privacy-preserving, scalable, and clinically relevant pathway for optimizing depression treatment through reinforcement learning, contributing to the advancement of intelligent mental health care systems.

Cite this paper

Filippis, R. D. and Foysal, A. A. (2025). Reinforcement Learning-Based Personalized Depression Treatment Using Synthetic Data and Real-Time Decision Support. Open Access Library Journal, 12, e13959. doi: http://dx.doi.org/10.4236/oalib.1113959.

References

[1]  Donohue, J.M. and Pincus, H.A. (2007) Reducing the Societal Burden of De-pression: A Review of Economic Costs, Quality of Care and Effects of Treatment. PharmacoEconomics, 25, 7-24. https://doi.org/10.2165/00019053-200725010-00003
[2]  Briley, M. and Lépine, (2011) The Increasing Burden of Depression. Neuropsychiatric Disease and Treatment, 7, 3-7. https://doi.org/10.2147/ndt.s19617
[3]  World Health Organization (2022) World Mental Health Report: Transforming Mental Health for All. World Health Organization.
[4]  Montano, C.B., Jackson, W.C., Vanacore, D. and Weisler, R. (2023) Considerations When Selecting an Antide-pressant: A Narrative Review for Primary Care Providers Treating Adults with Depression. Postgraduate Medicine, 135, 449-465. https://doi.org/10.1080/00325481.2023.2189868
[5]  Kroenke, K., Miksch, T.A., Spaulding, A.C., Mazza, G.L., DeStephano, C.C., Niazi, S.K., et al. (2022) Choosing and Using Patient-Reported Outcome Measures in Clinical Practice. Archives of Physical Medicine and Rehabilitation, 103, S108-S117. https://doi.org/10.1016/j.apmr.2020.12.033
[6]  Pasterkiewicz, U. (2017) Patients’ Accounts of Non-Acceptance and Non-Adherence to Drug Treatment in Depression: A Scoping Review and Narrative Synthesis of Research Findings on Patients’ Views on Antidepressants. Ph.D. Thesis, University of Water-loo.
[7]  Fayers, P.M. and Machin, D. (2013) Quality of Life: The Assessment, Analysis and Interpretation of Patient-Reported Outcomes. John Wiley & Sons.
[8]  Hengartner, M.P. (2022) Evidence-Biased Antidepressant Prescrip-tion: Overmedicalisation, Flawed Research, and Conflicts of Interest. Palgrave Macmillan.
[9]  Eap, C.B. (2016) Personalized Prescribing: A New Medical Model for Clinical Implementation of Psychotropic Drugs. Dialogues in Clinical Neuroscience, 18, 313-322. https://doi.org/10.31887/dcns.2016.18.3/ceap
[10]  Miller, D.B. and O’Callaghan, J.P. (2013) Personalized Medicine in Major Depressive Disor-der—Opportunities and Pitfalls. Metabolism, 62, S34-S39. https://doi.org/10.1016/j.metabol.2012.08.021
[11]  Perlis, R.H. (2016) Abandoning Personalization to Get to Precision in the Pharmacotherapy of De-pression. World Psychiatry, 15, 228-235. https://doi.org/10.1002/wps.20345
[12]  Ivany, E. and Lane, D.A. (2020) Pa-tient Satisfaction: A Key Component in Increasing Treatment Adherence and Persistence. Thrombosis and Haemostasis, 121, 255-257. https://doi.org/10.1055/s-0040-1718734
[13]  Baryakova, T.H., Pogostin, B.H., Langer, R. and McHugh, K.J. (2023) Overcoming Barriers to Patient Ad-herence: The Case for Developing Innovative Drug Delivery Systems. Nature Re-views Drug Discovery, 22, 387-409. https://doi.org/10.1038/s41573-023-00670-0
[14]  Kardas, P. (2024) From Non-Adherence to Adherence: Can Innovative Solutions Resolve a Longstanding Problem? European Journal of Internal Medicine, 119, 6-12. https://doi.org/10.1016/j.ejim.2023.10.012
[15]  Vermeire, E., Hearnshaw, H., Van Royen, P. and Denekens, J. (2001) Patient Adherence to Treatment: Three Decades of Research. A Comprehensive Review. Journal of Clinical Phar-macy and Therapeutics, 26, 331-342. https://doi.org/10.1046/j.1365-2710.2001.00363.x
[16]  Vangeli, E., Bakhshi, S., Baker, A., Fisher, A., Bucknor, D., Mrowietz, U., et al. (2015) A Sys-tematic Review of Factors Associated with Non-Adherence to Treatment for Immune-Mediated Inflammatory Diseases. Advances in Therapy, 32, 983-1028. https://doi.org/10.1007/s12325-015-0256-7
[17]  Riachi, E., Mamdani, M., Fralick, M. and Rudzicz, F. (2021) Challenges for Reinforcement Learning in Healthcare. arXiv: 2103.05612.
[18]  Costa, R.D. and Hirata, C.M. (2025) Rein-forcement Learning Applied to a Situation Awareness Decision-Making Model. Information Sciences, 704, Article 121928. https://doi.org/10.1016/j.ins.2025.121928
[19]  Hargrave, M., Spaeth, A. and Grosenick, L. (2024) EpiCare: A Reinforcement Learning Benchmark for Dy-namic Treatment Regimes. Advances in Neural Information Processing Systems, 37, 130536-130568.
[20]  Mashayekhi, H., Nazari, M., Jafarinejad, F. and Meskin, N. (2024) Deep Reinforcement Learning-Based Control of Chemo-Drug Dose in Cancer Treatment. Computer Methods and Programs in Biomedicine, 243, Article ID: 107884. https://doi.org/10.1016/j.cmpb.2023.107884
[21]  Teplytska, O., Ernst, M., Koltermann, L.M., Valderrama, D., Trunz, E., Vaisband, M., et al. (2024) Ma-chine Learning Methods for Precision Dosing in Anticancer Drug Therapy: A Scoping Review. Clinical Pharmacokinetics, 63, 1221-1237. https://doi.org/10.1007/s40262-024-01409-9
[22]  Li, L.C., Komorowski, M. and Faisal, A.A. (2019) Optimizing Sequential Medical Treatments with Au-to-Encoding Heuristic Search in POMDPS. arXiv: 1905.07465.
[23]  Ribba, B. (2023) Reinforcement Learning as an Innovative Model-Based Approach: Ex-amples from Precision Dosing, Digital Health and Computational Psychiatry. Frontiers in Pharmacology, 13, Article 1094281. https://doi.org/10.3389/fphar.2022.1094281
[24]  Olalekan Kehinde, A. (2025) Machine Learning in Predictive Modelling: Addressing Chronic Disease Management through Optimized Healthcare Processes. International Journal of Research Publication and Reviews, 6, 1525-1539.
[25]  Woodman, R.J. and Mangoni, A.A. (2023) A Comprehensive Review of Machine Learning Algo-rithms and Their Application in Geriatric Medicine: Present and Future. Aging Clinical and Experimental Research, 35, 2363-2397. https://doi.org/10.1007/s40520-023-02552-2
[26]  Bzdok, D. and Mey-er-Lindenberg, A. (2018) Machine Learning for Precision Psychiatry: Opportu-nities and Challenges. Biological Psychiatry: Cognitive Neuroscience and Neu-roimaging, 3, 223-230. https://doi.org/10.1016/j.bpsc.2017.11.007
[27]  Squartini, S., Lu, J. and Wei, Q. (2013) The Neural Paradigm for Complex Systems: New Algorithms and Ap-plications. Neural Computing and Applications, 22, 203-204. https://doi.org/10.1007/s00521-011-0713-4.
[28]  Baydili, İ., Tasci, B. and Tasci, G. (2025) Artificial Intelligence in Psychiatry: A Review of Biological and Behavioral Data Analyses. Diagnostics, 15, Article 434. https://doi.org/10.3390/diagnostics15040434
[29]  Ganatra, H.A. (2025) Machine Learning in Pediatric Healthcare: Current Trends, Challenges, and Fu-ture Directions. Journal of Clinical Medicine, 14, Article 807. https://doi.org/10.3390/jcm14030807
[30]  Thieme, A., Belgrave, D. and Doherty, G. (2020) Machine Learning in Mental Health: A Systematic Review of the HCI Literature to Support the Development of Effective and Implementable ML Systems. ACM Transactions on Computer-Human Interaction, 27, 1-53. https://doi.org/10.1145/3398069
[31]  Nag, P.K., Bhagat, A. and Priya, R.V. (2025) Expanding AI’s Role in Healthcare Applications: A Systematic Review of Emotional and Cognitive Analysis Techniques. IEEE Access, 13, 69129-69160.
[32]  Bouras, A. (2024) The Emerging Applications of Synthetic Data in Neurosurgery Re-search and Practice: A Systematic Review. medRxiv.
[33]  Giuffrè, M. and Shung, D.L. (2023) Harnessing the Power of Synthetic Data in Healthcare: Innovation, Application, and Privacy. npj Digital Medicine, 6, Article No. 186. https://doi.org/10.1038/s41746-023-00927-3
[34]  Pavlopoulos, A., Rachio-tis, T. and Maglogiannis, I. (2024) An Overview of Tools and Technologies for Anxiety and Depression Management Using AI. Applied Sciences, 14, Article 9068. https://doi.org/10.3390/app14199068
[35]  Gaggioli, A., Pallavicini, F., Morganti, L., Serino, S., Scaratti, C., Briguglio, M., et al. (2014) Experiential Vir-tual Scenarios with Real-Time Monitoring (Interreality) for the Management of Psychological Stress: A Block Randomized Controlled Trial. Journal of Medical In-ternet Research, 16, e167. https://doi.org/10.2196/jmir.3235
[36]  Ghazali, D.A., Breque, C., Sosner, P., Lesbordes, M., Chavagnat, J., Ragot, S., et al. (2019) Stress Response in the Daily Lives of Simulation Repeaters. a Randomized Con-trolled Trial Assessing Stress Evolution over One Year of Repetitive Immersive Simulations. PLOS ONE, 14, e0220111. https://doi.org/10.1371/journal.pone.0220111
[37]  Myin-Germeys, I., Oorschot, M., Collip, D., Lataster, J., Delespaul, P. and van Os, J. (2009) Experi-ence Sampling Research in Psychopathology: Opening the Black Box of Daily Life. Psychological Medicine, 39, 1533-1547. https://doi.org/10.1017/s0033291708004947
[38]  Neftci, E.O. and Aver-beck, B.B. (2019) Reinforcement Learning in Artificial and Biological Systems. Nature Machine Intelligence, 1, 133-143. https://doi.org/10.1038/s42256-019-0025-4
[39]  Ghesu, F., Georgescu, B., Zheng, Y., Grbic, S., Maier, A., Hornegger, J., et al. (2019) Multi-scale Deep Re-inforcement Learning for Real-Time 3D-Landmark Detection in CT Scans. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41, 176-189. https://doi.org/10.1109/tpami.2017.2782687
[40]  Calvaresi, D., Schumach-er, M. and Calbimonte, J. (2020) Agent-based Modeling for Ontology-Driven Analysis of Patient Trajectories. Journal of Medical Systems, 44, Article No. 158. https://doi.org/10.1007/s10916-020-01620-8
[41]  Eghbali, N., Alhanai, T. and Ghassemi, M.M. (2021) Patient-specific Sedation Management via Deep Reinforcement Learning. Frontiers in Digital Health, 3, Article 608893. https://doi.org/10.3389/fdgth.2021.608893
[42]  Busoniu, L., Babuska, R. and De Schutter, B. (2008) A Comprehensive Survey of Multiagent Reinforce-ment Learning. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Ap-plications and Reviews), 38, 156-172. https://doi.org/10.1109/tsmcc.2007.913919
[43]  Szepesvári, C. (2022) Al-gorithms for Reinforcement Learning. Springer.
[44]  Ipek, E., Mutlu, O., Mar-tínez, J.F. and Caruana, R. (2008) Self-Optimizing Memory Controllers: A Rein-forcement Learning Approach. ACM SIGARCH Computer Architecture News, 36, 39-50. https://doi.org/10.1145/1394608.1382172
[45]  Amershi, S., Cakmak, M., Knox, W.B. and Kulesza, T. (2014) Power to the People: The Role of Humans in Interactive Machine Learning. AI Magazine, 35, 105-120. https://doi.org/10.1609/aimag.v35i4.2513
[46]  Bani-Harouni, D., Pellegrini, C., özsoy, E., Keicher, M. and Navab, N. (2025) Language Agents for Hypothe-sis-driven Clinical Decision Making with Reinforcement Learning. arXiv: 2506.13474.
[47]  Deuschel, J., Ellington, C.N., Luo, Y.T., Lengerich, B.J., Friederich, P. and Xing, E.P. (2023) Contextualized Policy Recovery: Modeling and Interpreting Medical Decisions with Adaptive Imitation Learning. arXiv: 2310.07918.
[48]  Wen, M.N., Wan, Z.Y., Wang, J., Zhang, W.N. and Wen, Y. (2024) Reinforcing LLM Agents via Policy Optimization with Action Decomposi-tion. Advances in Neural Information Processing Systems, 37, 103774-103805.
[49]  Palaniyappan, L. (2019) Inefficient Neural System Sta-bilization: A Theory of Spontaneous Resolutions and Recurrent Relapses in Psychosis. Journal of Psychiatry and Neuroscience, 44, 367-383. https://doi.org/10.1503/jpn.180038
[50]  Deserno, L., Boehme, R., Heinz, A. and Schlagenhauf, F. (2013) Reinforcement Learning and Dopamine in Schizo-phrenia: Dimensions of Symptoms or Specific Features of a Disease Group? Frontiers in Psychiatry, 4, Article 172. https://doi.org/10.3389/fpsyt.2013.00172
[51]  Loh, M., Rolls, E.T. and Deco, G. (2007) A Dynamical Systems Hypothesis of Schizophrenia. PLOS Computa-tional Biology, 3, e228. https://doi.org/10.1371/journal.pcbi.0030228
[52]  Tesauro, G. and Kephart, J.O. (2002) Pricing in Agent Economies Using Multi-Agent Q-learning. Autono-mous Agents and Multi-Agent Systems, 5, 289-304. https://doi.org/10.1023/a:1015504423309
[53]  Hartmann, P. and Smets, F. (2018) The European Central Bank’s Monetary Policy during Its First 20 Years. Brookings Papers on Economic Activity, 2018, 1-146. https://doi.org/10.1353/eca.2018.0026
[54]  Hommes, C. and Zhu, M. (2014) Behavioral Learning Equilibria. Journal of Economic Theory, 150, 778-814. https://doi.org/10.1016/j.jet.2013.09.002
[55]  Gräßer, F., Tesch, F., Schmitt, J., Abraham, S., Malberg, H. and Zaunseder, S. (2021) A Pharma-ceutical Therapy Recommender System Enabling Shared Decision-Making. User Modeling and User-Adapted Interaction, 32, 1019-1062. https://doi.org/10.1007/s11257-021-09298-4
[56]  Suryadevara, C.K. (2020) Towards Personalized Healthcare—An Intelligent Medication Recom-mendation System. International Engineering Journal for Research & Develop-ment, 5, 16.
[57]  Ali, H. (2022) Reinforcement Learning in Healthcare: Opti-mizing Treatment Strategies, Dynamic Resource allocation, and Adaptive Clinical Decision-Making. International Journal of Computer Applications Technology and Research, 11, 88-104.
[58]  Williams, J.W., Gerrity, M., Holsinger, T., Dobscha, S., Gaynes, B. and Dietrich, A. (2007) Systematic Review of Multifaceted Inter-ventions to Improve Depression Care. General Hospital Psychiatry, 29, 91-116. https://doi.org/10.1016/j.genhosppsych.2006.12.003
[59]  Katon, W., Unützer, J. and Russo, J. (2010) Major Depression: The Importance of Clinical Characteristics and Treatment Response to Prognosis. Depression and Anxiety, 27, 19-26. https://doi.org/10.1002/da.20613
[60]  Kemp, A.H., Gordon, E., Rush, A.J. and Williams, L.M. (2008) Improving the Prediction of Treatment Response in Depression: Integration of Clinical, Cognitive, Psychophysiological, Neuroimaging, and Genetic Measures. CNS Spectrums, 13, 1066-1086. https://doi.org/10.1017/s1092852900017120
[61]  Kraus, C., Kadriu, B., Lanzenberger, R., Zarate, C.A. and Kasper, S. (2019) Prognosis and Improved Outcomes in Major Depression: A Review. Translational Psychiatry, 9, Article No. 127. https://doi.org/10.1038/s41398-019-0460-3
[62]  Grover, S., Gautam, S., Jain, A., Gautam, M. and Vahia, V. (2017) Clinical Practice Guidelines for the Management of Depression. Indian Journal of Psychiatry, 59, S34-S50. https://doi.org/10.4103/0019-5545.196973
[63]  Horsky, J., Schiff, G.D., Johnston, D., Mercincavage, L., Bell, D. and Middleton, B. (2012) Interface De-sign Principles for Usable Decision Support: A Targeted Review of Best Practices for Clinical Prescribing Interventions. Journal of Biomedical Informatics, 45, 1202-1216. https://doi.org/10.1016/j.jbi.2012.09.002
[64]  Solmi, M., Miola, A., Croatto, G., Pigato, G., Favaro, A., Fornaro, M., et al. (2021) How Can We Im-prove Antidepressant Adherence in the Management of Depression? A Target-ed Review and 10 Clinical Recommendations. Brazilian Journal of Psychiatry, 43, 189-202. https://doi.org/10.1590/1516-4446-2020-0935

Full-Text


Contact Us

service@oalib.com

QQ:3279437679

WhatsApp +8615387084133