Depression treatment often involves a complex and lengthy trial-and-error process, where clinicians sequentially prescribe medications to identify the most effective treatment for each patient. This approach can lead to delayed recovery, unnecessary side effects, and increased healthcare costs. To address this challenge, we present a Reinforcement Learning (RL)-based framework designed to optimize antidepressant treatment strategies through dynamic, patient-specific decision-making. The proposed system leverages synthetically generated patient data to simulate real-world treatment scenarios while ensuring privacy and scalability. A synthetic depression patient simulator was developed to model daily symptom trajectories influenced by medication type, dosage, adherence, side effects, and stochastic life events. This simulated data allowed the training of a deep Q-learning agent within a custom-built reinforcement learning environment. The agent learned to recommend treatment adjustments or continuations based on temporal symptom patterns and treatment history. Key components of the framework include experience replay, target model updates, and an epsilon-greedy exploration strategy to balance exploration and exploitation during training. The system was evaluated using unseen synthetic patients to assess generalization performance. Comprehensive visual analyses were conducted to characterize the symptom distribution, medication assignment, agent reward dynamics, and real-time treatment recommendations. The real-time recommendation system demonstrated the ability to provide timely, personalized treatment suggestions, switching medications when appropriate and main-taining stability when patient symptoms improved. The model’s decision-making process is closely aligned with clinical reasoning, supporting its potential as a decision support tool in precision psychiatry. This study offers a privacy-preserving, scalable, and clinically relevant pathway for optimizing depression treatment through reinforcement learning, contributing to the advancement of intelligent mental health care systems.
Cite this paper
Filippis, R. D. and Foysal, A. A. (2025). Reinforcement Learning-Based Personalized Depression Treatment Using Synthetic Data and Real-Time Decision Support. Open Access Library Journal, 12, e13959. doi: http://dx.doi.org/10.4236/oalib.1113959.
Donohue, J.M. and Pincus, H.A. (2007) Reducing the Societal Burden of De-pression: A Review of Economic Costs, Quality of Care and Effects of Treatment. PharmacoEconomics, 25, 7-24. https://doi.org/10.2165/00019053-200725010-00003
Montano, C.B., Jackson, W.C., Vanacore, D. and Weisler, R. (2023) Considerations When Selecting an Antide-pressant: A Narrative Review for Primary Care Providers Treating Adults with Depression. Postgraduate Medicine, 135, 449-465. https://doi.org/10.1080/00325481.2023.2189868
Kroenke, K., Miksch, T.A., Spaulding, A.C., Mazza, G.L., DeStephano, C.C., Niazi, S.K., et al. (2022) Choosing and Using Patient-Reported Outcome Measures in Clinical Practice. Archives of Physical Medicine and Rehabilitation, 103, S108-S117. https://doi.org/10.1016/j.apmr.2020.12.033
Pasterkiewicz, U. (2017) Patients’ Accounts of Non-Acceptance and Non-Adherence to Drug Treatment in Depression: A Scoping Review and Narrative Synthesis of Research Findings on Patients’ Views on Antidepressants. Ph.D. Thesis, University of Water-loo.
Eap, C.B. (2016) Personalized Prescribing: A New Medical Model for Clinical Implementation of Psychotropic Drugs. Dialogues in Clinical Neuroscience, 18, 313-322. https://doi.org/10.31887/dcns.2016.18.3/ceap
Miller, D.B. and O’Callaghan, J.P. (2013) Personalized Medicine in Major Depressive Disor-der—Opportunities and Pitfalls. Metabolism, 62, S34-S39. https://doi.org/10.1016/j.metabol.2012.08.021
Perlis, R.H. (2016) Abandoning Personalization to Get to Precision in the Pharmacotherapy of De-pression. World Psychiatry, 15, 228-235. https://doi.org/10.1002/wps.20345
Ivany, E. and Lane, D.A. (2020) Pa-tient Satisfaction: A Key Component in Increasing Treatment Adherence and Persistence. Thrombosis and Haemostasis, 121, 255-257. https://doi.org/10.1055/s-0040-1718734
Baryakova, T.H., Pogostin, B.H., Langer, R. and McHugh, K.J. (2023) Overcoming Barriers to Patient Ad-herence: The Case for Developing Innovative Drug Delivery Systems. Nature Re-views Drug Discovery, 22, 387-409. https://doi.org/10.1038/s41573-023-00670-0
Kardas, P. (2024) From Non-Adherence to Adherence: Can Innovative Solutions Resolve a Longstanding Problem? European Journal of Internal Medicine, 119, 6-12. https://doi.org/10.1016/j.ejim.2023.10.012
Vermeire, E., Hearnshaw, H., Van Royen, P. and Denekens, J. (2001) Patient Adherence to Treatment: Three Decades of Research. A Comprehensive Review. Journal of Clinical Phar-macy and Therapeutics, 26, 331-342. https://doi.org/10.1046/j.1365-2710.2001.00363.x
Vangeli, E., Bakhshi, S., Baker, A., Fisher, A., Bucknor, D., Mrowietz, U., et al. (2015) A Sys-tematic Review of Factors Associated with Non-Adherence to Treatment for Immune-Mediated Inflammatory Diseases. Advances in Therapy, 32, 983-1028. https://doi.org/10.1007/s12325-015-0256-7
Hargrave, M., Spaeth, A. and Grosenick, L. (2024) EpiCare: A Reinforcement Learning Benchmark for Dy-namic Treatment Regimes. Advances in Neural Information Processing Systems, 37, 130536-130568.
Mashayekhi, H., Nazari, M., Jafarinejad, F. and Meskin, N. (2024) Deep Reinforcement Learning-Based Control of Chemo-Drug Dose in Cancer Treatment. Computer Methods and Programs in Biomedicine, 243, Article ID: 107884. https://doi.org/10.1016/j.cmpb.2023.107884
Teplytska, O., Ernst, M., Koltermann, L.M., Valderrama, D., Trunz, E., Vaisband, M., et al. (2024) Ma-chine Learning Methods for Precision Dosing in Anticancer Drug Therapy: A Scoping Review. Clinical Pharmacokinetics, 63, 1221-1237. https://doi.org/10.1007/s40262-024-01409-9
Li, L.C., Komorowski, M. and Faisal, A.A. (2019) Optimizing Sequential Medical Treatments with Au-to-Encoding Heuristic Search in POMDPS. arXiv: 1905.07465.
Ribba, B. (2023) Reinforcement Learning as an Innovative Model-Based Approach: Ex-amples from Precision Dosing, Digital Health and Computational Psychiatry. Frontiers in Pharmacology, 13, Article 1094281. https://doi.org/10.3389/fphar.2022.1094281
Olalekan Kehinde, A. (2025) Machine Learning in Predictive Modelling: Addressing Chronic Disease Management through Optimized Healthcare Processes. International Journal of Research Publication and Reviews, 6, 1525-1539.
Woodman, R.J. and Mangoni, A.A. (2023) A Comprehensive Review of Machine Learning Algo-rithms and Their Application in Geriatric Medicine: Present and Future. Aging Clinical and Experimental Research, 35, 2363-2397. https://doi.org/10.1007/s40520-023-02552-2
Bzdok, D. and Mey-er-Lindenberg, A. (2018) Machine Learning for Precision Psychiatry: Opportu-nities and Challenges. Biological Psychiatry: Cognitive Neuroscience and Neu-roimaging, 3, 223-230. https://doi.org/10.1016/j.bpsc.2017.11.007
Squartini, S., Lu, J. and Wei, Q. (2013) The Neural Paradigm for Complex Systems: New Algorithms and Ap-plications. Neural Computing and Applications, 22, 203-204. https://doi.org/10.1007/s00521-011-0713-4.
Baydili, İ., Tasci, B. and Tasci, G. (2025) Artificial Intelligence in Psychiatry: A Review of Biological and Behavioral Data Analyses. Diagnostics, 15, Article 434. https://doi.org/10.3390/diagnostics15040434
Thieme, A., Belgrave, D. and Doherty, G. (2020) Machine Learning in Mental Health: A Systematic Review of the HCI Literature to Support the Development of Effective and Implementable ML Systems. ACM Transactions on Computer-Human Interaction, 27, 1-53. https://doi.org/10.1145/3398069
Nag, P.K., Bhagat, A. and Priya, R.V. (2025) Expanding AI’s Role in Healthcare Applications: A Systematic Review of Emotional and Cognitive Analysis Techniques. IEEE Access, 13, 69129-69160.
Giuffrè, M. and Shung, D.L. (2023) Harnessing the Power of Synthetic Data in Healthcare: Innovation, Application, and Privacy. npj Digital Medicine, 6, Article No. 186. https://doi.org/10.1038/s41746-023-00927-3
Pavlopoulos, A., Rachio-tis, T. and Maglogiannis, I. (2024) An Overview of Tools and Technologies for Anxiety and Depression Management Using AI. Applied Sciences, 14, Article 9068. https://doi.org/10.3390/app14199068
Gaggioli, A., Pallavicini, F., Morganti, L., Serino, S., Scaratti, C., Briguglio, M., et al. (2014) Experiential Vir-tual Scenarios with Real-Time Monitoring (Interreality) for the Management of Psychological Stress: A Block Randomized Controlled Trial. Journal of Medical In-ternet Research, 16, e167. https://doi.org/10.2196/jmir.3235
Ghazali, D.A., Breque, C., Sosner, P., Lesbordes, M., Chavagnat, J., Ragot, S., et al. (2019) Stress Response in the Daily Lives of Simulation Repeaters. a Randomized Con-trolled Trial Assessing Stress Evolution over One Year of Repetitive Immersive Simulations. PLOS ONE, 14, e0220111. https://doi.org/10.1371/journal.pone.0220111
Myin-Germeys, I., Oorschot, M., Collip, D., Lataster, J., Delespaul, P. and van Os, J. (2009) Experi-ence Sampling Research in Psychopathology: Opening the Black Box of Daily Life. Psychological Medicine, 39, 1533-1547. https://doi.org/10.1017/s0033291708004947
Calvaresi, D., Schumach-er, M. and Calbimonte, J. (2020) Agent-based Modeling for Ontology-Driven Analysis of Patient Trajectories. Journal of Medical Systems, 44, Article No. 158. https://doi.org/10.1007/s10916-020-01620-8
Eghbali, N., Alhanai, T. and Ghassemi, M.M. (2021) Patient-specific Sedation Management via Deep Reinforcement Learning. Frontiers in Digital Health, 3, Article 608893. https://doi.org/10.3389/fdgth.2021.608893
Busoniu, L., Babuska, R. and De Schutter, B. (2008) A Comprehensive Survey of Multiagent Reinforce-ment Learning. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Ap-plications and Reviews), 38, 156-172. https://doi.org/10.1109/tsmcc.2007.913919
Amershi, S., Cakmak, M., Knox, W.B. and Kulesza, T. (2014) Power to the People: The Role of Humans in Interactive Machine Learning. AI Magazine, 35, 105-120. https://doi.org/10.1609/aimag.v35i4.2513
Bani-Harouni, D., Pellegrini, C., özsoy, E., Keicher, M. and Navab, N. (2025) Language Agents for Hypothe-sis-driven Clinical Decision Making with Reinforcement Learning. arXiv: 2506.13474.
Wen, M.N., Wan, Z.Y., Wang, J., Zhang, W.N. and Wen, Y. (2024) Reinforcing LLM Agents via Policy Optimization with Action Decomposi-tion. Advances in Neural Information Processing Systems, 37, 103774-103805.
Palaniyappan, L. (2019) Inefficient Neural System Sta-bilization: A Theory of Spontaneous Resolutions and Recurrent Relapses in Psychosis. Journal of Psychiatry and Neuroscience, 44, 367-383. https://doi.org/10.1503/jpn.180038
Deserno, L., Boehme, R., Heinz, A. and Schlagenhauf, F. (2013) Reinforcement Learning and Dopamine in Schizo-phrenia: Dimensions of Symptoms or Specific Features of a Disease Group? Frontiers in Psychiatry, 4, Article 172. https://doi.org/10.3389/fpsyt.2013.00172
Loh, M., Rolls, E.T. and Deco, G. (2007) A Dynamical Systems Hypothesis of Schizophrenia. PLOS Computa-tional Biology, 3, e228. https://doi.org/10.1371/journal.pcbi.0030228
Tesauro, G. and Kephart, J.O. (2002) Pricing in Agent Economies Using Multi-Agent Q-learning. Autono-mous Agents and Multi-Agent Systems, 5, 289-304. https://doi.org/10.1023/a:1015504423309
Hartmann, P. and Smets, F. (2018) The European Central Bank’s Monetary Policy during Its First 20 Years. Brookings Papers on Economic Activity, 2018, 1-146. https://doi.org/10.1353/eca.2018.0026
Gräßer, F., Tesch, F., Schmitt, J., Abraham, S., Malberg, H. and Zaunseder, S. (2021) A Pharma-ceutical Therapy Recommender System Enabling Shared Decision-Making. User Modeling and User-Adapted Interaction, 32, 1019-1062. https://doi.org/10.1007/s11257-021-09298-4
Suryadevara, C.K. (2020) Towards Personalized Healthcare—An Intelligent Medication Recom-mendation System. International Engineering Journal for Research & Develop-ment, 5, 16.
Ali, H. (2022) Reinforcement Learning in Healthcare: Opti-mizing Treatment Strategies, Dynamic Resource allocation, and Adaptive Clinical Decision-Making. International Journal of Computer Applications Technology and Research, 11, 88-104.
Williams, J.W., Gerrity, M., Holsinger, T., Dobscha, S., Gaynes, B. and Dietrich, A. (2007) Systematic Review of Multifaceted Inter-ventions to Improve Depression Care. General Hospital Psychiatry, 29, 91-116. https://doi.org/10.1016/j.genhosppsych.2006.12.003
Katon, W., Unützer, J. and Russo, J. (2010) Major Depression: The Importance of Clinical Characteristics and Treatment Response to Prognosis. Depression and Anxiety, 27, 19-26. https://doi.org/10.1002/da.20613
Kemp, A.H., Gordon, E., Rush, A.J. and Williams, L.M. (2008) Improving the Prediction of Treatment Response in Depression: Integration of Clinical, Cognitive, Psychophysiological, Neuroimaging, and Genetic Measures. CNS Spectrums, 13, 1066-1086. https://doi.org/10.1017/s1092852900017120
Kraus, C., Kadriu, B., Lanzenberger, R., Zarate, C.A. and Kasper, S. (2019) Prognosis and Improved Outcomes in Major Depression: A Review. Translational Psychiatry, 9, Article No. 127. https://doi.org/10.1038/s41398-019-0460-3
Grover, S., Gautam, S., Jain, A., Gautam, M. and Vahia, V. (2017) Clinical Practice Guidelines for the Management of Depression. Indian Journal of Psychiatry, 59, S34-S50. https://doi.org/10.4103/0019-5545.196973
Horsky, J., Schiff, G.D., Johnston, D., Mercincavage, L., Bell, D. and Middleton, B. (2012) Interface De-sign Principles for Usable Decision Support: A Targeted Review of Best Practices for Clinical Prescribing Interventions. Journal of Biomedical Informatics, 45, 1202-1216. https://doi.org/10.1016/j.jbi.2012.09.002
Solmi, M., Miola, A., Croatto, G., Pigato, G., Favaro, A., Fornaro, M., et al. (2021) How Can We Im-prove Antidepressant Adherence in the Management of Depression? A Target-ed Review and 10 Clinical Recommendations. Brazilian Journal of Psychiatry, 43, 189-202. https://doi.org/10.1590/1516-4446-2020-0935