We present EAGAA, an emotionally intelligent agent integrating real-time sentiment and emotion analysis with Retrieval-Augmented Generation (RAG) to deliver empathetic customer support at scale. EAGAA employs a multilayered pipeline: SentimentAgent, ManagerAgent, TaskAgents, ResponseGenerator, and SessionManager to adapt tone and personality dynamically. In extensive evaluations including 100 simulated sessions and a 200-user survey, EAGAA achieved a 77% F1 on emotion benchmarks, 98.6% top-1 retrieval accuracy, sub-second median latency, and a 4.85/5 CSAT score, outperforming baselines by over 20%. This work demonstrates the viability of human-like, scalable emotion-aware AI.
References
[1]
Designs, C.S. (2016) 89% of Companies Compete Mostly on Customer Experience. https://www.cmsystemdesigns.com/blog/companie-compete-mostly-on-customer-experience
[2]
McKinsey Company (2021) The Value of Getting Personalization Right or Wrong Is Multiplying. https://www.mckinsey.com/capabilities/growth-marketing-and-sales/our-insights/the-value-of-getting-personalization-right-or-wrong-is-multiplying
[3]
Accenture (2022) Accenture Report Finds 3.5x Revenue Growth for Companies That View Customer Service as a Value Center. https://newsroom.accenture.com/news/2022/accenture-report-finds-3-5x-revenue-growth-for-companies-that-view-customer-service-as-a-value-center
[4]
PWC (2020) 32% of Customers Will Walk away from a Brand They Love after a Single Bad Experience. https://visionpoint.systems/statistic/32-of-customers-will-walk-away-from-a-brand-they-love-after-a-single-bad-experience/
[5]
Salesforce (2020) State of the Connected Customer. https://c1.sfdcstatic.com/content/dam/web/en_us/www/documents/research/salesforce-state-of-the-connected-customer-4th-ed.pdf
[6]
Microsoft (2020) Retail Trends Playbook 2020. https://info.microsoft.com/rs/157-GQE-382/images/EN-CNTNT-eBook-RetailTrendsPlaybook2020.pdf
[7]
Raileanu, G. (2025) CSAT: Definition, Calculation & 2025 Benchmarks. https://www.retently.com/blog/customer-satisfaction-score-csat/
[8]
Bain & Company (2024) Measuring your net promoter score system. https://www.netpromotersystem.com/about/measuring-your-net-promoter-score
[9]
Cheng, Z.B., Cheng, Z.-Q., He, J-Y., et al. (2024) Emotion-Llama: Multimodal Emotion Recognition and Reasoning with Instruction Tuning. https://arxiv.org/pdf/2406.11161v1
[10]
Gamage, G., De Silva, D., Mills, N., Alahakoon, D. and Manic, M. (2024) Emotion AWARE: An Artificial Intelligence Framework for Adaptable, Robust, Explainable, and Multi-Granular Emotion Analysis. Journal of Big Data, 11, Article No 93. https://doi.org/10.1186/s40537-024-00953-2
[11]
Kim, J., Hong, J. and Choi, Y. (2024) Causal Inference for Modality Debiasing in Multimodal Emotion Recognition. Applied Sciences, 14, 11397. https://doi.org/10.3390/app142311397
[12]
UNESCO-IRCAI (2024) Challenging Systematic Prejudices: An Investigation into Bias Against Women and Girls in Large Language Models. https://ircai.org/project/
[13]
Mohammad, S.M. and Turney, P.D. (2013) NRC Emotion Lexicon. National Research Council Canada.
[14]
Pang, B. and Lee, L. (2008) Opinion Mining and Sentiment Analysis. Foundations and Trends in Information Retrieval, 2, 1-135. https://doi.org/10.1561/1500000011
[15]
Kim, Y. (2014) Convolutional Neural Networks for Sentence Classification. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, October 2014, 1746-1751. https://doi.org/10.3115/v1/d14-1181
[16]
Devlin, J., Chang, M.W., Lee, K. and Toutanova, K. (2019) Bert: Pre-Training of Deep Bididirectional Transformers for Language Understanding. NAACL-HLT, Minneapolis, 2-7 June 2019, 4171-4186.
[17]
Cao, Y., Li, X. and Li, J. (2022) Model Priming with Triplet Loss for Few-Shot Emotion Classification in Text. The 2022 Conference on Empirical Methods in Natural Language Processing, Abu Dhabi, 7-11 December 2022, 1234-1245.
[18]
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K. and Samek, W. (2015) On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation. PLOS ONE, 10, e0130140. https://doi.org/10.1371/journal.pone.0130140
[19]
Blodgett, S.L., Green, L. and O’Connor, B. (2016) Demographic Dialectal Variation in Social Media: A Case Study of African-American English. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, November 2016, 1119-1130. https://doi.org/10.18653/v1/d16-1120
[20]
Mesquita, B. and Frijda, N.H. (1992) Cultural Variations in Emotions: A Review. Psychological Bulletin, 112, 179-204. https://doi.org/10.1037//0033-2909.112.2.179
[21]
Lian, H., Lu, C., Li, S., Zhao, Y., Tang, C. and Zong, Y. (2023) A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face. Entropy, 25, 1440. https://doi.org/10.3390/e25101440
[22]
Umair, M., Rashid, N., Shahbaz Khan, U., Hamza, A. and Iqbal, J. (2024) Emotion Fusion-Sense (Emo Fu-Sense)—A Novel Multimodal Emotion Classification Technique. Biomedical Signal Processing and Control, 94, 106224. https://doi.org/10.1016/j.bspc.2024.106224
[23]
Wang, X., Li, Y.Z., Fu, C.Y., Shen, Y.H., et al. (2024) Freeze-Omni: A Smart and Low-Latency Speech-to-Speech Dialogue Model. https://arxiv.org/abs/2411.00774
[24]
Nayinzira, J.P. and Adda, M. (2024) Sentimentcarebot: Retrieval-Augmented Generation Chatbot for Mental Health Support with Sentiment Analysis. Procedia Computer Science, 251, 334-341. https://doi.org/10.1016/j.procs.2024.11.118
[25]
Roselli, C., Lapomarda, L. and Datteri, E. (2025) How Culture Modulates Anthropomorphism in Human-Robot Interaction: A Review. Acta Psychologica, 255, 104871. https://doi.org/10.1016/j.actpsy.2025.104871
[26]
Li, A.J., Krishna, S. and Lakkaraju, H. (2025) More RLHF, More Trust? on the Impact of Preference Alignment on Trustworthiness. International Conference on Learning Representations. https://github.com/AI4LIFE-GROUP/RLHF_Trust
[27]
Sharma, R., Mehta, M. and Raina, S.T. (2025) RLHF: A Comprehensive Survey for Cultural, Multimodal and Low Latency Alignment Methods. arXiv preprint, 2025. http://arxiv.org/abs/2511.03939
[28]
Alyoubi, A.A. and Alyoubi, B.A. (2025) Interpretable Multimodal Emotion Recognition Using Optimized Transformer Model with SHAP-Based Transparency. The Journal of Supercomputing, 81, Article No. 1044. https://doi.org/10.1007/s11227-025-07515-0
[29]
Poria, S., Hazarika, D., Majumder, N., Naik, G., Cambria, E. and Mihalcea, R. (2019) MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, 28 July-2 August 2019, 527-536. https://doi.org/10.18653/v1/p19-1050
[30]
SQM Group (2025) Call Center FCR Benchmark 2024 Results by Industry. https://www.sqmgroup.com/resources/library/blog/call-center-fcr-benchmark-2024-results-by-industry
