|
|
Open Access Library Journal 11 2026
Visual Semiotics-Based Emotional Expression and Cross-Cultural Visual Narrative Construction in AI Text-to-Image Generation
|
Abstract:
With the rapid advancement of multimodal generative artificial intelligence, Text-to-Image technology is transitioning from simple pixel imitation to advanced semantic and emotional generation. This study selects two contrasting emotional themes, “loneliness” and “warmth”, as experimental samples to compare the performance of international models (Leonardo.Ai) and Chinese domestic counterparts (Doubao, Kling) in visualizing abstract emotions. Grounded in Roland Barthes’ theory of visual semiotics, this paper establishes an evaluation framework encompassing Signifier-Signified Matching Degree and Cultural Adaptability and Innovation to systematically analyze how different models translate abstract emotions into concrete visual symbols (color, lighting, and composition). The findings reveal that domestic large models exhibit superior performance in high-context decoding and localized symbol construction within Chinese cultural settings, though they still face challenges in balancing photorealistic perfection with feature diversity. Finally, this paper discusses the shift in aesthetic intent of human creators and the pathways of human-AI co-creation in the AIGC era, aiming to provide theoretical and empirical references for understanding human-AI emotional interaction and digital visual narration.
| [1] | Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., <i>et al</i>. (2020) Generative Adversarial Networks. <i>Communications of the ACM</i>, 63, 139-144. <br>https://doi.org/10.1145/3422622 |
| [2] | Liu, H.L. (2024) Generative AI and Knowledge Production. <i>Editorial Friend</i>, 3, 5-13. (In Chinese) |
| [3] | Yu, G.M. and Su, J.W. (2023) Communication Revolution and Media Ecology in the Context of Generative AI—From ChatGPT to a Comprehensively Intelligent Future. <i>Journal of </i><i>Xinjiang</i><i> Normal University</i> (<i>Edition of Philosophy and Social Sciences</i>), 44, 81-90. (In Chinese) |
| [4] | Gan, L.H. and Wang, H. (2024) From Emotional Projection to Digital Emotion: The Emotional Evolution of Human-Computer Interaction in Digital Landscapes. <i>Modern Publishing</i>, 3, 27-38. (In Chinese) |
| [5] | Bender, E.M., Gebru, T., McMillan-Major, A. and Shmitchell, S. (2021) On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? <i>Proceedings of the </i>2021 <i>ACM Conference on Fairness</i>,<i> Accountability</i>,<i> and Transparency</i>, Virtual Event, 3-10 March 2021, 610-623. <br>https://doi.org/10.1145/3442188.3445922 |
| [6] | Su, T. and Peng, L. (2025) Communication and the New Human Condition in the Generative AI Era—An Overview of New Media Research in 2024. <i>Chinese Journal of Journalism & Communication</i>, 47, 53-70. (In Chinese) |
| [7] | Malchiodi, C.A. (2011) Handbook of art Therapy. Guilford Press. |
| [8] | Westberg, G. and Kvåle, G. (2025) The Generic Uniqueness of AI Imagery: A Critical Approach to Dall-E as Semiotic Technology. <i>Discourse & Society</i>, 36, 575-598. <br>https://doi.org/10.1177/09579265241274788 |
| [9] | Strikovic, E. and Cools, H. (2025) Reality Re-Imag(in)Ed. Mapping Publics’ Perceptions and Evaluations of Ai-Generated Images in News Contexts. <i>Digital Journalism</i>, 14, 1301-1323. <br>https://doi.org/10.1080/21670811.2025.2573076 |
| [10] | Ye, H. (2026) Algorithmic Reproduction of Occupational Stigma: Visual Prototype Lock-In in Chinese Text-to-Image AI Models. <i>Social Semiotics</i>, 36, 1-30. <br>https://doi.org/10.1080/10350330.2026.2659880 |
| [11] | Putland, E., Chikodzore-Paterson, C. and Brookes, G. (2023) Artificial Intelligence and Visual Discourse: A Multimodal Critical Discourse Analysis of AI-Generated Images of “Dementia”. <i>Social Semiotics</i>, 35, 228-253. <br>https://doi.org/10.1080/10350330.2023.2290555 |
| [12] | Hart, C. (2025) Using Generative AI to Investigate the Multimodal Representation of Refugees and Migrants. <i>Discourse & Society</i>, 37, 470-502. <br>https://doi.org/10.1177/09579265251378665 |
| [13] | Barthes, R. (1977) Rhetoric of the Image. In: Heath, S., Ed. and Trans., <i>Image</i>,<i> Music</i>,<i> Text</i>, Fontana, 32-51. |