Integrating Multi-Agent Reinforcement Learning and Evolutionary Game Theory for Adaptive Virtual Bidding Strategies in Electricity Markets

As the electricity market is progressively liberalized, virtual bidding has emerged as a novel participation mechanism attracting increasing attention. This paper integrates evolutionary game theory with multi-agent reinforcement learning to propose an adaptive virtual bidding strategy for electricity market participants, enabling them to continuously adjust their bidding behavior in dynamic market environments to maximize individual profits. First, we apply multi-agent reinforcement learning to the fundamental virtual bidding model, incorporating risk constraints and corresponding limitations to construct a comprehensive bidding framework encompassing fixed costs, trigger conditions, and Conditional Value at Risk (CVaR) constraints. Second, we introduce heterogeneous virtual bidders with varying budget endowments and initialization configurations, rendering the model more representative of real-world electricity markets and thereby optimizing participants’ strategy selection. Finally, considering the collective impact of virtual bidders’ behavior on market prices and the influence of participant numbers on bidding strategies, we incorporate an evolutionary game model to analyze the evolutionary dynamics and equilibrium stability of virtual bidding strategies. Experimental results demonstrate that the MADDPG model achieves an average per-episode reward 13.7 times higher than that of independent DDPG, while exhibiting a five-fold faster convergence speed. The introduction of heterogeneous participants yields a 30.6% improvement in average returns compared with homogeneous settings. Evolutionary game analysis identifies multiple Nash equilibria across different scenarios, revealing the regulatory effects of participant numbers and price impact coefficients on equilibrium structures.