|
|
基于强化学习的多智能体路径规划研究与应用
|
Abstract:
研究聚焦于智能仓储中AGV的路径规划问题，构建了基于强化学习的多智能体寻路算法。每个AGV能从环境和以往经验中学习，利用不同行为产生的奖励机制，训练智能体自主选择更高效策略，以达到预定目标。本研究在ACTOR-CRITIC算法基础上加入经验回放机制并采用了中心化训练和去中心化决策的方法，以提高智能体的路径规划效率。同时，将ACTOR-CRITIC算法在智能仓的环境下进行模拟训练，验证AGV的路径规划效果。
The research focuses on the path planning problem of AGV in intelligent warehousing, constructs a multi-agent path finding algorithm based on reinforcement learning. Each AGV can learn from the environment and previous experience, use the reward mechanism generated by different behaviors to train the agent to independently choose more efficient strategies to achieve the predetermined goal. In this research, an experience playback mechanism is added to the ACTOR-CRITIC algorithm, centralized training and decentralized decision-making methods are adopted to improve the path planning efficiency of the agent. At the same time, the ACTOR-CRITIC algorithm is simulated and trained in the environment of the smart warehouse to verify the path planning effect of the AGV.
| [1] | Foerster, J., Assael, I.A., De Freitas, N. and Whiteson, S. (2016) Learning to Communicate with Deep Multi-Agent Reinforce-ment Learning. arXiv: 1605.06676. https://arxiv.org/abs/1605.06676 |
| [2] | Sukhbaatar, S. and Fergus, R. (2016) Learning Multiagent Communication with Backpropagation. arXiv: 1605.07736.
https://arxiv.org/abs/1605.07736 |
| [3] | Wang, C. and Mao, J. (2019) Summary of AGV Path Planning. 2019 3rd Interna-tional Conference on Electronic Information Technology and Computer Engineering (EITCE), Xiamen, 18-20 October 2019, 332-335.
https://doi.org/10.1109/EITCE47263.2019.9094825 |
| [4] | Bai, X., Fielbaum, A., Kronmuller, M., Knoedler, L. and Alonso-Mora, J. (2022) Group-Based Distributed Auction Algorithms for Multi-Robot Task Assignment. IEEE Transactions on Automation Science and Engineering, 20, 1292- 1303. https://doi.org/10.1109/TASE.2022.3175040 |
| [5] | Hu, J., Niu, H., Carrasco, J., Lennox, B. and Arvin, F. (2022) Fault-Tolerant Cooperative Navigation of Networked UAV Swarms for For-est Fire Monitoring. Aerospace Science and Technology, 123, 107494.
https://doi.org/10.1016/j.ast.2022.107494 |
| [6] | Chen, M., Wang, T., Ota, K., Dong, M., Zhao, M. and Liu, A. (2020) In-telligent Resource Allocation Management for Vehicles Network: An A3C Learning Approach. Computer Communications, 151, 485-494.
https://doi.org/10.1016/j.comcom.2019.12.054 |
| [7] | Chen, M., Liu, W., Wang, T., Liu, A. and Zeng, Z. (2021) Edge Intel-ligence Computing for Mobile Augmented Reality with Deep Reinforcement Learning Approach. Computer Networks, 195, 108186.
https://doi.org/10.1016/j.comnet.2021.108186 |
| [8] | Garaffa, L.C., Basso, M., Konzen, A.A. and de Freitas, E.P. (2021) Reinforcement Learning for Mobile Robotics Exploration: A Survey. IEEE Transactions on Neural Networks and Learning Systems, 34, 3796-3810.
https://doi.org/10.1109/TNNLS.2021.3124466 |
| [9] | Wei, E., Wicke, D., Freelan, D. and Luke, S. (2018) Multiagent Soft q-Learning. arXiv: 1804.09817. |
| [10] | Foerster, J., et al. (2017) Stabilising Experience Replay for Deep Multi-Agent Reinforce-ment Learning. Proceedings of the 34th International Conference on Machine Learning, 70, 1146-1155. |
| [11] | Omidshafiei, S., et al. (2017) Deep Decentralized Multi-Task Multi-Agent Reinforcement Learning Under Partial Observability. Proceedings of the 34th International Conference on Machine Learning, 70, 2681-2690. |
| [12] | Oliehoek, F.A., Spaan, M.T.J. and Vlassis, N. (2008) Optimal and Approximate Q-Value Functions for Decentralized POMDPs. Journal of Artificial Intelligence Research, 32, 289-353. https://doi.org/10.1613/jair.2447 |
| [13] | Oliehoek, F.A. and Amato, C. (2016) A Concise Introduction to Decen-tralized POMDPs. Springer International Publishing, Switzerland. https://doi.org/10.1007/978-3-319-28929-8 |
| [14] | Lowe, R., et al. (2017) Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments. arXiv: 1706.02275.
https://arxiv.org/abs/1706.02275 |
