|
基于车边云协同的服务迁移优化策略研究
|
Abstract:
随着车联网的发展和智能交通系统的完善,移动边缘计算和边缘缓存技术不断发展,在车联网服务迁移问题上发挥了巨大作用。为了更加合理利用边缘缓存资源、减小时延,本文提出了一种联合优化时延和缓存策略的“车–边–云”协同服务迁移方案。在该架构下,网络中的车载终端、路侧单元和云服务器可以协同进行任务计算,进行动态服务迁移。通过使用基于Hawkes过程的方法,根据历史请求信息更新不同内容类型的流行度,选择合适的边缘计算节点和缓存内容。本文以最大化系统的长期效益为目标,降低时延和提高命中率,形式化服务迁移问题。这是一个具有挑战性的非凸问题。通过将该问题建模为马尔科夫决策过程,并引入深度强化学习求解最优问题,提出了一种结合长短期记忆网络和近端策略优化算法的服务迁移算法。仿真实验结果表明,提出的策略优化方法比其他策略有更好的性能。
With the development of the Internet of Vehicles (IoV) and the improvement of intelligent transportation system, mobile edge computing (MEC) and edge cache technology are constantly developing, which play a huge role in the service migration of the Internet of vehicles. In order to make more reasonable use of edge cache resources and reduce the delay, this paper proposes a “Vehicle-Edge-Cloud” collaborative service migration scheme which jointly optimizes the delay and cache strategy. In this architecture, vehicles, Roadside Units (RSUs) and cloud server in the network can collaborate on task computation and dynamic service migration. By using a Hawkes process-based approach, the popularity of different content types is updated based on historical request information, and appropriate edge compute nodes and cached content are selected. To maximize the long-term benefits of the system, reduce the delay and improve the hit rate, formalize the problem of service migration. This is a challenging non-convex problem. By modeling the problem as Markov Decision Process (MDP) and introducing deep reinforcement learning (DRL) to solve the optimal problem, a service migration algorithm combining Long Short-Term Memory (LSTM) and Proximal Policy Optimization (PPO) algorithm is proposed. Simulation results show that the proposed strategy optimization method has better performance than other strategies.
[1] | Sun, Y., Hu, Y., Zhang, H., Chen, H. and Wang, F. (2023) A Parallel Emission Regulatory Framework for Intelligent Transportation Systems and Smart Cities. IEEE Transactions on Intelligent Vehicles, 8, 1017-1020. https://doi.org/10.1109/tiv.2023.3246045 |
[2] | Zhang, Y., Zhao, Y. and Zhou, Y. (2022) User-Centered Cooperative-Communication Strategy for 5G Internet of Vehicles. IEEE Internet of Things Journal, 9, 13486-13497. https://doi.org/10.1109/jiot.2022.3143124 |
[3] | Anwar, M.R., Wang, S., Akram, M.F., Raza, S. and Mahmood, S. (2022) 5G-Enabled MEC: A Distributed Traffic Steering for Seamless Service Migration of Internet of Vehicles. IEEE Internet of Things Journal, 9, 648-661. https://doi.org/10.1109/jiot.2021.3084912 |
[4] | Tang, H., Wu, H., Zhao, Y. and Li, R. (2022) Joint Computation Offloading and Resource Allocation under Task-Overflowed Situations in Mobile-Edge Computing. IEEE Transactions on Network and Service Management, 19, 1539-1553. https://doi.org/10.1109/tnsm.2021.3135389 |
[5] | Park, C. and Lee, J. (2021) Mobile Edge Computing-Enabled Heterogeneous Networks. IEEE Transactions on Wireless Communications, 20, 1038-1051. https://doi.org/10.1109/twc.2020.3030178 |
[6] | Peng, Y., Tang, X., Zhou, Y., Li, J., Qi, Y., Liu, L., et al. (2024) Computing and Communication Cost-Aware Service Migration Enabled by Transfer Reinforcement Learning for Dynamic Vehicular Edge Computing Networks. IEEE Transactions on Mobile Computing, 23, 257-269. https://doi.org/10.1109/tmc.2022.3225239 |
[7] | Wang, X., Ning, Z. and Guo, S. (2021) Multi-Agent Imitation Learning for Pervasive Edge Computing: A Decentralized Computation Offloading Algorithm. IEEE Transactions on Parallel and Distributed Systems, 32, 411-425. https://doi.org/10.1109/tpds.2020.3023936 |
[8] | Chen, X., Wu, C., Chen, T., Liu, Z., Zhang, H., Bennis, M., et al. (2022) Information Freshness-Aware Task Offloading in Air-Ground Integrated Edge Computing Systems. IEEE Journal on Selected Areas in Communications, 40, 243-258. https://doi.org/10.1109/jsac.2021.3126075 |
[9] | Lan, D., Taherkordi, A., Eliassen, F., Liu, L. and Yuan, X. (2023) Joint Optimization of Service Migration and Resource Management for Vehicular Edge Computing. 2023 19th International Conference on Distributed Computing in Smart Systems and the Internet of Things (DCOSS-IoT), Pafos, 19-21 June 2023, 155-160. https://doi.org/10.1109/dcoss-iot58021.2023.00036 |
[10] | Chen, W., Chen, Y. and Liu, J. (2023) Service Migration for Mobile Edge Computing Based on Partially Observable Markov Decision Processes. Computers and Electrical Engineering, 106, Article ID: 108552. https://doi.org/10.1016/j.compeleceng.2022.108552 |
[11] | Zhang, D., Wang, W., Zhang, J., Zhang, T., Du, J. and Yang, C. (2023) Novel Edge Caching Approach Based on Multi-Agent Deep Reinforcement Learning for Internet of Vehicles. IEEE Transactions on Intelligent Transportation Systems, 24, 8324-8338. https://doi.org/10.1109/tits.2023.3264553 |
[12] | Wu, H., Jin, J., Ma, H. and Xing, L. (2024) Federation-Based Deep Reinforcement Learning Cooperative Cache in Vehicular Edge Networks. IEEE Internet of Things Journal, 11, 2550-2560. https://doi.org/10.1109/jiot.2023.3292374 |
[13] | Zhou, H., Jiang, K., He, S., Min, G. and Wu, J. (2023) Distributed Deep Multi-Agent Reinforcement Learning for Cooperative Edge Caching in Internet-of-Vehicles. IEEE Transactions on Wireless Communications, 22, 9595-9609. https://doi.org/10.1109/twc.2023.3272348 |
[14] | Huang, J., Wan, J., Lv, B., Ye, Q. and Chen, Y. (2023) Joint Computation Offloading and Resource Allocation for Edge-Cloud Collaboration in Internet of Vehicles via Deep Reinforcement Learning. IEEE Systems Journal, 17, 2500-2511. https://doi.org/10.1109/jsyst.2023.3249217 |
[15] | 李媛媛, 付晓东, 刘骊, 等. 利用Hawkes过程模型的移动边缘计算服务质量预测[J]. 小型微型计算机系统, 2023, 44(7): 1571-1577. |
[16] | Ram, A. and Srijith, P.K. (2018) Accelerating Hawkes Process for Event History Data: Application to Social Networks and Recommendation Systems. 2018 10th International Conference on Communication Systems & Networks (COMSNETS), Bengaluru, 3-7 January 2018, 396-399. https://doi.org/10.1109/comsnets.2018.8328226 |
[17] | Khamari, S., Rachedi, A., Ahmed, T. and Mosbah, M. (2023) Adaptive Deep Reinforcement Learning Approach for Service Migration in MEC-Enabled Vehicular Networks. 2023 IEEE Symposium on Computers and Communications (ISCC), Gammarth, 9-12 July 2023, 1075-1079. https://doi.org/10.1109/iscc58397.2023.10218103 |