With the advancement of technology and the continuous innovation of applications, low-latency applications such as drones, online games and virtual reality are gradually becoming popular demands in modern society. However, these applications pose a great challenge to the traditional centralized mobile cloud computing paradigm, and it is obvious that the traditional cloud computing model is already struggling to meet such demands. To address the shortcomings of cloud computing, mobile edge computing has emerged. Mobile edge computing provides users with computing and storage resources by offloading computing tasks to servers at the edge of the network. However, most existing work only considers single-objective performance optimization in terms of latency or energy consumption, but not balanced optimization in terms of latency and energy consumption. To reduce task latency and device energy consumption, the problem of joint optimization of computation offloading and resource allocation in multi-cell, multi-user, multi-server MEC environments is investigated. In this paper, a dynamic computation offloading algorithm based on Multi-Agent Deep Deterministic Policy Gradient (MADDPG) is proposed to obtain the optimal policy. The experimental results show that the algorithm proposed in this paper reduces the delay by 5 ms compared to PPO, 1.5 ms compared to DDPG and 10.7 ms compared to DQN, and reduces the energy consumption by 300 compared to PPO, 760 compared to DDPG and 380 compared to DQN. This fully proves that the algorithm proposed in this paper has excellent performance.
References
[1]
Zeng, S., Li, Z., Yu, H., Zhang, Z., Luo, L., Li, B., et al. (2023) Hfedms: Heterogeneous Federated Learning with Memorable Data Semantics in Industrial Metaverse. IEEE Transactions on Cloud Computing, 11, 3055-3069. https://doi.org/10.1109/tcc.2023.3254587
[2]
Queralta, J.P., Qingqing, L., Zou, Z. and Westerlund, T. (2020). Enhancing Autonomy with Blockchain and Multi-Access Edge Computing in Distributed Robotic Systems. 2020 Fifth International Conference on Fog and Mobile Edge Computing (FMEC), Paris, 20-23 April 2020, 180-187. https://doi.org/10.1109/fmec49853.2020.9144809
[3]
Ma, X.T., Zhao, J.H., Gong, Y., et al. (2017) Key Technologies of MEC towards 5G-Enabled Vehicular Networks. In: Wang, L., Qiu, T. and Zhao, W., Eds., Quality, Reliability, Security and Robustness in HeterogeneousSystems, Springer, 153-159.
[4]
Andrews, J.G., Claussen, H., Dohler, M., Rangan, S. and Reed, M.C. (2012) Femtocells: Past, Present, and Future. IEEE Journal on Selected Areas in Communications, 30, 497-508. https://doi.org/10.1109/jsac.2012.120401
[5]
Dhillon, H.S., Ganti, R.K., Baccelli, F. and Andrews, J.G. (2012) Modeling and Analysis of K-Tier Downlink Heterogeneous Cellular Networks. IEEE Journal on Selected Areas in Communications,30, 550-560. https://doi.org/10.1109/jsac.2012.120405
[6]
Wu, T., Xu, L. and Liu, C. (2020) Research on 5G MEC Edge Intelligence Architecture. Information and Communication Technology, 14, 46-49. (In Chinese)
[7]
Ge, H.B., Feng, A.Q. and Wang, Y. (2020) Offloading Strategy for Workflow Tasks in 5G Edge Computing Environment. Sensors and Microsystems, 39, 130-133, 137. (In Chinese)
[8]
Xu, Z., Liang, W., Xu, W., Jia, M. and Guo, S. (2016) Efficient Algorithms for Capacitated Cloudlet Placements. IEEE Transactions on Parallel and Distributed Systems, 27, 2866-2880. https://doi.org/10.1109/tpds.2015.2510638
[9]
Zhu, X., Jia, Z., Pang, X. and Zhao, S. (2024) Joint Optimization of Task Caching and Computation Offloading for Multiuser Multitasking in Mobile Edge Computing. Electronics, 13, Article No. 389. https://doi.org/10.3390/electronics13020389
[10]
Yu, B.W., Pu, L.J., Xie, Y.T., et al. (2018) Research on Mobile Edge Computing Task Offloading and Base Station Association Collaborative Decision-Making Problem. Computer Research and Development, 55, 537-550. (In Chinese)
[11]
Liu, G.Q., Zhang, X.X., Ma, H.Y. and Yan, H. (2023) A Computational Task Offloading Algorithm for Multi-Edge Node Scenarios. Information and Control, 52, 679-688. (In Chinese)
[12]
Chen, Z. and Gong, B.C. (2024) UAV-Assisted Two-Layer Deep Reinforcement Learning Task Offloading Algorithm. Computer Applications Research, 41, 426-431. (In Chinese)
[13]
Sun, Y., Wei, T., Li, H., Zhang, Y. and Wu, W. (2020) Energy-Efficient Multimedia Task Assignment and Computing Offloading for Mobile Edge Computing Networks. IEEE Access, 8, 36702-36713. https://doi.org/10.1109/access.2020.2973359
[14]
Zhou, H., Jiang, K., Liu, X., Li, X. and Leung, V.C.M. (2022) Deep Reinforcement Learning for Energy-Efficient Computation Offloading in Mobile-Edge Computing. IEEE Internet of Things Journal,9, 1517-1530. https://doi.org/10.1109/jiot.2021.3091142
[15]
Chen, X., Ge, H., Liu, L., Li, S., Han, J. and Gong, H. (2021). Computing Offloading Decision Based on DDPG Algorithm in Mobile Edge Computing. 2021 IEEE 6th International Conference on Cloud Computing and Big Data Analytics (ICCCBDA), Chengdu, 24-26 April 2021, 391-399. https://doi.org/10.1109/icccbda51879.2021.9442599
[16]
Zhao, N., Ye, Z., Pei, Y., Liang, Y. and Niyato, D. (2022) Multi-Agent Deep Reinforcement Learning for Task Offloading in UAV-Assisted Mobile Edge Computing. IEEE Transactions on Wireless Communications, 21, 6949-6960. https://doi.org/10.1109/twc.2022.3153316
[17]
Tong, M., Wang, X., Li, S. and Peng, L. (2022) Joint Offloading Decision and Resource Allocation in Mobile Edge Computing-Enabled Satellite-Terrestrial Network. Symmetry, 14, Article No. 564. https://doi.org/10.3390/sym14030564
[18]
Yeganeh, S., Babazadeh Sangar, A. and Azizi, S. (2023) A Novel Q-Learning-Based Hybrid Algorithm for the Optimal Offloading and Scheduling in Mobile Edge Computing Environments. Journal of Network and Computer Applications, 214, Article ID: 103617. https://doi.org/10.1016/j.jnca.2023.103617
[19]
Zaman, S.K.u., Jehangiri, A.I., Maqsood, T., Haq, N.u., Umar, A.I., Shuja, J., et al. (2022) Limpo: Lightweight Mobility Prediction and Offloading Framework Using Machine Learning for Mobile Edge Computing. Cluster Computing, 26, 99-117. https://doi.org/10.1007/s10586-021-03518-7
[20]
Wu, H., Geng, J., Bai, X. and Jin, S. (2024) Deep Reinforcement Learning-Based Online Task Offloading in Mobile Edge Computing Networks. Information Sciences, 654, Article ID: 119849. https://doi.org/10.1016/j.ins.2023.119849
[21]
Lowe, R., Wu, Y.I., Tamar, A., et al. (2017) Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, 4-9 December 2017, 6382-6393.
