This paper presents our endeavors in developing the large-scale, ultra-high-resolution E3SM Land Model (uELM), specifically designed for exascale computers furnished with accelerators such as Nvidia GPUs. The uELM is a sophisticated code that substantially relies on High-Performance Computing (HPC) environments, necessitating particular machine and software configurations. To facilitate community-based uELM developments employing GPUs, we have created a portable, standalone software environment preconfigured with uELM input datasets, simulation cases, and source code. This environment, utilizing Docker, encompasses all essential code, libraries, and system software for uELM development on GPUs. It also features a functional unit test framework and an offline model testbed for comprehensive numerical experiments. From a technical perspective, the paper discusses GPU-ready container generations, uELM code management, and input data distribution across computational platforms. Lastly, the paper demonstrates the use of environment for functional unit testing, end-to-end simulation on CPUs and GPUs, and collaborative code development.
References
[1]
Golaz, J.-C., Caldwell, P.M., Van Roekel, L.P., Petersen, M.R., Tang, Q., Wolfe, J.D., Abeshu, G., Anantharaj, V., Asay-Davis, X.S., Bader, D.C., et al. (2019) The Doe e3sm Coupled Model Version 1: Overview and Evaluation at Standard Resolution. Journal of Advances in Modeling Earth Systems, 11, 2089-2129.
[2]
Burrows, S.M., Maltrud, M., Yang, X., Zhu, Q., Jeffery, N., Shi, X., et al. (2020) The DOE E3SM V1.1 Biogeochemistry Configuration: Description and Simulated Ecosystem‐Climate Responses to Historical Changes in Forcing. Journal of Advances in Modeling Earth Systems, 12, 9. https://doi.org/10.1029/2019ms001766
[3]
Xu, Y., Wang, D., Janjusic, T., Wu, W., Pei, Y. and Yao, Z. (2017) A Web-Based Visual Analytic Framework for Understanding Large-Scale Environmental Models: A Use Case for the Community Land Model. Procedia Computer Science, 108, 1731-1740. https://doi.org/10.1016/j.procs.2017.05.181
[4]
Zheng, W., Wang, D. and Song, F. (2019) Xscan: An Integrated Tool for Understanding Open Source Community-Based Scientific Code. In: Lecture Notes in Computer Science, Springer International Publishing, Faro, 226-237. https://doi.org/10.1007/978-3-030-22734-0_17
[5]
Wang, D., Schwartz, P., Yuan, F., Thornton, P. and Zheng, W. (2022) Toward Ultrahigh-Resolution E3SM Land Modeling on Exascale Computers. Computing in Science & Engineering, 24, 44-53. https://doi.org/10.1109/mcse.2022.3218990
[6]
Yuan, F., Wang, D., Kao, S., Thornton, M., Ricciuto, D., Salmon, V., et al. (2023) An Ultrahigh-Resolution E3SM Land Model Simulation Framework and Its First Application to the Seward Peninsula in Alaska. Journal of Computational Science, 73, Article ID: 102145. https://doi.org/10.1016/j.jocs.2023.102145
[7]
Schwartz, P., Wang, D., Yuan, F. and Thornton, P. (2022) SPEL: Software Tool for Porting E3SM Land Model with Openacc in a Function Unit Test Framework. 2022 Workshop on Accelerator Programming Using Directives (WACCPD), Dallas, TX, 13-18 November 2022, 43-51. https://doi.org/10.1109/waccpd56842.2022.00010
[8]
Schwartz, P., Wang, D., Yuan, F. and Thornton, P. (2022) Developing an Elm Ecosystem Dynamics Model on Gpu with Openacc. Computational Science-ICCS 2022: 22nd International Conference, London, UK, 21-23 June 2022, 291-303.
[9]
Wang, D., Xu, Y., Thornton, P., King, A., Steed, C., Gu, L., et al. (2014) A Functional Test Platform for the Community Land Model. Environmental Modelling & Software, 55, 25-31. https://doi.org/10.1016/j.envsoft.2014.01.015
