With the development of the Industrial Internet of Things (IIoT) and cloud computing technologies, intelligent sensors in the field that can generate large volumes of time-series data continuously have emerged. Due to the lack of equipment and network impacts, highly distributed industrial applications cannot capture and transfer all production data to a distant cloud server in real time. Consequently, a portion of critical production data is lost, which poses the significant challenge of the timely replenishment of missing data. Employing deep learning in the cloud center for data trend prediction based on relevant data can solve this problem. The objective of this study was to develop a time-series prediction model that combines a Transformer model with a sparse Mixture of Experts (MoE). The model is designed specifically for an IIoT system that is used in oil-well operations. The proposed TransMoE prediction model combines the advantages of the MoE and the Transformer model. The MoE can effectively handle multiple subtasks while the Transformer algorithm can reflect the long-range dependency of the input data series. The proposed model was used to predict oil-well yields, and the predicted outcomes were compared with those obtained using a CNN-GRU and CNN-LSTM models, as well as the actual recorded data. The experimental results indicated that the proposed TransMoE model can significantly increase the efficiency and accuracy of oil well production sequence data prediction, with an average relative error of 6.26%, which can satisfy the requirements of enterprise data usage.
References
[1]
Obodoeze, F.C. (2012) Wireless Sensor Network in Niger Delta Oil and Gas Field Monitoring: The Security Challenges and Countermeasures. InternationalJournalofDistributedandParallelsystems, 3, 65-77. https://doi.org/10.5121/ijdps.2012.3606
[2]
Shi, F., Yan, L., Zhao, X. and Xian-Ke Gao, R. (2022) Machine Learning-Based Time-Series Data Analysis in Edge-Cloud-Assisted Oil Industrial IoT System. MobileInformationSystems, 2022, Article ID: 5988164. https://doi.org/10.1155/2022/5988164
[3]
Al-Shabandar, R., Jaddoa, A., Liatsis, P. and Hussain, A.J. (2021) A Deep Gated Recurrent Neural Network for Petroleum Production Forecasting. MachineLearningwithApplications, 3, Article ID: 100013. https://doi.org/10.1016/j.mlwa.2020.100013
[4]
Zhang, F., Liu, M., Zhou, Z. and Shen, W. (2016) An Iot-Based Online Monitoring System for Continuous Steel Casting. IEEEInternetofThingsJournal, 3, 1355-1363. https://doi.org/10.1109/jiot.2016.2600630
[5]
Wanasinghe, T.R., Gosine, R.G., James, L.A., Mann, G.K.I., de Silva, O. and Warrian, P.J. (2020) The Internet of Things in the Oil and Gas Industry: A Systematic Review. IEEEInternetofThingsJournal, 7, 8654-8673. https://doi.org/10.1109/jiot.2020.2995617
[6]
Aalsalem, M.Y., Khan, W.Z., Gharibi, W. and Armi, N. (2017) An Intelligent Oil and Gas Well Monitoring System Based on Internet of Things. 2017 International Conference on Radar, Antenna, Microwave, Electronics, and Telecommunications (ICRAMET), Jakarta, 23-24 October 2017, 124-127. https://doi.org/10.1109/icramet.2017.8253159
[7]
Gao, M.J., Xu, J., Tian, J.W. and Zhang, F. (2008) Zigbee Wireless Mesh Networks for Remote Monitoring System of Pumping Unit. 2008 7th World Congress on Intelligent Control and Automation, Chongqing, 25-27 June 2008, 5901-5905. https://doi.org/10.1109/wcica.2008.4592834
[8]
Trappey, A.J.C., Trappey, C.V., Hareesh Govindarajan, U., Chuang, A.C. and Sun, J.J. (2017) A Review of Essential Standards and Patent Landscapes for the Internet of Things: A Key Enabler for Industry 4.0. AdvancedEngineeringInformatics, 33, 208-229. https://doi.org/10.1016/j.aei.2016.11.007
[9]
Gubbi, J., Buyya, R., Marusic, S. and Palaniswami, M. (2013) Internet of Things (IoT): A Vision, Architectural Elements, and Future Directions. FutureGenerationComputerSystems, 29, 1645-1660. https://doi.org/10.1016/j.future.2013.01.010
[10]
Borgia, E. (2014) The Internet of Things Vision: Key Features, Applications and Open Issues. ComputerCommunications, 54, 1-31. https://doi.org/10.1016/j.comcom.2014.09.008
[11]
Wu, Y., Liu, Y., Ahmed, S.H., Peng, J. and Abd El-Latif, A.A. (2020) Dominant Data Set Selection Algorithms for Electricity Consumption Time-Series Data Analysis Based on Affine Transformation. IEEEInternetofThingsJournal, 7, 4347-4360. https://doi.org/10.1109/jiot.2019.2946753
[12]
Heidari Kapourchali, M. and Banerjee, B. (2018) Unsupervised Feature Learning from Time-Series Data Using Linear Models. IEEEInternetofThingsJournal, 5, 3918-3926. https://doi.org/10.1109/jiot.2018.2845340
[13]
Akbar, A., Khan, A., Carrez, F. and Moessner, K. (2017) Predictive Analytics for Complex IoT Data Streams. IEEEInternetofThingsJournal, 4, 1571-1582. https://doi.org/10.1109/jiot.2017.2712672
[14]
Sagheer, A. and Kotb, M. (2019) Time Series Forecasting of Petroleum Production Using Deep LSTM Recurrent Networks. Neurocomputing, 323, 203-213. https://doi.org/10.1016/j.neucom.2018.09.082
[15]
Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., et al. (2021) Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting. ProceedingsoftheAAAIConferenceonArtificialIntelligence, 35, 11106-11115. https://doi.org/10.1609/aaai.v35i12.17325
[16]
Wu, H., Xiao, B., Codella, N., Liu, M., Dai, X., Yuan, L., et al. (2021) CvT: Introducing Convolutions to Vision Transformers. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, 10-17 October 2021, 22-31. https://doi.org/10.1109/iccv48922.2021.00009
[17]
Lepikhin, D., Lee, H.J., Xu, Y., et al. (2020) GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding. arXiv: 2006.16668.
[18]
Barret, Z., Bello, Z., Kumar, S., et al. (2022) ST-MoE: Designing Stable and Transferable Sparse Expert Models. arXiv: 2202.08906.
[19]
Stassin, S., Corduant, V., Mahmoudi, S.A. and Siebert, X. (2024) Explainability and Evaluation of Vision Transformers: An In-Depth Experimental Study. Electronics, 13, Article 175. https://doi.org/10.3390/electronics13010175
[20]
Yan, F., Li, S., Zhou, Z. and Shi, Y. (2024) A Residual Network with Efficient Transformer for Lightweight Image Super-Resolution. Electronics, 13, Article 94. https://doi.org/10.3390/electronics13010194
[21]
Minoli, D., Sohraby, K. and Occhiogrosso, B. (2017) IoT Considerations, Requirements, and Architectures for Smart Buildings—Energy Optimization and Next-Generation Building Management Systems. IEEEInternetofThingsJournal, 4, 269-283. https://doi.org/10.1109/jiot.2017.2647881
[22]
Verma, S., Kawamoto, Y., Fadlullah, Z.M., Nishiyama, H. and Kato, N. (2017) A Survey on Network Methodologies for Real-Time Analytics of Massive IoT Data and Open Research Issues. IEEECommunicationsSurveys&Tutorials, 19, 1457-1477. https://doi.org/10.1109/comst.2017.2694469
[23]
Seren, H.R., Buzi, E., Al-Maghrabi, L., Ham, G., Bernero, G. and Deffenbaugh, M. (2018) An Untethered Sensor Platform for Logging Vertical Wells. IEEETransactionsonInstrumentationandMeasurement, 67, 798-803. https://doi.org/10.1109/tim.2017.2774183
[24]
Akbar, A., Kousiouris, G., Pervaiz, H., Sancho, J., Ta-Shma, P., Carrez, F., et al. (2018) Real-time Probabilistic Data Fusion for Large-Scale IoT Applications. IEEEAccess, 6, 10015-10027. https://doi.org/10.1109/access.2018.2804623
[25]
Wang, Y., Shen, Y., Mao, S., Chen, X. and Zou, H. (2019) LASSO and LSTM Integrated Temporal Model for Short-Term Solar Intensity Forecasting. IEEEInternetofThingsJournal, 6, 2933-2944. https://doi.org/10.1109/jiot.2018.2877510
[26]
Zhang, Y., Thorburn, P.J., Xiang, W. and Fitch, P. (2019) SSIM—A Deep Learning Approach for Recovering Missing Time Series Sensor Data. IEEEInternetofThingsJournal, 6, 6618-6628. https://doi.org/10.1109/jiot.2019.2909038
[27]
Wang, X., Pi, D., Zhang, X., Liu, H. and Guo, C. (2022) Variational Transformer-Based Anomaly Detection Approach for Multivariate Time Series. Measurement, 191, Article ID: 110791. https://doi.org/10.1016/j.measurement.2022.110791
[28]
Morid, M.A., Sheng, O.R.L. and Dunbar, J. (2023) Time Series Prediction Using Deep Learning Methods in Healthcare. ACMTransactionsonManagementInformationSystems, 14, 1-29. https://doi.org/10.1145/3531326
[29]
Basha, E., Jurdak, R. and Rus, D. (2014) In-network Distributed Solar Current Prediction. ACMTransactionsonSensorNetworks, 11, 1-28. https://doi.org/10.1145/2629593
[30]
Piccialli, F., Giampaolo, F., Prezioso, E., Crisci, D. and Cuomo, S. (2021) Predictive Analytics for Smart Parking: A Deep Learning Approach in Forecasting of IoT Data. ACMTransactionsonInternetTechnology, 21, 1-21. https://doi.org/10.1145/3412842
[31]
Chang, X., Li, G., Xing, G., Zhu, K. and Tu, L. (2021) DeepHeart: A Deep Learning Approach for Accurate Heart Rate Estimation from PPG Signals. ACMTransactionsonSensorNetworks, 17, 1-18. https://doi.org/10.1145/3441626
[32]
Fang, H., Liu, Y., Chen, C. and Hwang, F. (2022) Travel Time Prediction Method Based on Spatial-Feature-Based Hierarchical Clustering and Deep Multi-Input Gated Recurrent Unit. ACMTransactionsonSensorNetworks, 19, 1-21. https://doi.org/10.1145/3544976
[33]
Vallés-Pérez, I., Soria-Olivas, E., Martínez-Sober, M., Serrano-López, A.J., Gómez-Sanchís, J. and Mateo, F. (2022) Approaching Sales Forecasting Using Recurrent Neural Networks and Transformers. ExpertSystemswithApplications, 201, Article ID: 116993. https://doi.org/10.1016/j.eswa.2022.116993
