|
|
动态正则格兰杰因果学习方法
|
Abstract:
在医学和金融学等实际领域中，了解动态系统中的底层结构关系对于调节系统中的变量和预测系统未来状态至关重要。系统的动态变化会生成时间序列数据，通过观察时间序列数据可以分析系统的底层结构。格兰杰因果关系分析方法可以应用于一维或多维时间序列系统，现有的方法以组件式的建模方式分析每个系统变量特定的因果关系，受限于时间方向的强假设性和组件模型的单一性，其无法准确地挖掘出时间序列中的因果关系结构。本文提出了一种基于动态稀疏正则化的格兰杰因果发现方法DRGC (Dynamic Regularity Granger Causlity)。DRGC模型从卷积网络的输入权重中周期性地发掘变量在时间维度上的依赖信息，并以此为据向网络施加稀疏惩罚，以获得精确的格兰杰因果关系；同时，使用采样输入的循环网络提取数据中的长程依赖关系，同步优化卷积网络的权重，增强了模型发现因果关系的精确性和稳定性。在模拟数据集和真实系统生成的数据集上的实验表明，DRGC优于最先进的基线方法。
In practical fields such as medicine and finance, understanding the underlying structural relationships in dynamic systems is crucial for regulating system variables and predicting the system’s future state. The dynamic changes of a system generate time series data, and by observing these time series, the underlying structure of the system can be analyzed. Granger causality analysis methods can be applied to univariate or multivariate time series systems. Existing methods analyze the specific causal relationships of each system variable using a modular modeling approach. However, these methods are limited by strong assumptions regarding the time direction and the simplicity of the modular models, which prevents them from accurately uncovering the causal relationship structure in multivariate systems. This paper proposes a Granger causality discovery method based on dynamic sparse regularization, called DRGC (Dynamic Regularized Granger Causality). The DRGC model periodically uncovers the temporal dependencies of variables from the input weights of a convolutional network and applies sparse penalties to the network accordingly, to obtain precise Granger causal relationships. Additionally, a cyclic network is used to extract long-range dependencies from the sampled input data, and the convolutional network’s weights are optimized simultaneously, which enhances the precision and stability of the model in discovering causal relationships. Experiments conducted on simulated datasets and real-world system-generated datasets show that DRGC outperforms state-of-the-art baseline methods.
| [1] | Vicente, R., Wibral, M., Lindner, M. and Pipa, G. (2010) Transfer Entropy—A Model-Free Measure of Effective Connectivity for the Neurosciences. Journal of Computational Neuroscience, 30, 45-67. https://doi.org/10.1007/s10827-010-0262-3 |
| [2] | Runge, J. (2018) Causal Network Reconstruction from Time Series: From Theoretical Assumptions to Practical Estimation. Chaos: An Interdisciplinary Journal of Nonlinear Science, 28, Article ID: 075310. https://doi.org/10.1063/1.5025050 |
| [3] | Gerhardus, A. and Runge, J. (2020) High-Recall Causal Discovery for Autocorrelated Time Series with Latent Confounders. Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, 6-12 December 2020, 12615-12625. |
| [4] | Tank, A., Covert, I., Foti, N., Shojaie, A. and Fox, E.B. (2021) Neural Granger Causality. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44, 4267-4279. https://doi.org/10.1109/tpami.2021.3065601 |
| [5] | Khanna, S. and Tan, V.Y.F. (2019) Economy Statistical Recurrent Units for Inferring Nonlinear Granger Causality. |
| [6] | Pamfil, R., Sriwattanaworachai, N., Desai, S., et al. (2020) Dynotears: Structure Learning from Time-Series Data. International Conference on Artificial Intelligence and Statistics, 26-28 August 2020, 1595-1605. |
| [7] | Granger, C.W.J. (1969) Investigating Causal Relations by Econometric Models and Cross-Spectral Methods. Econometrica, 37, 424-438. https://doi.org/10.2307/1912791 |
| [8] | Marinazzo, D., Pellicoro, M. and Stramaglia, S. (2008) Kernel-Granger Causality and the Analysis of Dynamical Networks. Physical Review E, 77, Article ID: 056215. https://doi.org/10.1103/physreve.77.056215 |
| [9] | Lütkepohl, H. (2005) New Introduction to Multiple Time Series Analysis. Springers Science & Business Media. |
| [10] | Lusch, B., Maia, P.D. and Kutz, J.N. (2016) Inferring Connectivity in Networked Dynamical Systems: Challenges Using Granger Causality. Physical Review E, 94, Article ID: 032220. https://doi.org/10.1103/physreve.94.032220 |
| [11] | Amblard, P. and Michel, O.J.J. (2010) On Directed Information Theory and Granger Causality Graphs. Journal of Computational Neuroscience, 30, 7-16. https://doi.org/10.1007/s10827-010-0231-x |
| [12] | Yu, R., Zheng, S., Anandkumar, A., et al. (2018) Long-Term Forecasting Using Tensor-Train RNNs. |
| [13] | Li, Y., Yu, R., Shahabi, C., et al. (2017) Graph Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting. |
| [14] | Lozano, A.C., Abe, N., Liu, Y. and Rosset, S. (2009) Grouped Graphical Granger Modeling Methods for Temporal Causal Modeling. Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Paris, 28 June-1 July 2009, 577-586. https://doi.org/10.1145/1557019.1557085 |
| [15] | Tibshirani, R. (1996) Regression Shrinkage and Selection via the Lasso. Journal of the Royal Statistical Society Series B: Statistical Methodology, 58, 267-288. https://doi.org/10.1111/j.2517-6161.1996.tb02080.x |
| [16] | Lozano, A.C., Abe, N., Liu, Y. and Rosset, S. (2009) Grouped Graphical Granger Modeling for Gene Expression Regulatory Networks Discovery. Bioinformatics, 25, i110-i118. https://doi.org/10.1093/bioinformatics/btp199 |
| [17] | Runge, J., Heitzig, J., Petoukhov, V. and Kurths, J. (2012) Escaping the Curse of Dimensionality in Estimating Multivariate Transfer Entropy. Physical Review Letters, 108, Article ID: 258701. https://doi.org/10.1103/physrevlett.108.258701 |
| [18] | Wu, T., Breuel, T., Skuhersky, M., et al. (2020) Discovering Nonlinear Relations with Minimum Predictive Information Regularization. |
| [19] | Xu, C., Huang, H. and Yoo, S. (2019) Scalable Causal Graph Learning through a Deep Neural Network. Proceedings of the 28th ACM International Conference on Information and Knowledge Management, Beijing, 3-7 November 2019, 1853-1862. https://doi.org/10.1145/3357384.3357864 |
| [20] | Singh, R., Wu, A.P. and Berger, B. (2022) Granger Causal Inference on DAGs Identifies Genomic Loci Regulating Transcription. |
| [21] | Absar, S., Wu, Y. and Zhang, L. (2023) Neural Time-Invariant Causal Discovery from Time Series Data. 2023 International Joint Conference on Neural Networks (IJCNN), Gold Coast, 18-23 June 2023, 1-8. https://doi.org/10.1109/ijcnn54540.2023.10192004 |
| [22] | Cheng, Y., Yang, R., Xiao, T., et al. (2023) Cuts: Neural Causal Discovery from Irregular Time-Series Data. |
| [23] | Cheng, Y., Li, L., Xiao, T., Li, Z., Suo, J., He, K., et al. (2024) CUTS+: High-Dimensional Causal Discovery from Irregular Time-Series. Proceedings of the AAAI Conference on Artificial Intelligence, 38, 11525-11533. https://doi.org/10.1609/aaai.v38i10.29034 |
| [24] | Sultan, M.S., Horvath, S. and Ombao, H. (2022) Granger Causality Using Neural Networks. |
| [25] | Jang, E., Gu, S. and Poole, B. (2016) Categorical Reparameterization with Gumbel-Softmax. |
| [26] | Zhang, J. and Ghanem, B. (2018) ISTA-Net: Interpretable Optimization-Inspired Deep Network for Image Compressive Sensing. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, 18-23 June 2018, 1828-1837. https://doi.org/10.1109/cvpr.2018.00196 |
| [27] | Lorenz, E.N. (1996) Predictability: A Problem Partly Solved. Proceedings on Seminar on Predictability, 1, 1-18. |
| [28] | Prill, R.J., Marbach, D., Saez-Rodriguez, J., Sorger, P.K., Alexopoulos, L.G., Xue, X., et al. (2010) Towards a Rigorous Assessment of Systems Biology Models: The DREAM3 Challenges. PLOS ONE, 5, e9202. https://doi.org/10.1371/journal.pone.0009202 |
| [29] | Runge, J., Nowack, P., Kretschmer, M., Flaxman, S. and Sejdinovic, D. (2019) Detecting and Quantifying Causal Associations in Large Nonlinear Time Series Datasets. Science Advances, 5, eaau4996. https://doi.org/10.1126/sciadv.aau4996 |
