People learn causal relations since childhood using counterfactual reasoning. Counterfactual reasoning uses counterfactual examples which take the form of “what if this has happened differently”. Counterfactual examples are also the basis of counterfactual explanation in explainable artificial intelligence (XAI). However, a framework that relies solely on optimization algorithms to find and present counterfactual samples cannot help users gain a deeper understanding of the system. Without a way to verify their understanding, the users can even be misled by such explanations. Such limitations can be overcome through an interactive and iterative framework that allows the users to explore their desired “what-if” scenarios. The purpose of our research is to develop such a framework. In this paper, we present our “what-if” XAI framework (WiXAI), which visualizes the artificial intelligence (AI) classification model from the perspective of the user’s sample and guides their “what-if” exploration. We also formulated how to use the WiXAI framework to generate counterfactuals and understand the feature-feature and feature-output relations in-depth for a local sample. These relations help move the users toward causal understanding.
References
[1]
Georgiev, P., Bhattacharya, S., Lane, N.D. and Mascolo, C. (2017) Low-Resource Multi-Task Audio Sensing for Mobile and Embedded Devices via Shared Deep Neural Network Representations. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 1, Article No. 50. https://doi.org/10.1145/3131895
[2]
Noreen, U., Shafique, A., Ahmed, Z. and Ashfaq, M. (2023) Banking 4.0: Artificial Intelligence (AI) in Banking Industry & Consumer’s Perspective. Sustainability, 15, Article 3682. https://doi.org/10.3390/su15043682
[3]
Sarker, I.H., Furhad, M.H. and Nowrozy, R. (2021) AI-Driven Cybersecurity: An Overview, Security Intelligence Modeling and Research Directions. SN Computer Science, 2, Article No. 173. https://doi.org/10.1007/s42979-021-00557-0
[4]
Pisoni, G. and Díaz-Rodríguez, N. (2023) Responsible and Human Centric AI-Based Insurance Advisors. Information Processing & Management, 60, Article 103273. https://doi.org/10.1016/j.ipm.2023.103273
[5]
Goswami, G., Bhardwaj, R., Singh, R. and Vatsa, M. (2014). MDLface: Memorability Augmented Deep Learning for Video Face Recognition. IEEE International Joint Conference on Biometrics, Clearwater, 29 September-2 October 2014, 1-7. https://doi.org/10.1109/btas.2014.6996299
[6]
Sarker, I.H. (2021) Machine Learning: Algorithms, Real-World Applications and Research Directions. SN Computer Science, 2, Article No. 160. https://doi.org/10.1007/s42979-021-00592-x
[7]
Samek, W., Montavon, G., Lapuschkin, S., Anders, C.J. and Muller, K. (2021) Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications. Proceedings of the IEEE, 109, 247-278. https://doi.org/10.1109/jproc.2021.3060483
[8]
Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J.M., Confalonieri, R., et al. (2023) Explainable Artificial Intelligence (XAI): What We Know and What Is Left to Attain Trustworthy Artificial Intelligence. Information Fusion, 99, Article 101805. https://doi.org/10.1016/j.inffus.2023.101805
Cheng, F., Ming, Y. and Qu, H. (2021) DECE: Decision Explorer with Counterfactual Explanations for Machine Learning Models. IEEE Transactions on Visualization and Computer Graphics, 27, 1438-1447. https://doi.org/10.1109/tvcg.2020.3030342
[11]
Pearl, J. (2019) The Seven Tools of Causal Inference, with Reflections on Machine Learning. Communications of the ACM, 62, 54-60. https://doi.org/10.1145/3241036
[12]
Plumb, G., Molitor, D. and Talwalkar, A.S. (2018) Model Agnostic Supervised Local Explanations. Proceedings of the 32nd International Conference on Neural Information Processing Systems, Montréal, 3-8 December 2018, 2520-2529.
[13]
Dhurandhar, A., Chen, P.-Y., Luss, R., Tu, C.-C., Ting, P., Shanmugam, K.-K. and Das, P. (2018) Explanations Based on the Missing: Towards Contrastive Explanations with Pertinent Negatives. Proceedings of the 32nd International Conference on Neural Information Processing Systems, Montréal, 3-8 December 2018, 590-601.
[14]
Mothilal, R.K., Sharma, A. and Tan, C. (2020) Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations. Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, Barcelona, 27-30 January 2020, 607-617. https://doi.org/10.1145/3351095.3372850
[15]
Poyiadzi, R., Sokol, K., Santos-Rodriguez, R., De Bie, T. and Flach, P. (2020) FACE: Feasible and Actionable Counterfactual Explanations. Proceedings of the AAAI/ ACM Conference on AI, Ethics, and Society, New York, 7-9 February 2020, 344-350. https://doi.org/10.1145/3375627.3375850
[16]
Albini, E., Rago, A., Baroni, P. and Toni, F. (2020) Relation-Based Counterfactual Explanations for Bayesian Network Classifiers. Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, 451-457. https://doi.org/10.24963/ijcai.2020/63
[17]
Chou, Y.-L., Moreira, C., Bruza, P., Ouyang, C. and Jorge, J.A. (2021) Counterfactuals and Causability in Explainable Artificial Intelligence: Theory, Algorithms, and Applications. Information Fusion, 81, 59-83. https://doi.org/10.1016/j.inffus.2021.11.003
[18]
Wachter, S., Mittelstadt, B. and Russell, C. (2017) Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR. Harvard Journal of Law & Technology, 31, 842-887.
[19]
Verma, S., Boonsanong, V., Hoang, M., Hines, K.E., Dickerson, J.P. and Shah, C. (2020) Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Review. arXiv: 2010.10596. https://doi.org/10.48550/arXiv.2010.10596
[20]
White, A. and d’Avila Garcez, A. (2019) Measurable Counterfactual Local Explanations for Any Classifier. arXiv: 1908.03020. https://doi.org/10.48550/arXiv.1908.03020
[21]
Galhotra, S., Pradhan, R. and Salimi, B. (2021) Explaining Black-Box Algorithms Using Probabilistic Contrastive Counterfactuals. Proceedings of the 2021 International Conference on Management of Data, Virtual Event China, 20-25 June 2021, 577-590. https://doi.org/10.1145/3448016.3458455
[22]
Goyal, Y., Wu, Z., Ernst, J., Batra, D., Parikh, D. and Lee, S. (2019) Counterfactual Visual Explanations. Proceedings of the 36th International Conference on Machine Learning, May 2019, 2376-2384.
[23]
Akula, A.R., Wang, K., Liu, C., Saba-Sadiya, S., Lu, H., Todorovic, S., et al. (2022) CX-toM: Counterfactual Explanations with Theory-of-Mind for Enhancing Human Trust in Image Recognition Models. iScience, 25, Article 103581. https://doi.org/10.1016/j.isci.2021.103581
[24]
Mertes, S., Huber, T., Weitz, K., Heimerl, A. and André, E. (2022) GANterfactual—Counterfactual Explanations for Medical Non-Experts Using Generative Adversarial Learning. Frontiers in Artificial Intelligence, 5, Article 825565. https://doi.org/10.3389/frai.2022.825565
[25]
Kenny, E.M. and Keane, M.T. (2021) On Generating Plausible Counterfactual and Semi-Factual Explanations for Deep Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 35, 11575-11585. https://doi.org/10.1609/aaai.v35i13.17377
[26]
Thiagarajan, J.J., Thopalli, K., Rajan, D. and Turaga, P. (2022) Training Calibration-Based Counterfactual Explainers for Deep Learning Models in Medical Image Analysis. Scientific Reports, 12, Article No. 597. https://doi.org/10.1038/s41598-021-04529-5
[27]
Duong, T.D., Li, Q. and Xu, G. (2023) CeFlow: A Robust and Efficient Counterfactual Explanation Framework for Tabular Data Using Normalizing Flows. Advances in Knowledge Discovery and Data Mining, Osaka, 25-28 May 2023, 133-144. https://doi.org/10.1007/978-3-031-33377-4_11
[28]
Krause, J., Perer, A. and Ng, K. (2016) Interacting with Predictions. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, 7-12 May 2016, 5686-5697. https://doi.org/10.1145/2858036.2858529
[29]
Lash, M.T., Lin, Q., Street, N., Robinson, J.G. and Ohlmann, J. (2017) Generalized Inverse Classification. Proceedings of the 2017 SIAM International Conference on Data Mining, 162-170. https://doi.org/10.1137/1.9781611974973.19
[30]
Laugel, T., Lesot, M.-J., Marsala, C., Renard, X. and Detyniecki, M. (2018) Comparison-Based Inverse Classification for Interpretability in Machine Learning. Information Processing and Management of Uncertainty in Knowledge-Based Systems. Theory and Foundations, Cádiz, 11-15 June 2018, 100-111. https://doi.org/10.1007/978-3-319-91473-2_9
[31]
Kanamori, K., Takagi, T., Kobayashi, K. and Arimura, H. (2021) Distribution-Aware Counterfactual Explanation by Mixed-Integer Linear Optimization. Transactions of the Japanese Society for Artificial Intelligence, 36, 1-12. https://doi.org/10.1527/tjsai.36-6_c-l44
[32]
Guidotti, R., Monreale, A., Ruggieri, S., Pedreschi, D., Turini, F. and Giannotti, F. (2018) Local Rule-Based Explanations of Black Box Decision Systems. arXiv: 1805.10820. https://doi.org/10.48550/arXiv.1805.10820
[33]
Russell, C. (2019) Efficient Search for Diverse Coherent Explanations. Proceedings of the Conference on Fairness, Accountability, and Transparency, Atlanta, 29-31 January 2019, 20-28. https://doi.org/10.1145/3287560.3287569
[34]
Baron, S. (2023) Explainable AI and Causal Understanding: Counterfactual Approaches Considered. Minds and Machines, 33, 347-377. https://doi.org/10.1007/s11023-023-09637-x
[35]
Wexler, J., Pushkarna, M., Bolukbasi, T., Wattenberg, M., Viegas, F. and Wilson, J. (2019) The What-If Tool: Interactive Probing of Machine Learning Models. IEEE Transactions on Visualization and Computer Graphics, 26, 56-65. https://doi.org/10.1109/tvcg.2019.2934619
[36]
Tenney, I., Wexler, J., Bastings, J., Bolukbasi, T., Coenen, A., Gehrmann, S., et al. (2020). The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Models. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 107-118. https://doi.org/10.18653/v1/2020.emnlp-demos.15
[37]
Gathani, S., Hulsebos, M., Gale, J., Haas, P.J. and Demiralp, Ç. (2022) Augmenting Decision Making via Interactive What-If Analysis. arXiv: 2109.06160. https://doi.org/10.48550/arXiv.2109.06160
[38]
Guidotti, R. (2022) Counterfactual Explanations and How to Find Them: Literature Review and Benchmarking. Data Mining and Knowledge Discovery. https://doi.org/10.1007/s10618-022-00831-6
[39]
Byrne, R.M.J. (2019) Counterfactuals in Explainable Artificial Intelligence (XAI): Evidence from Human Reasoning. Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, 6276-6282. https://doi.org/10.24963/ijcai.2019/876
[40]
Mahajan, D., Tan, C. and Sharma, A. (2019) Preserving Causal Constraints in Counterfactual Explanations for Machine Learning Classifiers. ar-Xiv: 1912.03277. https://doi.org/10.48550/arXiv.1912.03277
[41]
Crupi, R., San Miguel González, B., Castelnovo, A. and Regoli, D. (2022) Leveraging Causal Relations to Provide Counterfactual Explanations and Feasible Recommendations to End Users. Proceedings of the 14th International Conference on Agents and Artificial Intelligence, Location, Date, 24-32. https://doi.org/10.5220/0010761500003116
[42]
Paul, S.K., Firdausi, T.J., Jana, S., Das, A. and Nandi, P. (2021) Counterfactual Causal Analysis on Structured Data. In: Reddy, V.S., Prasad, V.K., Wang, J. and Reddy, K., Eds., International Conference on Soft Computing and Signal Processing, Springer, 187-195. https://doi.org/10.1007/978-981-16-7088-6_16
[43]
Pearl, J. (2009) Causality. 2nd Edition, Cambridge University Press. https://doi.org/10.1017/cbo9780511803161
Wang, P.Y., Galhotra, S., Pradhan, R. and Salimi, B. (2021) Demonstration of Generating Explanations for Black-Box Algorithms Using Lewis. Proceedings of the VLDB Endowment, 14, 2787-2790. https://doi.org/10.14778/3476311.3476345
[46]
Ming, Y., Qu, H. and Bertini, E. (2019) RuleMatrix: Visualizing and Understanding Classifiers with Rules. IEEE Transactions on Visualization and Computer Graphics, 25, 342-352. https://doi.org/10.1109/tvcg.2018.2864812
