All Title Author
Keywords Abstract

Publish in OALib Journal
ISSN: 2333-9721
APC: Only $99


Relative Articles


Using Cross Entropy as a Performance Metric for Quantifying Uncertainty in DNN Image Classifiers: An Application to Classification of Lung Cancer on CT Images

DOI: 10.4236/jbise.2024.171001, PP. 1-12

Keywords: Cross Entropy, Performance Metrics, DNN Image Classifiers, Lung Cancer, Prediction Uncertainty

Full-Text   Cite this paper   Add to My Lib


Cross entropy is a measure in machine learning and deep learning that assesses the difference between predicted and actual probability distributions. In this study, we propose cross entropy as a performance evaluation metric for image classifier models and apply it to the CT image classification of lung cancer. A convolutional neural network is employed as the deep neural network (DNN) image classifier, with the residual network (ResNet) 50 chosen as the DNN archi-tecture. The image data used comprise a lung CT image set. Two classification models are built from datasets with varying amounts of data, and lung cancer is categorized into four classes using 10-fold cross-validation. Furthermore, we employ t-distributed stochastic neighbor embedding to visually explain the data distribution after classification. Experimental results demonstrate that cross en-tropy is a highly useful metric for evaluating the reliability of image classifier models. It is noted that for a more comprehensive evaluation of model perfor-mance, combining with other evaluation metrics is considered essential.


[1]  Siegel, R.L., Miller, K.D., Wagle, N.S. and Jemal, A. (2023) Cancer Statistics, 2023. CA: A Cancer Journal for Clinicians, 73, 17-48.
[2]  Xu, R., Lu, T., Wang, C., Li, Q., Peng, B., Zhao, J., et al. (2023) Single-Cell Data Analysis of Malignant Epithelial Cell Heterogeneity in Lung Adenocarcinoma for Patient Classification and Prognosis Prediction. Heliyon, 9, e20164.
[3]  Niu, Z., Jin, R., Zhang, Y. and Li, H. (2022) Signaling Pathways and Targeted Therapies in Lung Squamous Cell Carcinoma: Mechanisms and Clinical Trials. Signal Transduction and Targeted Therapy, 7, 353.
[4]  Copin, M.-C. (2016) Carcinome à Grandes Cellules, Carcinome Lymphoepithelioma-Like, Carcinome NUT Large Cell Carcinoma, Lymphoepithelioma-Like Carcinoma, NUT Carcinoma. Annales de Pathologie, 36, 24-33.
[5]  The National Lung Screening Trial Research Team (2011) Reduced Lung-Cancer Mortality with Low-Dose Computed Tomographic Screening. The New England Journal of Medicine, 365, 395-409.
[6]  Aberle, D.R., DeMello, S., Berg, C.D., Black, W.C., Brewer, B., Church, T.R., et al. (2013) Results of the Two Incidence Screenings in the National Lung Screening Trial. The New England Journal of Medicine, 369, 920-931.
[7]  The National Lung Screening Trial Research Team (2013) Results of Initial Low-Dose Computed Tomographic Screening for Lung Cancer. The New England Journal of Medicine, 368, 1980-1991.
[8]  Kramer, B.S., Berg, C.D., Aberle, D.R. and Prorok, P.C. (2011) Lung Cancer Screening with Low-Dose Helical CT: Results from the National Lung Screening Trial (NLST). Journal of Medical Screening, 18, 109-111.
[9]  Midthun, D.E. (2011) Screening for Lung Cancer. Clinics in Chest Medicine, 32, 659-668.
[10]  Goo, J.M. (2011) A Computer-Aided Diagnosis for Evaluating Lung Nodules on Chest CT: The Current Status and Perspective. Korean Journal of Radiology, 12, 145-155.
[11]  Suzuki, K. (2012) A Review of Computer-Aided Diagnosis in Thoracic and Colonic Imaging. Quantitative Imaging in Medicine and Surgery, 2, 163-176.
[12]  El-Baz, A., Beache, G.M., Gimel’farb, G., Suzuki, K., Okada, K., et al. (2013) Computer-Aided Diagnosis Systems for Lung Cancer: Challenges and Methodologies. International Journal of Biomedical Imaging, 2013, Article ID: 942353.
[13]  Retico, A. (2013) Computer-Aided Detection for Pulmonary Nodule Identification: Improving the Radiologist’s Performance? Imaging in Medicine, 5, 249-263.
[14]  Firmino, M., Morais, A.H., Mendoca, R.M., Dantas, M.R., Hekis, H.R. and Valentim, R. (2014) Computer-Aided Detection System for Lung Cancer in Computed Tomography Scans: Review and Future Prospective. Biomedical Engineering Online, 13, 1-16.
[15]  Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., et al. (2017) Attention Is All You Need.
[16]  Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., et al. (2020) An Image Is Worth 16x16 Words: Transformers for Image Recognition at Scale.
[17]  Anthimopoulos, M., Christodoulidis, S., Ebner, L., Christe, A. and Mougiakakou, S. (2016) Lung Pattern Classification for Interstitial Lung Diseases Using a Deep Convolutional Neural Network. IEEE Transactions on Medical Imaging, 35, 1207-1216.
[18]  Gao, M., Bagci, U., Lu, L., Wu, A., Buty, M., Shin, H.C., et al. (2018) Holistic Classification of CT Attenuation Patterns for Interstitial Lung Diseases via Deep Convolutional Neural Networks. Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, 6, 1-6.
[19]  Matsuyama, E., Lee, Y., Takahashi, N. and Tsai, D.Y. (2019)A Wavelet Coefficient-Based Convolutional Neural Network for Histological Classification of Lung Cancer in CT Images. Japanese Journal of Imaging and Information Sciences in Medicine (In Japanese), 36, 64-71.
[20]  Zech, J.R., Badgeley, M.A., Liu, M., Costa, A.B., Titano, J. and Oermann, E.K. (2018) Variable Generalization Performance of a Deep Learning Model to Detect Pneumonia in Chest Radiographs: A Cross-Sectional Study. PLOS Medicine, 15, e1002683.
[21]  Ovadia, Y., Fertig, E., Ren, J., Nado. Z., Sculley, D., Nowozin, S., et al. (2019) Can You Trust Your Model’s Uncertainty? Evaluating Predictive Uncertainty under Dataset Shift. 33rd International Conference on Neural Information Processing Systems, Vancouver, 8-14 December 2019, 13969-13980.
[22]  Guo, C., Pleiss, G., Sun Y. and Weinberger, K.Q. (2017)On Calibration of Modern Neural Networks. Proceedings of the 34th International Conference on Machine Learning, Sydney, Vol. 70, 1321-1330.
[23]  Gal, Y. and Ghahramani, Z. (2016) Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. Proceedings of the 33rd International Conference on Machine Learning, Vol. 48, 1050-1059.
[24]  Chest CT-Scan Images Dataset.
[25]  He, K., Zhang, X., Ren, S. and Sun, J. (2015) Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 27-30 June 2016, 770-778.
[26]  Narayanan, B.N., De Silva, M.S., Hardie, R.C., Kueterman, N.K. and Ali, R. (2019) Understanding Deep Neural Network Predictions for Medical Imaging Applications.
[27]  van der Maaten, L.J.P. and Hinton, G.E. (2008) Visualizing High-Dimensional Data Using t-SNE. Journal of Machine Learning Research, 9, 2579-2605.
[28]  Shan, B. and Fang, Y. (2020) A Cross Entropy Based Deep Neural Network Model for Road Extraction from Satellite Images. Entropy, 22, Article No. 535.
[29]  Kurian, N.C., Meshram, P.S., Patil, A., Patel S. and Sethi, A. (2021) Sample Specific Generalized Cross Entropy for Robust Histology Image Classification. 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), Nice, 13-16 April 2021, 1934-1938.
[30]  Mannor, S., Peleg, D. and Rubinstein, R. (2005) The Cross Entropy Method for Classification. Proceedings of the 22nd International Conference on Machine Learning, Bonn, 7-11 August 2005, 561-568.
[31]  Brownlee, J. (2020) A Gentle Introduction to Cross-Entropy for Machine Learning.
[32]  Mao, A., Mohri, M. and Zhong, Y. (2023) Cross-Entropy Loss Functions: Theoretical Analysis and Applications. Proceedings of the 40th International Conference on Machine Learning, Honolulu, Vol. 202, 23803-23828.
[33]  Nova (2023) A Comprehensive Guide to Cross Entropy in Machine Learning.
[34]  Sheikh, I. (2023) Understanding Cross-Entropy Loss and Its Role in Classification Problems.


comments powered by Disqus

Contact Us


WhatsApp +8615387084133

WeChat 1538708413