Ordinal outcome neural networks represent an innovative and robust methodology for analyzing high-dimensional health data characterized by ordinal outcomes. This study offers a comparative analysis of several ordinal activation functions that utilize four key cumulative link functions used in ordinal regression: logit (logistic distribution), probit (normal distribution), complementary log log (cloglog), based on the Gompertz distribution, and log log (loglog), derived from the Gumbel distribution. The objective of this study was to systematically assess the performance of various activation functions in relation to the four link functions, focusing on how these functions affect the predictive accuracy of ordinal outcome neural networks. The findings indicate that the logit link consistently outperformed the other cumulative links across different datasets, achieving the highest percentage of correctly classified observations. The results of this study contribute to model selection considerations when analyzing complex health-related data, leading to more accurate predictions and an improved understanding of biomedical phenomena.
References
[1]
Douglas, K. (2014) How Data Is Changing Our World. Nurse Leader, 12, 37-67. https://doi.org/10.1016/j.mnl.2014.07.003
[2]
Francis, J.G. and Francis, L.P. (2021) Enhancing Surveillance: New Data, New Technologies, and New Actors. In: Francis, J.G. and Francis, L.P., Eds., Sustaining Surveillance: The Importance of Information for Public Health, Springer, 119-158. https://doi.org/10.1007/978-3-030-63928-0_5
[3]
Gambhir, S.S., Ge, T.J., Vermesh, O. and Spitler, R. (2018) Toward Achieving Precision Health. Science Translational Medicine, 10, eaao3612. https://doi.org/10.1126/scitranslmed.aao3612
[4]
Feldman, K., Johnson, R.A. and Chawla, N.V. (2018) The State of Data in Healthcare: Path Towards Standardization. Journal of Healthcare Informatics Research, 2, 248-271. https://doi.org/10.1007/s41666-018-0019-8
[5]
Hull, S. (2015) Patient-Generated Health Data Foundation for Personalized Collaborative Care. CIN: Computers, Informatics, Nursing, 33, 177-180. https://doi.org/10.1097/cin.0000000000000159
[6]
Cantor, M.N. (2009) Enabling Personalized Medicine through the Use of Healthcare Information Technology. Personalized Medicine, 6, 589-594. https://doi.org/10.2217/pme.09.35
[7]
Reza Soroushmehr, S.M. and Najarian, K. (2016) Transforming Big Data into Computational Models for Personalized Medicine and Health Care. Dialogues in Clinical Neuroscience, 18, 339-343. https://doi.org/10.31887/dcns.2016.18.3/ssoroushmehr
[8]
Fröhlich, H., Balling, R., Beerenwinkel, N., Kohlbacher, O., Kumar, S., Lengauer, T., et al. (2018) From Hype to Reality: Data Science Enabling Personalized Medicine. BMC Medicine, 16, Article No. 150. https://doi.org/10.1186/s12916-018-1122-7
[9]
Amarasingham, R., Patzer, R.E., Huesch, M., Nguyen, N.Q. and Xie, B. (2014) Implementing Electronic Health Care Predictive Analytics: Considerations and Challenges. Health Affairs, 33, 1148-1154. https://doi.org/10.1377/hlthaff.2014.0352
[10]
Feldman, K., Faust, L., Wu, X., Huang, C. and Chawla, N.V. (2017) Beyond Volume: The Impact of Complex Healthcare Data on the Machine Learning Pipeline. In: Holzinger, A., Goebel, R., Ferri, M. and Palade, V., Eds., Towards Integrative Machine Learning and Knowledge Extraction, Springer, 150-169. https://doi.org/10.1007/978-3-319-69775-8_9
[11]
Williams, A.A.A. (2019) Ordinal Outcome Modeling: The Application of the Adaptive Moment Estimation Optimizer to the Elastic Net Penalized Stereotype Logit. Journal of Data Analysis and Information Processing, 7, 14-27. https://doi.org/10.4236/jdaip.2019.71002
[12]
Agresti, A. (2010). Analysis of Ordinal Categorical Data. Wiley. https://doi.org/10.1002/9780470594001
[13]
Jakobsson, U. (2004) Statistical Presentation and Analysis of Ordinal Data in Nursing Research. Scandinavian Journal of Caring Sciences, 18, 437-440. https://doi.org/10.1111/j.1471-6712.2004.00305.x
[14]
Larasati, A., DeYong, C. and Slevitch, L. (2011) Comparing Neural Network and Ordinal Logistic Regression to Analyze Attitude Responses. Service Science, 3, 304-312. https://doi.org/10.1287/serv.3.4.304
[15]
Verhulst, B. and Neale, M.C. (2021) Best Practices for Binary and Ordinal Data Analyses. Behavior Genetics, 51, 204-214. https://doi.org/10.1007/s10519-020-10031-x
[16]
Nwankpa, C., Ijomah, W., Gachagan, A., et al. (2018) Activation Functions: Comparison of Trends in Practice and Research for Deep Learning. arXiv: 1811.03378
[17]
Wang, L. and Zhu, D. (2021) Tackling Ordinal Regression Problem for Heterogeneous Data: Sparse and Deep Multi-Task Learning Approaches. Data Mining and Knowledge Discovery, 35, 1134-1161. https://doi.org/10.1007/s10618-021-00746-8
[18]
Vargas, V.M., Gutiérrez, P.A. and Hervás, C. (2019) Deep Ordinal Classification Based on the Proportional Odds Model. In: Ferrández Vicente, J., Álvarez-Sánchez, J., de la Paz López, F., Toledo Moreo, J. and Adeli, H., Eds., From Bioinspired Systems and Biomedical Applications to Machine Learning, Springer, 441-451. https://doi.org/10.1007/978-3-030-19651-6_43
[19]
Kook, L., Herzog, L., Hothorn, T., et al (2020) Ordinal Neural Network Transformation Models: Deep and Interpretable Regression Models for Ordinal Outcomes. arXiv: 2010.08376 https://api.semanticscholar.org/CorpusID:223953349
[20]
Kook, L., Herzog, L., Hothorn, T., Dürr, O. and Sick, B. (2022) Deep and Interpretable Regression Models for Ordinal Outcomes. Pattern Recognition, 122, Article ID: 108263. https://doi.org/10.1016/j.patcog.2021.108263
[21]
Moayed, F.A. and Shell, R.L. (2010) Application of Artificial Neural Network Models in Occupational Safety and Health Utilizing Ordinal Variables. The Annals of Occupational Hygiene, 55, 132-142.
[22]
Vargas, V.M., Gutiérrez, P.A. and Hervás-Martínez, C. (2020) Cumulative Link Models for Deep Ordinal Classification. Neurocomputing, 401, 48-58. https://doi.org/10.1016/j.neucom.2020.03.034
[23]
Tutz, G. (2011) Regression for Categorical Data. Cambridge University Press. https://doi.org/10.1017/cbo9780511842061
[24]
Daykin, A.R. and Moffatt, P.G. (2002) Analyzing Ordered Responses: A Review of the Ordered Probit Model. Understanding Statistics, 1, 157-166. https://doi.org/10.1207/s15328031us0103_02
[25]
Wilson, D.L. (1994) The Analysis of Survival (Mortality) Data: Fitting Gompertz, Weibull, and Logistic Functions. Mechanisms of Ageing and Development, 74, 15-33. https://doi.org/10.1016/0047-6374(94)90095-7
[26]
González, R.G., Parra, M.I., Acero, F.J., et al. (2019) An Improved Method for the Estimation of the Gumbel Distribution Parameters. arXiv: 1902.07963
[27]
Archer, K.J., Hou, J., Zhou, Q., Ferber, K., Layne, J.G. and Gentry, A.E. (2014) Ordinalgmifs: An R Package for Ordinal Regression in High-Dimensional Data Settings. Cancer Informatics, 13, 187-195. https://doi.org/10.4137/cin.s20806
[28]
Ramachandran, P., Zoph, B. and Le, Q.V. (2017) Searching for Activation Functions. arXiv: 171005941
[29]
Kingma, D.P. and Ba, J. (2014) Adam: A Method for Stochastic Optimization. arXiv: 1412.6980
[30]
R Core Team (2023) R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing. https://www.R-project.org/
[31]
Gomes, M.A., Priolli, D.G., Tralhão, J.G. and Botelho, M.F. (2013) Carcinoma hepatocelular: Epidemiologia, biologia, diagnóstico e terapias. Revista da Associação Médica Brasileira, 59, 514-524. https://doi.org/10.1016/j.ramb.2013.03.005
[32]
Afonso Rabelo-Gonalves, E.M., Maria, B. and Robilotta Zeitune, J.M. (2014) Helicobacter pylori and Liver—Detection of Bacteria in Liver Tissue from Patients with Hepatocellular Carcinoma Using Laser Capture Microdissection Technique (LCM). In: Roesler, B.M., Ed., Trends in Helicobacter pylori Infection, InTech Open, 1. https://doi.org/10.5772/57080
[33]
Uzzau, A., Laura, M., Cherchi, V., Bertozzi, S., Avellini, C. and Soardo, G. (2013) Surgical Treatment Strategies and Prognosis of Hepatocellular Carcinoma. In: Kaseb, A.O., Ed., Hepatocellular Carcinoma—Future Outlook, InTech Open, 1. https://doi.org/10.5772/55890
[34]
Alberts, C.J., Clifford, G.M., Georges, D., Negro, F., Lesi, O.A., Hutin, Y.J., et al. (2022) Worldwide Prevalence of Hepatitis B Virus and Hepatitis C Virus among Patients with Cirrhosis at Country, Region, and Global Levels: A Systematic Review. The Lancet Gastroenterology & Hepatology, 7, 724-735. https://doi.org/10.1016/s2468-1253(22)00050-4
[35]
Rawla, P., Sunkara, T., Muralidharan, P. and Raj, J.P. (2018) Update in Global Trends and Aetiology of Hepatocellular Carcinoma. WspółczesnaOnkologia, 22, 141-150. https://doi.org/10.5114/wo.2018.78941
[36]
Singh, G. and Hilton Fick, G. (2020) Ordinal Outcomes: A Cumulative Probability Model with the Log Link and an Assumption of Proportionality. Statistics in Medicine, 39, 1343-1361. https://doi.org/10.1002/sim.8479
[37]
Smith, T.J., Walker, D.A. and McKenna, C.M. (2020) An Exploration of Link Functions Used in Ordinal Regression. Journal of Modern Applied Statistical Methods, 18, 2-15. https://doi.org/10.22237/jmasm/1556669640
