Extension of discrete Shannon entropy to continuous or differential entropy is reexamined. It is shown that the differential entropy, which is applied in many fields, is not the direct limit of discrete entropy, which is also responsible for its shortcomings.
References
[1]
Shannon, C.E. (1948) A Mathematical Theory of Communication. BellSystemTechnicalJournal, 27, 623-656. https://doi.org/10.1002/j.1538-7305.1948.tb00917.x
[2]
Rioul, O. (2021) This Is IT: A Primer on Shannon’s Entropy and Information. In: Duplantier, B. and Rivasseau, V., Eds., Information Theory, Birkhäuser, 49-86.
[3]
Viznyuk, S. (2015) Shannon’s Entropy Revisited. arXiv: 1504.01407. https://arxiv.org/pdf/1504.01407v1
[4]
Mohazzabi, P., Jones, N.H., Tucker, R.H. and Winter, N.J. (2025) The Entropy of a DNA Strand. JournalofAppliedMathematicsandPhysics, 13, 4490-4497. https://doi.org/10.4236/jamp.2025.1312246
[5]
Mandl, F. (1991) Statistical Physics. 2nd Edition, Wiley, 61 p.
[6]
Ihara, S. (1993) Information Theory for Continuous Systems. World Scientific Publishing Co. Pte. Ltd. https://doi.org/10.1142/9789814355827
[7]
Marsh, C. (2013) Introduction to Continuous Entropy. https://www.crmarsh.com/pdf/Charles_Marsh_Continuous_Entropy.pdf
[8]
Cover, T.M. and Thomas, J.A. (1991) Elements of Information Theory. Wiley, 224 p.
[9]
Dubois, D.M. (2019) Generalization of the Discrete and Continuous Shannon Entropy by Positive Definite Functions Related to the Constant of Motion of the Non-Linear Lotka-Volterra System. JournalofPhysics: ConferenceSeries, 1251, Article ID: 012012. https://doi.org/10.1088/1742-6596/1251/1/012012
