An optimal entropy estimator for discrete random variables

This paper presents analytical formulations of the most important estimation errors - averaged squared bias error and mean squared error - for the class of entropy estimator expressed as a sum of single variable functions. The class of entropy estimator includes almost all important entropy estimators that have been proposed heretofore. Furthermore, this paper presents an optimal entropy estimator that can minimize mean squared error of the estimate under the condition that averaged squared bias error of the estimate is restricted to below an arbitrary value. A numerical experiment demonstrates that the proposed entropy estimator provides a lower mean squared error than conventional entropy estimators when entropy is estimated as an ensemble mean over plural entropy estimates obtained for different independent data. Such estimation is often utilized for biological signals, e.g., neural signals, because of biological tiredness and adaptation property.