TY - GEN

T1 - An optimal entropy estimator for discrete random variables

AU - Shiga, Motoki

AU - Yokota, Yasunari

PY - 2005

Y1 - 2005

N2 - This paper presents analytical formulations of the most important estimation errors - averaged squared bias error and mean squared error - for the class of entropy estimator expressed as a sum of single variable functions. The class of entropy estimator includes almost all important entropy estimators that have been proposed heretofore. Furthermore, this paper presents an optimal entropy estimator that can minimize mean squared error of the estimate under the condition that averaged squared bias error of the estimate is restricted to below an arbitrary value. A numerical experiment demonstrates that the proposed entropy estimator provides a lower mean squared error than conventional entropy estimators when entropy is estimated as an ensemble mean over plural entropy estimates obtained for different independent data. Such estimation is often utilized for biological signals, e.g., neural signals, because of biological tiredness and adaptation property.

AB - This paper presents analytical formulations of the most important estimation errors - averaged squared bias error and mean squared error - for the class of entropy estimator expressed as a sum of single variable functions. The class of entropy estimator includes almost all important entropy estimators that have been proposed heretofore. Furthermore, this paper presents an optimal entropy estimator that can minimize mean squared error of the estimate under the condition that averaged squared bias error of the estimate is restricted to below an arbitrary value. A numerical experiment demonstrates that the proposed entropy estimator provides a lower mean squared error than conventional entropy estimators when entropy is estimated as an ensemble mean over plural entropy estimates obtained for different independent data. Such estimation is often utilized for biological signals, e.g., neural signals, because of biological tiredness and adaptation property.

UR - http://www.scopus.com/inward/record.url?scp=33745953160&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33745953160&partnerID=8YFLogxK

U2 - 10.1109/IJCNN.2005.1556038

DO - 10.1109/IJCNN.2005.1556038

M3 - Conference contribution

AN - SCOPUS:33745953160

SN - 0780390482

SN - 9780780390485

T3 - Proceedings of the International Joint Conference on Neural Networks

SP - 1280

EP - 1285

BT - Proceedings of the International Joint Conference on Neural Networks, IJCNN 2005

T2 - International Joint Conference on Neural Networks, IJCNN 2005

Y2 - 31 July 2005 through 4 August 2005

ER -