On the bias of predictive distribution in pattern recognition

Masakazu Iwamura, Shinichiro Omachi, Hirotomo Aso

Research output: Contribution to journalArticlepeer-review

Abstract

To estimate distribution from training samples, maximum likelihood estimation treats the unknown parameter of the distribution as a constant. On the other hand, Bayesian estimation treats the parameter as a random variable. In pattern recognition, Bayesian estimation has been known to improve recognition accuracy. However, it was pointed out that Bayesian estimation is not effective due to the bias of the likelihood when sample sizes of classes are not the same. In this paper, we show that recognition accuracy is improved by modifying the bias of the likelihood when sample sizes are not the same. This indicates that the cause of the ineffectiveness is the bias. We derive the formula of the bias of Geisser's predictive distribution without any approximation, and show a way of modification of the bias of the likelihood. We confirm the effectiveness of the proposed method in experiments. In addition, the derived formula gives the theoretical background of a known empirical knowledge.

Original languageEnglish
Pages (from-to)45-54
Number of pages10
JournalSystems and Computers in Japan
Volume36
Issue number5
DOIs
Publication statusPublished - 2005 May

Keywords

  • Bayesian estimation
  • Bias of likelihood
  • Geisser's predictive distribution
  • Pattern recognition
  • Predictive distribution
  • Wishart distribution

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Information Systems
  • Hardware and Architecture
  • Computational Theory and Mathematics

Fingerprint Dive into the research topics of 'On the bias of predictive distribution in pattern recognition'. Together they form a unique fingerprint.

Cite this