Learning entropy as a learning-based information concept

Ivo Bukovsky, Witold Kinsner, Noriyasu Homma

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

Recently, a novel concept of a non-probabilistic novelty detection measure, based on a multi-scale quantification of unusually large learning efforts of machine learning systems, was introduced as learning entropy (LE). The key finding with LE is that the learning effort of learning systems is quantifiable as a novelty measure for each individually observed data point of otherwise complex dynamic systems, while the model accuracy is not a necessary requirement for novelty detection. This brief paper extends the explanation of LE from the point of an informatics approach towards a cognitive (learning-based) information measure emphasizing the distinction from Shannon's concept of probabilistic information. Fundamental derivations of learning entropy and of its practical estimations are recalled and further extended. The potentials, limitations, and, thus, the current challenges of LE are discussed.

Original languageEnglish
Article number166
JournalEntropy
Volume21
Issue number2
DOIs
Publication statusPublished - 2019 Feb 1

Keywords

  • Information
  • Learning
  • Learning systems
  • Non-probabilistic entropy
  • Novelty detection

ASJC Scopus subject areas

  • Physics and Astronomy(all)

Fingerprint Dive into the research topics of 'Learning entropy as a learning-based information concept'. Together they form a unique fingerprint.

  • Cite this