TY - JOUR

T1 - Generalization ability of a perceptron with nonmonotonic transfer function

AU - Inoue, Jun ichi

AU - Nishimori, Hidetoshi

AU - Kabashima, Yoshiyuki

N1 - Copyright:
Copyright 2017 Elsevier B.V., All rights reserved.

PY - 1998

Y1 - 1998

N2 - We investigate the generalization ability of a perceptron with nonmonotonic transfer function of a reversed-wedge type in on-line mode. This network is identical to a parity machine, a multilayer network. We consider several learning algorithms. By the perceptron algorithm the generalization error is shown to decrease by the [Formula Presented]-law similarly to the case of a simple perceptron in a restricted range of the parameter [Formula Presented] characterizing the nonmonotonic transfer function. For other values of [Formula Presented], the perceptron algorithm leads to the state where the weight vector of the student is just opposite to that of the teacher. The Hebbian learning algorithm has a similar property; it works only in a limited range of the parameter. The conventional AdaTron algorithm does not give a vanishing generalization error for any values of [Formula Presented]. We thus introduce a modified AdaTron algorithm that yields a good performance for all values of [Formula Presented]. We also investigate the effects of optimization of the learning rate as well as of the learning algorithm. Both methods give excellent learning curves proportional to [Formula Presented]. The latter optimization is related to the Bayes statistics and is shown to yield useful hints to extract maximum amount of information necessary to accelerate learning processes.

AB - We investigate the generalization ability of a perceptron with nonmonotonic transfer function of a reversed-wedge type in on-line mode. This network is identical to a parity machine, a multilayer network. We consider several learning algorithms. By the perceptron algorithm the generalization error is shown to decrease by the [Formula Presented]-law similarly to the case of a simple perceptron in a restricted range of the parameter [Formula Presented] characterizing the nonmonotonic transfer function. For other values of [Formula Presented], the perceptron algorithm leads to the state where the weight vector of the student is just opposite to that of the teacher. The Hebbian learning algorithm has a similar property; it works only in a limited range of the parameter. The conventional AdaTron algorithm does not give a vanishing generalization error for any values of [Formula Presented]. We thus introduce a modified AdaTron algorithm that yields a good performance for all values of [Formula Presented]. We also investigate the effects of optimization of the learning rate as well as of the learning algorithm. Both methods give excellent learning curves proportional to [Formula Presented]. The latter optimization is related to the Bayes statistics and is shown to yield useful hints to extract maximum amount of information necessary to accelerate learning processes.

UR - http://www.scopus.com/inward/record.url?scp=11744325654&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=11744325654&partnerID=8YFLogxK

U2 - 10.1103/PhysRevE.58.849

DO - 10.1103/PhysRevE.58.849

M3 - Article

AN - SCOPUS:11744325654

VL - 58

SP - 849

EP - 860

JO - Physical Review E

JF - Physical Review E

SN - 2470-0045

IS - 1

ER -