TY - GEN
T1 - On the use of metaheuristics in hyperparameters optimization of Gaussian processes
AU - Palar, Pramudita Satria
AU - Zuhal, Lavi Rizki
AU - Shimoyama, Koji
N1 - Publisher Copyright:
© 2019 Association for Computing Machinery.
Copyright:
Copyright 2020 Elsevier B.V., All rights reserved.
PY - 2019/7/13
Y1 - 2019/7/13
N2 - Due to difficulties such as multiple local optima and flat landscape, it is suggested to use global optimization techniques to discover the global optimum of the auxiliary optimization problem of finding good Gaussian Processes (GP) hyperparameters. We investigated the performance of genetic algorithms (GA), particle swarm optimization (PSO), differential evolution (DE), and covariance matrix adaptation evolution strategy (CMA-ES) for optimizing hyperparameters of GP. The study was performed on two artificial problems and also one real-world problem. From the results, we observe that PSO, CMA-ES, and DE/local-to-best/1 consistently outperformed two variants of GA and DE/rand/1 with per-generation-dither on all problems. In particular, CMA-ES is an attractive method since it is quasi-parameter free and it also demonstrates good exploitative and explorative power on optimizing the hyperparameters.
AB - Due to difficulties such as multiple local optima and flat landscape, it is suggested to use global optimization techniques to discover the global optimum of the auxiliary optimization problem of finding good Gaussian Processes (GP) hyperparameters. We investigated the performance of genetic algorithms (GA), particle swarm optimization (PSO), differential evolution (DE), and covariance matrix adaptation evolution strategy (CMA-ES) for optimizing hyperparameters of GP. The study was performed on two artificial problems and also one real-world problem. From the results, we observe that PSO, CMA-ES, and DE/local-to-best/1 consistently outperformed two variants of GA and DE/rand/1 with per-generation-dither on all problems. In particular, CMA-ES is an attractive method since it is quasi-parameter free and it also demonstrates good exploitative and explorative power on optimizing the hyperparameters.
KW - Gaussian Process Regression
KW - Hyperparameters optimization
KW - Likelihood function
KW - Metaheuristics
UR - http://www.scopus.com/inward/record.url?scp=85070637239&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85070637239&partnerID=8YFLogxK
U2 - 10.1145/3319619.3322012
DO - 10.1145/3319619.3322012
M3 - Conference contribution
AN - SCOPUS:85070637239
T3 - GECCO 2019 Companion - Proceedings of the 2019 Genetic and Evolutionary Computation Conference Companion
SP - 263
EP - 264
BT - GECCO 2019 Companion - Proceedings of the 2019 Genetic and Evolutionary Computation Conference Companion
PB - Association for Computing Machinery, Inc
T2 - 2019 Genetic and Evolutionary Computation Conference, GECCO 2019
Y2 - 13 July 2019 through 17 July 2019
ER -