TY - GEN
T1 - Fast newton-CG method for batch learning of conditional random fields
AU - Tsuboi, Yuta
AU - Unno, Yuya
AU - Kashima, Hisashi
AU - Okazaki, Naoaki
PY - 2011/11/2
Y1 - 2011/11/2
N2 - We propose a fast batch learning method for linear-chain Conditional Random Fields (CRFs) based on Newton-CG methods. Newton-CG methods are a variant of Newton method for high-dimensional problems. They only require the Hessian-vector products instead of the full Hessian matrices. To speed up Newton-CG methods for the CRF learning, we derive a novel dynamic programming procedure for the Hessian-vector products of the CRF objective function. The proposed procedure can reuse the byproducts of the time-consuming gradient computation for the Hessian-vector products to drastically reduce the total computation time of the Newton-CG methods. In experiments with tasks in natural language processing, the proposed method outperforms a conventional quasi-Newton method. Remarkably, the proposed method is competitive with online learning algorithms that are fast but unstable.
AB - We propose a fast batch learning method for linear-chain Conditional Random Fields (CRFs) based on Newton-CG methods. Newton-CG methods are a variant of Newton method for high-dimensional problems. They only require the Hessian-vector products instead of the full Hessian matrices. To speed up Newton-CG methods for the CRF learning, we derive a novel dynamic programming procedure for the Hessian-vector products of the CRF objective function. The proposed procedure can reuse the byproducts of the time-consuming gradient computation for the Hessian-vector products to drastically reduce the total computation time of the Newton-CG methods. In experiments with tasks in natural language processing, the proposed method outperforms a conventional quasi-Newton method. Remarkably, the proposed method is competitive with online learning algorithms that are fast but unstable.
UR - http://www.scopus.com/inward/record.url?scp=80055049543&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=80055049543&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:80055049543
SN - 9781577355083
T3 - Proceedings of the National Conference on Artificial Intelligence
SP - 489
EP - 494
BT - AAAI-11 / IAAI-11 - Proceedings of the 25th AAAI Conference on Artificial Intelligence and the 23rd Innovative Applications of Artificial Intelligence Conference
T2 - 25th AAAI Conference on Artificial Intelligence and the 23rd Innovative Applications of Artificial Intelligence Conference, AAAI-11 / IAAI-11
Y2 - 7 August 2011 through 11 August 2011
ER -