Speaker: Assistant Professor Yuanzhe XI, Emory University
Date and Time: December 2, 2:00-3:00 p.m.
Location: Scheller College of Business Room 102
Host: CSE Assistant Professor Florian Schäfer
Title: An Adaptive Factorized Nystrom Preconditioner for Kernal Matrices
Abstract: Kernel matrices can help handle nonlinearities in the data in many machine learning applications. The entries of a kernel matrix are the values of a kernel function for all pairs of points in a given dataset. Since the spectrum of the kernel matrix associated with the same dataset can vary dramatically as the parameters of the kernel function change, developing robust numerical schemes for kernel matrices associated with a wide range of parameters is a challenging task. In this talk we will present the newly developed Adaptive Factorized Nystrom (AFN) preconditioner for solving regularized linear systems associated with kernel matrices. The AFN preconditioner is superior to other Nystrom preconditioners when the rank of the Nystrom approximation is large, i.e., for kernel function parameters that lead to kernel matrices with eigenvalues that decay slowly and can adaptively choose the approximation rank to balance accuracy and cost. We will also demonstrate that AFN preconditioner can accelerate convergence and reduce variance of the stochastic algorithms used to estimate the log-determinant and its derivative of the kernel matrix, which are two main operations required in Gaussian process hyperparameter optimization. Experiments on various synthetic and real datasets ranging from low to high dimensions verify the effectiveness and robustness of the proposed preconditioner.
Bio: Yuanzhe Xi is currently an assistant professor in the Department of Mathematics at Emory University. He received his Ph.D. degree in Mathematics at Purdue University in 2014 and worked as a postdoc in the Department of Computer Science and Engineering at the University of Minnesota between 2014-2018. His research interests lie primarily in numerical linear algebra, scientific computing and data science. He is now working on fast algorithms for kernel methods and acceleration methods for solving nonlinear optimization and scientific computing problems.