<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div dir="auto" class="" style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;"><div class=""><div dir="auto" class="" style="word-wrap: break-word; line-break: after-white-space;"><font face="Arial" class=""><font color="#831100" class=""><span class="" style="font-size: 14px;"><b class="">University of Chicago and Toyota Technological Institute at Chicago<br class="">Machine Learning Seminar Series</b></span></font><br class=""><br class=""><br class=""><b class="" style="font-size: 14px;">Tengyuan Liang</b><br class=""><span class="" style="font-size: 14px;">University of Chicago Booth</span><br class=""><br class=""><b class="" style="font-size: 14px;">Wednesday, May 01, 2019 at 1:00 - 2:00 pm<br class="">Harper Center (Booth) Room 219</b><br class=""><br class=""><br class=""><b class="" style="font-size: 14px;">Title:</b></font></div><div dir="auto" class="" style="word-wrap: break-word; line-break: after-white-space;"><font face="Arial" class=""><span class="" style="font-size: 14px;">New Thoughts on Adaptivity, Generalization and Interpolation Motivated from Neural Networks.</span><br class=""><b class="" style="font-size: 14px;"><font face="Arial" class="" style="font-weight: normal;"><b class=""><br class=""></b></font></b></font></div><div dir="auto" class="" style="word-wrap: break-word; line-break: after-white-space;"><font face="Arial" class=""><span class=""><font face="Arial" class="" style="font-size: 14px;"><b class="">Abstract:</b><br class=""></font><div class=""><span class="" style="font-size: 14px;">Consider the problem: given data pair (x, y) drawn from a kernel Hilbert space (RKHS) approach indexed by the training process of neural networks. We show that when reaching any local stationarity, gradient flow learns an adaptive RKHS representation, and performs the global least squares projection onto the adaptive RKHS, simultaneously. In addition, we prove that as the RKHS is data-adaptive and task-specific, the residual for f_* lies in a subspace that is smaller than the orthogonal complement of the RKHS, formalizing the representation and approximation benefits of neural networks. Then we will move to discuss generalization for interpolation methods in RKHS. In the absence of explicit regularization, Kernel “Ridgeless” Regression with nonlinear kernels has the potential to fit the training data perfectly. It has been observed empirically, however, that such interpolated solutions can still generalize well on test data. We isolate a phenomenon of implicit regularization for minimum-norm interpolated solutions which is due to a combination of high dimensionality of the input data, curvature of the kernel function, and favorable geometric properties of the data such as an eigenvalue decay of the empirical covariance and kernel matrices. In addition to deriving a data-dependent upper bound on the out-of-sample error, we present experimental evidence suggesting that the phenomenon occurs in the MNIST dataset</span></div><font face="Arial" class="" style="font-size: 14px;"><br class=""><b class="">Bio:</b><br class=""></font><div class=""><span class="" style="font-size: 14px;">Tengyuan Liang works on problems at the intersection of statistical inference, machine learning & optimization. His main research focuses on the mathematical and algorithmic aspects of inference and learning under computational budgets, especially for large-scale datasets. Specific topics include: computational difficulty and efficiency in statistical inference, statistical learning theory, and high-dimensional statistics. He is also interested in network science, online learning, stochastic optimization, and applied probability. His research has been published in top statistics journals, as well as in leading peer-reviewed machine learning conferences. Outside of academia, Liang has experience as a research scientist at Yahoo! Research at New York in 2016, collaborating on large-scale machine learning problems with industrial applications. Liang earned a PhD in statistics from the Wharton School at University of Pennsylvania in 2017, and a BS in mathematics and applied mathematics from Peking University in China in 2012. He joined the Chicago Booth faculty in 2017.</span></div><font face="Arial" class="" style="font-size: 14px;"><b class=""><i class="">Host: </i></b></font></span></font><font face="Arial" class=""><span class="" style="font-size: 14px;"><b class=""><i class="">Rebecca Willett </i></b></span></font></div></div><div class=""><br class=""></div><div class=""></div></div></body></html>