[Theory] TODAY: [Talks at TTIC] 12/11 Jingfeng Wu, Berkeley

[Brandie Jones via Theory]

*When:*        Thursday, December 11th at *11am CT*

*Where:       *Talk will be given *live, in-person* at

                       TTIC, 6045 S. Kenwood Avenue

                       5th Floor, Room 530


*Virtually:*  via Panopto (livestream
<https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=e42b8450-149b-4282-bedd-b3af01060aed>
)



*Who: *         Jingfeng Wu, Berkeley

*Title:*          A Statistical View on Implicit Regularization: Gradient
Descent Dominates Ridge
*Abstract: *A key puzzle in deep learning is how simple gradient methods
find generalizable solutions without explicit regularization. This talk
discusses the implicit regularization of gradient descent (GD) through the
lens of statistical dominance. Using linear regression as a clean proxy, we
present three surprising findings.

First, GD dominates ridge regression: with comparable regularization, the
excess risk of GD is always within a constant factor of ridge, but ridge
can be polynomially worse even when tuned optimally. Second, GD is
incomparable with online stochastic gradient descent (SGD). While it is
known that for certain problems GD can be polynomially better than SGD, the
reverse is also true: we construct problems, inspired by benign overfitting
theory, where optimally stopped GD is polynomially worse. Finally, GD
dominates SGD for a significant subclass of problems -- those with fast and
continuously decaying covariance spectra -- which includes all problems
satisfying the standard capacity condition.

This is joint work with Peter Bartlett, Sham Kakade, Jason Lee, and Bin Yu.

*Bio*: Jingfeng Wu is a postdoctoral fellow at the Simons Institute for the
Theory of Computing at UC Berkeley. His research focuses on deep learning
theory, optimization, and statistical learning. He earned his Ph.D. in
Computer Science from Johns Hopkins University in 2023. Prior to that, he
received a B.S. in Mathematics (2016) and an M.S. in Applied Mathematics
(2019), both from Peking University. In 2023, he was recognized as a Rising
Star in Data Science by the University of Chicago and UC San Diego.

*Host: Nati Srebro <nati at ttic.edu>*


*Brandie Jones *
*Executive **Administrative Assistant*
*Outreach Administrator *
Toyota Technological Institute
6045 S. Kenwood Avenue
Chicago, IL  60637
www.ttic.edu
*OOO: November 24th - December 7th*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/theory/attachments/20251211/7ae844b5/attachment-0001.html>