[Theory] NOW: [TTIC Talks] 3/31 Talks at TTIC: Gal Vardi, TTIC/ Hebrew University

[Brandie Jones]

*When:*         Friday, March 31st at 10:30AM CT



*Where:*       Talk will be given *live, in-person* at

                       TTIC, 6045 S. Kenwood Avenue

                       5th Floor, Room 530


*Virtually:*     via Panopto (Livestream
<https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=f7dd56c5-dee0-4ba1-b905-afbd011e7577>
)


*Who:          *Gal Vardi, TTIC/ Hebrew University


*Title:*           On Implicit Bias and Provable Generalization in
Overparameterized Neural Networks


*Abstract*:   When training large neural networks, there are generally many
weight combinations that perfectly fit the training data. However,
gradient-based training methods somehow tend to reach those which
generalize well, and understanding this "implicit bias" has been a subject
of extensive research. In this talk, I will discuss three recent works
which show settings where the implicit bias provably implies generalization
(in two-layer neural networks trained with gradient flow w.r.t. the
logistic loss): First, the implicit bias implies generalization in
univariate ReLU networks. Second, in ReLU networks where the data consists
of clusters and the correlations between cluster means are small, the
implicit bias leads to solutions that generalize well, but are highly
vulnerable to adversarial examples. Third, in Leaky-ReLU networks (as well
as linear classifiers), under certain assumptions on the input
distribution, the implicit bias implies benign overfitting: the estimators
interpolate noisy training data and simultaneously generalize well to test
data.

Based on joint works with Spencer Frei, Itay Safran, Peter L. Bartlett,
Jason D. Lee, and Nati Srebro.

*Bio:*   Gal is a postdoc at TTI-Chicago and the Hebrew University, hosted
by Nati Srebro and Amit Daniely as part of the NSF/Simons Collaboration on
the Theoretical Foundations of Deep Learning. Prior to that, he was a
postdoc at the Weizmann Institute, hosted by Ohad Shamir, and a PhD student
at the Hebrew University advised by Orna Kupferman. His research focuses on
theoretical machine learning, with an emphasis on deep-learning theory.


-- 
*Brandie Jones *
*Executive **Administrative Assistant*
Toyota Technological Institute
6045 S. Kenwood Avenue
Chicago, IL  60637
www.ttic.edu
Working Remotely on Tuesdays
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/theory/attachments/20230331/1d051364/attachment.html>