<div dir="ltr"><div class="gmail_default" style="font-family:georgia,serif;font-size:small"><div class="gmail_default"><div class="gmail_default"><div class="gmail_default"><div><font color="#000000" face="georgia, serif"><b>When:</b> <b>       </b> Friday, March 31st<b> </b>at<span style="font-weight:bold;background-color:rgb(255,255,0)"> 10:30AM CT </span></font><div dir="ltr"><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000" face="georgia, serif"> <br></font></p></div></div></div></div><div><div class="gmail_default"><div><div dir="ltr"><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000" face="georgia, serif"><b>Where:</b><b>  </b>     Talk will be given<span style="background-color:rgb(255,255,0)"> </span><span style="background-color:rgb(255,255,0)"><font style="font-weight:bold"><u>live, in-person</u></font><font style="font-weight:bold"> </font></span>at</font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000" face="georgia, serif">                       TTIC, 6045 S. Kenwood Avenue</font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000" face="georgia, serif">                       5th Floor, Room 530<b>              </b>   </font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000" face="georgia, serif"><br></font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"><b><span class="gmail_default"></span>Virtually:</b>     </font><span style="color:rgb(0,0,0)">via Panopto (</span><a href="https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=f7dd56c5-dee0-4ba1-b905-afbd011e7577" target="_blank">Livestream</a><span style="color:rgb(0,0,0)">)</span></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"><br></font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000"><font face="georgia, serif"><b>Who:          </b>Gal Vardi, TTIC/ Hebrew University </font></font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000"><font face="georgia, serif"><b><br></b></font></font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000"><font face="georgia, serif"><b>Title:</b>           </font><font face="georgia, serif"></font></font>On Implicit Bias and Provable Generalization in Overparameterized Neural Networks</p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"><br></font></p><font face="georgia, serif"><b>Abstract</b>:   When training large neural networks, there are generally many weight combinations that perfectly fit the training data. However, gradient-based training methods somehow tend to reach those which generalize well, and understanding this "implicit bias" has been a subject of extensive research. In this talk, I will discuss three recent works which show settings where the implicit bias provably implies generalization (in two-layer neural networks trained with gradient flow w.r.t. the logistic loss): First, the implicit bias implies generalization in univariate ReLU networks. Second, in ReLU networks where the data consists of clusters and the correlations between cluster means are small, the implicit bias leads to solutions that generalize well, but are highly vulnerable to adversarial examples. Third, in Leaky-ReLU networks (as well as linear classifiers), under certain assumptions on the input distribution, the implicit bias implies benign overfitting: the estimators interpolate noisy training data and simultaneously generalize well to test data.<br><br>Based on joint works with Spencer Frei, Itay Safran, Peter L. Bartlett, Jason D. Lee, and Nati Srebro.</font></div><div dir="ltr"><br></div></div></div><div class="gmail_default"><font face="georgia, serif" color="#000000"><b>Bio:</b>   </font>Gal is a postdoc at TTI-Chicago and the Hebrew University, hosted by Nati Srebro and Amit Daniely as part of the NSF/Simons Collaboration on the Theoretical Foundations of Deep Learning. Prior to that, he was a postdoc at the Weizmann Institute, hosted by Ohad Shamir, and a PhD student at the Hebrew University advised by Orna Kupferman. His research focuses on theoretical machine learning, with an emphasis on deep-learning theory.</div></div><div class="gmail_default"><br></div><div class="gmail_default"><br></div></div></div><div><br></div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><b style="background-color:rgb(255,255,255)"><font color="#3d85c6">Brandie Jones </font></b><div><div><div><font color="#3d85c6"><b><i>Executive </i></b></font><b style="color:rgb(61,133,198)"><i>Administrative Assistant</i></b></div></div><div><span style="background-color:rgb(255,255,255)"><font color="#3d85c6">Toyota Technological Institute</font></span></div><div><span style="background-color:rgb(255,255,255)"><font color="#3d85c6">6045 S. Kenwood Avenue</font></span></div><div><span style="background-color:rgb(255,255,255)"><font color="#3d85c6">Chicago, IL  60637</font></span></div></div><div><span style="background-color:rgb(255,255,255)"><font color="#3d85c6"><a href="http://www.ttic.edu" target="_blank">www.ttic.edu</a> </font></span></div><div><font color="#3d85c6"><div style="background-color:rgb(238,238,238)"><div style="color:rgb(32,33,36)"><span style="color:rgb(61,133,198)">Working Remotely on Tuesdays</span></div></div></font></div></div></div></div>