<div dir="ltr"><div class="gmail_default" style="font-family:georgia,serif;font-size:small"><div id="m_-1900821992702953967gmail-:3nu"><div id="m_-1900821992702953967gmail-:3zh" aria-label="Message Body" role="textbox" aria-multiline="true" style="direction:ltr;min-height:376px" aria-controls=":4fk" aria-expanded="false"><div class="gmail_default"><div class="gmail_default"><div><font color="#000000" face="georgia, serif"><b>When:</b>        Monday, March 4th<b> </b>at<b> <span style="background-color:rgb(255,255,0)">10</span><span style="background-color:rgb(255,255,0)">AM CT  </span></b></font><div dir="ltr"><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000" face="georgia, serif"> <br><b>Where:</b><b>  </b>     Talk will be given<span style="background-color:rgb(255,255,0)"> </span><span style="background-color:rgb(255,255,0)"><font style="font-weight:bold"><u>live, in-person</u></font><font style="font-weight:bold"> </font></span>at</font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000" face="georgia, serif">                       TTIC, 6045 S. Kenwood Avenue</font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000" face="georgia, serif">                       5th Floor, Room 530<b>              </b>   </font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000" face="georgia, serif"><br></font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"><b><span class="gmail_default"></span>Virtually:</b>     via Panopto (<a href="https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=eee0e89c-b2cc-4696-86cb-b11d011177ce" target="_blank">Livestream</a>)</font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"><br></font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif"><font color="#000000"><b>Who:          </b></font>Gon Buzaglo<font color="#000000">, T</font>echnion</font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"><b><br></b></font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif"><font color="#000000"><b>Title:</b>           </font>How Uniform Random Weights Induce Non-uniform Bias: Typical Interpolating Neural Networks Generalize with Narrow Teachers</font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"><br></font></p><font face="georgia, serif"><b>Abstract:</b>   A main theoretical puzzle is why over-parameterized Neural Networks (NNs) generalize well when trained to zero error (i.e., so they interpolate the data). Usually, the NN is trained with Stochastic Gradient Descent (SGD) or one of its variants. However, recent empirical work examined the generalization of a random NN that interpolates the data: the NN was sampled from a seemingly uniform prior over the parameters, conditioned on that the NN perfectly classifying the training set. Interestingly, such a NN sample typically generalized as well as SGD-trained NNs.<br><br>I will talk about our new paper, where we prove that such a random NN interpolator typically generalizes well if there exists an underlying narrow “teacher NN” that agrees with the labels. Specifically, we show that such a ‘flat’ prior over the NN parametrization induces a rich prior over the NN functions, due to the redundancy in the NN structure. In particular, this creates a bias towards simpler functions, which require less relevant parameters to represent --- enabling learning with a sample complexity approximately proportional to the complexity of the teacher (roughly, the number of non-redundant parameters), rather than the student's.</font><p class="MsoNormal" style="margin:0in 0in 8pt;line-height:14.95px"><font face="georgia, serif" color="#000000"><span style="background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><br></span></font></p></div></div></div><div class="gmail_default"><font face="georgia, serif"><font color="#000000"><b>Bio:</b>    </font>Gon Buzaglo is currently pursuing an MSc in Electrical & Computer Engineering at the Technion, under the guidance of Prof. Daniel Soudry. Concurrently, he is completing his final year of undergraduate studies at the Technion, with a dual major in Computer Science and Physics. His undergraduate research included a collaboration with Prof. Michal Irani in the Department of Computer Science and Applied Mathematics at the Weizmann Institute of Science. Gon's research interests are centered around theoretical machine learning, specifically employing mathematical insights and experiments to explore concepts such as generalization, memorization, and forgetting in neural networks.</font></div><div class="gmail_default" style="font-family:Arial,Helvetica,sans-serif"><font face="georgia, serif" color="#000000"><br></font></div><div class="gmail_default" style="font-family:Arial,Helvetica,sans-serif"><font face="georgia, serif" color="#000000"><b>Host:<a href="mailto:nati@ttic.edu" target="_blank"> Nati Srebro</a></b></font></div></div><div><br></div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><b><font color="#3d85c6">Brandie Jones </font></b><div><div><div><font color="#3d85c6"><b><i>Executive </i></b></font><b style="color:rgb(61,133,198)"><i>Administrative Assistant</i></b></div></div><div><font color="#3d85c6">Toyota Technological Institute</font></div><div><font color="#3d85c6">6045 S. Kenwood Avenue</font></div><div><font color="#3d85c6">Chicago, IL  60637</font></div></div><div><font color="#3d85c6"><a href="http://www.ttic.edu" target="_blank">www.ttic.edu</a> </font></div></div></div></div></div></div></div>