<div dir="ltr"><div><div class="gmail_default"><div class="gmail_default"><div class="gmail_default"><font color="#000000" face="georgia, serif"><span style="letter-spacing:0.2px"><b>When:    </b>Wednesday, April 9th<b> at </b></span><b style="letter-spacing:0.2px"><span style="background-color:rgb(255,255,0)">11AM CT</span></b></font></div><div class="gmail_default"><b style="letter-spacing:0.2px"><font face="georgia, serif" color="#000000"><br></font></b></div><div class="gmail_default"><font face="georgia, serif" color="#000000"><b style="letter-spacing:0.2px">Where:   </b>Talk will be given<span style="background-color:rgb(255,255,0)"> </span><span style="background-color:rgb(255,255,0)"><font style="font-weight:bold"><u>live, in-person</u></font><font style="font-weight:bold"> </font></span>at</font></div><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000" face="georgia, serif">                    TTIC, 6045 S. Kenwood Avenue</font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000" face="georgia, serif">                    5th Floor, Room 530<b>  </b></font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><b style="letter-spacing:0.2px"><font face="georgia, serif" color="#000000"><br></font></b></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"><b style="letter-spacing:0.2px">Virtually: </b><span style="letter-spacing:0.2px">via Panopto (</span><a href="https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=4803398c-95cb-4dc2-9632-b26e0116ffba" target="_blank" style="letter-spacing:0.2px">Livestream</a><span style="letter-spacing:0.2px">)</span><br></font></p><div class="gmail_default"><b style="letter-spacing:0.2px"><font face="georgia, serif" color="#000000"><br></font></b></div><div class="gmail_default"><font face="georgia, serif" color="#000000"><span style="letter-spacing:0.2px"><b>Who:      </b></span>Daniel Kunin, Stanford University </font></div><div class="gmail_default"><span style="letter-spacing:0.2px"><font face="georgia, serif" color="#000000"><br></font></span></div><div class="gmail_default"><font face="georgia, serif" color="#000000"><b style="letter-spacing:0.2px">Title:</b>       Learning Mechanics of Neural Networks: Conservation Laws, Implicit Biases, and Feature Learning</font></div><div class="gmail_default"><font face="georgia, serif" color="#000000"><br><b style="letter-spacing:0.2px">Abstract: </b>The success of neural networks is often attributed to their ability to extract task-relevant features from data through training, yet a precise understanding of this process remains elusive. In this talk, I will explore the learning dynamics of neural networks, focusing on when and how they learn features. First, I will review how the parameter initialization scale influences learning -- large scales result in a "kernel regime" that stays close to its initialization, while small scales lead to an "active regime" traversing between saddle points. In the second part of the talk, I will present ongoing work describing the saddle-to-saddle dynamics for two-layer neural networks with vanishing initializations. We conjecture that training follows a recursive optimization process, alternating between maximizing a utility function over "dormant neurons" and minimizing a cost function over "active neurons". We demonstrate how this framework unifies existing theories of feature learning, such as those for diagonal linear networks and matrix factorization, and extends to new settings, such as quadratic networks for modular addition.</font></div><div><font face="georgia, serif" color="#000000"><br></font></div><div><div class="gmail_default"><font face="georgia, serif" color="#000000"><b>Bio:</b> Daniel is currently a PhD candidate at the Institute for Computational and Mathematical Engineering at Stanford University, advised by Surya Ganguli. Daniel will join UC Berkeley as a Miller Fellow in the neuroscience and statistics departments this fall. His research focuses on understanding the learning dynamics of neural networks, particularly how inductive biases emerge during training and how networks extract meaningful representations from data.</font></div><div><font color="#000000" face="georgia, serif"><br></font></div></div></div><div><div class="gmail_default"><b><font face="georgia, serif" color="#000000">Host: <a href="mailto:nati@ttic.edu">Nati Srebro</a></font></b></div></div></div><font color="#888888"><font color="#888888"><div><br style="font-family:georgia,serif"></div></font></font></div><div><br></div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><b style="background-color:rgb(255,255,255)"><font color="#3d85c6">Brandie Jones </font></b><div><div><div><font color="#3d85c6"><b><i>Executive </i></b></font><b style="color:rgb(61,133,198)"><i>Administrative Assistant</i></b></div></div><div><span style="background-color:rgb(255,255,255)"><font color="#3d85c6">Toyota Technological Institute</font></span></div><div><span style="background-color:rgb(255,255,255)"><font color="#3d85c6">6045 S. Kenwood Avenue</font></span></div><div><span style="background-color:rgb(255,255,255)"><font color="#3d85c6">Chicago, IL  60637</font></span></div></div><div><span style="background-color:rgb(255,255,255)"><font color="#3d85c6"><a href="http://www.ttic.edu" target="_blank">www.ttic.edu</a> </font></span></div><div><span style="background-color:rgb(255,255,255)"><font color="#3d85c6"><br></font></span></div></div></div></div>