<div dir="ltr"><div class="gmail_default" style="font-family:georgia,serif;font-size:small"><div class="gmail_default" style="font-family:Arial,Helvetica,sans-serif"><div><font color="#000000" face="georgia, serif"><b>When:</b>         Tuesday, February 13th<b> </b>at<b> <span style="background-color:rgb(255,255,0)">11AM CT  </span></b></font><div dir="ltr"><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000" face="georgia, serif"> <br><b>Where:</b><b>  </b>     Talk will be given<span style="background-color:rgb(255,255,0)"> </span><span style="background-color:rgb(255,255,0)"><font style="font-weight:bold"><u>live, in-person</u></font><font style="font-weight:bold"> </font></span>at</font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000" face="georgia, serif">                       TTIC, 6045 S. Kenwood Avenue</font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000" face="georgia, serif">                       5th Floor, Room 530<b>              </b>   </font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000" face="georgia, serif"><br></font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"><b><span class="gmail_default"></span>Virtually:</b>     via Panopto (<a href="https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=edb9fb11-e93f-4e11-8e82-b109010ac95e" target="_blank">Livestream</a>)</font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"><br></font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000" face="georgia, serif"><b>Who:          </b>Jeremy Cohen, Carnegie Mellon</font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"><b><br></b></font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"><b>Title:</b>           The Dynamics of Gradient Descent in Deep Learning</font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"><br></font></p><p class="MsoNormal" style="margin:0in 0in 8pt;line-height:14.95px"><font face="georgia, serif" color="#000000"><b><span style="background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial">Abstract:   </span></b>Since 1986, neural networks have been trained using variants of gradient descent.  Why does gradient descent work?  The conventional wisdom holds that gradient descent works because its learning rate is set sufficiently small relative to the curvature of the objective function, which is treated as fixed a priori.  In this talk, we see that this is not the reason why gradient descent works.  Instead, we will see that gradient descent possesses an inbuilt mechanism for automatically regulating the curvature along its own optimization trajectory.  This mechanism, not previously known to optimization theory, is responsible for the successful convergence of gradient descent in neural network training.</font></p><p class="MsoNormal" style="margin:0in 0in 8pt;line-height:14.95px"><font face="georgia, serif" color="#000000"><span style="background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><br></span></font></p></div></div></div><div class="gmail_default" style="font-family:Arial,Helvetica,sans-serif"><font face="georgia, serif" color="#000000"><b>Bio:</b>    Jeremy Cohen is a PhD student at Carnegie Mellon, advised by Zico Kolter and Ameet Talwalkar.  He is interested in making deep learning less like alchemy and more like engineering.</font></div><div class="gmail_default" style="font-family:Arial,Helvetica,sans-serif"><font face="georgia, serif" color="#000000"><br></font></div><div class="gmail_default" style="font-family:Arial,Helvetica,sans-serif"><font face="georgia, serif" color="#000000"><b>Host:<a href="mailto:zhiyuanli@ttic.edu" target="_blank"> Zhiyuan Li</a> & <a href="mailto:nati@ttic.edu" target="_blank">Nati Srebro </a></b></font></div></div><div><br></div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><b style="background-color:rgb(255,255,255)"><font color="#3d85c6">Brandie Jones </font></b><div><div><div><font color="#3d85c6"><b><i>Executive </i></b></font><b style="color:rgb(61,133,198)"><i>Administrative Assistant</i></b></div></div><div><span style="background-color:rgb(255,255,255)"><font color="#3d85c6">Toyota Technological Institute</font></span></div><div><span style="background-color:rgb(255,255,255)"><font color="#3d85c6">6045 S. Kenwood Avenue</font></span></div><div><span style="background-color:rgb(255,255,255)"><font color="#3d85c6">Chicago, IL  60637</font></span></div></div><div><span style="background-color:rgb(255,255,255)"><font color="#3d85c6"><a href="http://www.ttic.edu" target="_blank">www.ttic.edu</a> </font></span></div></div></div></div>