<div dir="ltr"><div style="font-size:12.8px"><span style="font-size:12.8px;font-family:arial,helvetica,sans-serif">When:</span><span style="font-size:12.8px;font-family:arial,helvetica,sans-serif"> Wednesday, October 25th at </span><span style="font-size:12.8px;font-family:arial,helvetica,sans-serif">11:00 am</span><br></div><div style="font-size:12.8px"><div id="gmail-m_8452836149434112396gmail-m_2038649208801028506gmail-m_-3191619973981293488gmail-m_-8851913240295298494gmail-m_-2556873356046171619gmail-m_6077973053988611529gmail-m_5169817330970536202gmail-:jk" class="gmail-m_8452836149434112396gmail-m_2038649208801028506gmail-m_-3191619973981293488gmail-m_-8851913240295298494gmail-m_-2556873356046171619gmail-m_6077973053988611529gmail-m_5169817330970536202gmail-ii gmail-m_8452836149434112396gmail-m_2038649208801028506gmail-m_-3191619973981293488gmail-m_-8851913240295298494gmail-m_-2556873356046171619gmail-m_6077973053988611529gmail-m_5169817330970536202gmail-gt gmail-m_8452836149434112396gmail-m_2038649208801028506gmail-m_-3191619973981293488gmail-m_-8851913240295298494gmail-m_-2556873356046171619gmail-m_6077973053988611529gmail-m_5169817330970536202gmail-adP gmail-m_8452836149434112396gmail-m_2038649208801028506gmail-m_-3191619973981293488gmail-m_-8851913240295298494gmail-m_-2556873356046171619gmail-m_6077973053988611529gmail-m_5169817330970536202gmail-adO" style="margin-bottom:0px;margin-left:0px;padding-bottom:5px"><div id="gmail-m_8452836149434112396gmail-m_2038649208801028506gmail-m_-3191619973981293488gmail-m_-8851913240295298494gmail-m_-2556873356046171619gmail-m_6077973053988611529gmail-m_5169817330970536202gmail-:qr" class="gmail-m_8452836149434112396gmail-m_2038649208801028506gmail-m_-3191619973981293488gmail-m_-8851913240295298494gmail-m_-2556873356046171619gmail-m_6077973053988611529gmail-m_5169817330970536202gmail-a3s gmail-m_8452836149434112396gmail-m_2038649208801028506gmail-m_-3191619973981293488gmail-m_-8851913240295298494gmail-m_-2556873356046171619gmail-m_6077973053988611529gmail-m_5169817330970536202gmail-aXjCH gmail-m_8452836149434112396gmail-m_2038649208801028506gmail-m_-3191619973981293488gmail-m_-8851913240295298494gmail-m_-2556873356046171619gmail-m_6077973053988611529gmail-m_5169817330970536202gmail-m157fdbb4a58e966d"><div dir="ltr"><div style="font-size:12.8px"><font face="arial, helvetica, sans-serif" style="font-size:12.8px"><span style="font-size:12.8px"><br></span><span style="font-size:12.8px">Where: TTIC, 6045 S. Kenwood Avenue, 5th Floor, Room 526</span><span style="font-size:12.8px"><br><br></span><span style="font-size:12.8px">Who: </span></font><span style="font-size:12.8px;font-family:arial,helvetica,sans-serif"> Ashia Wilson, UC Berkeley</span></div><div style="font-size:12.8px"><span style="font-size:small"><br></span></div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px"><p class="MsoNormal" style="margin-bottom:0.0001pt"><font face="arial, helvetica, sans-serif">Title: A Dynamical View of
Optimization Algorithms<span></span></font></p>
<p class="MsoNormal" style="margin-bottom:0.0001pt"><span><font face="arial, helvetica, sans-serif"> </font></span></p>
<p class="MsoNormal" style="margin-bottom:0.0001pt"><font face="arial, helvetica, sans-serif">Abstract: Optimization is a core primitive in statistics, machine
learning, and data analytics. Within these fields, the rapid growth in the scale and complexity of modern datasets
has led to a focus on two classes of algorithms: gradient methods and momentum
methods (also referred to as accelerated methods). Momentum methods, first
proposed by Nesterov in 1983, achieve faster convergence rates than gradient
methods. However, unlike gradient methods, they are not descent methods and providing
robust performance guarantees remains a challenge. In the Euclidean setting,
momentum methods can be understood as modeling the dynamics of a damped
harmonic oscillator; making this intuition precise however, and generalizing it
to other geometries has been difficult. Furthermore, derivations of momentum
methods do not flow from a single underlying principle, but tend to rely on case-specific
algebra using a technique - considered by many to be esoteric - called <i>estimate sequences.</i><span></span></font></p>
<p class="MsoNormal" style="margin-bottom:0.0001pt"><span><font face="arial, helvetica, sans-serif"> </font></span></p>
<p class="MsoNormal" style="margin-bottom:0.0001pt"><font face="arial, helvetica, sans-serif">The first part of our work introduces a variational, continuous-time
framework for understanding </font><span style="font-family:arial,helvetica,sans-serif;font-size:12.8px">momentum methods. We show that there is a family of Lagrangian
functionals, that we call </span><i style="font-family:arial,helvetica,sans-serif;font-size:12.8px">Bregman Lagrangians</i><span style="font-family:arial,helvetica,sans-serif;font-size:12.8px">,
which generate dynamics corresponding to momentum methods in continuous</span><i style="font-family:arial,helvetica,sans-serif;font-size:12.8px"> </i><span style="font-family:arial,helvetica,sans-serif;font-size:12.8px">time. In particular, momentum methods
can be understood as arising from various discretization</span><i style="font-family:arial,helvetica,sans-serif;font-size:12.8px"> </i><span style="font-family:arial,helvetica,sans-serif;font-size:12.8px">techniques applied to these continuous time dynamics. The second
part of our work strengthens</span><i style="font-family:arial,helvetica,sans-serif;font-size:12.8px"> </i><span style="font-family:arial,helvetica,sans-serif;font-size:12.8px">this
connection. We demonstrate how to derive families of Lyapunov functions which
can certify</span><i style="font-family:arial,helvetica,sans-serif;font-size:12.8px"> rates of convergence</i><span style="font-family:arial,helvetica,sans-serif;font-size:12.8px"> for
the continuous time momentum dynamics. We further demonstrate how</span><i style="font-family:arial,helvetica,sans-serif;font-size:12.8px"> </i><span style="font-family:arial,helvetica,sans-serif;font-size:12.8px">the proofs of convergence of momentum
methods can be understood as bounding discretization</span><i style="font-family:arial,helvetica,sans-serif;font-size:12.8px"> </i><span style="font-family:arial,helvetica,sans-serif;font-size:12.8px">errors of the Lyapunov function when moving to discrete time. Along
the way, we prove an equivalence between these family of Lyapunov functions and
the technique of estimate sequences. The</span><i style="font-family:arial,helvetica,sans-serif;font-size:12.8px"> </i><span style="font-family:arial,helvetica,sans-serif;font-size:12.8px">following
is joint work with Andre Wibisono, Stephen Tu, Shivaram Venkataraman, Alex
Gittens, Benjamin Recht, and Michael I. Jordan.</span></p></div></div></div></div></div><div class="gmail_extra" style="font-size:12.8px"><div style="font-size:12.8px"><br></div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px">Host: <a href="mailto:nati@ttic.edu">Nathan Srebro</a></div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px"><span style="font-family:arial,helvetica,sans-serif;font-size:12.8px">******************************</span><span style="font-family:arial,helvetica,sans-serif;font-size:12.8px"><wbr>******************************</span><span style="font-family:arial,helvetica,sans-serif;font-size:12.8px"><wbr>******************************</span><span style="font-family:arial,helvetica,sans-serif;font-size:12.8px"><wbr>********</span><br></div><div style="font-size:12.8px"><div style="font-size:12.8px"><p class="MsoNormal" style="margin-bottom:0.0001pt;font-size:12.8px"><font face="arial, helvetica, sans-serif"> </font></p><p class="MsoNormal" style="margin-bottom:0.0001pt;font-size:12.8px"><font face="arial, helvetica, sans-serif">The TTIC <span class="gmail-m_8452836149434112396gmail-m_2038649208801028506gmail-m_-3191619973981293488gmail-m_-8851913240295298494gmail-m_-2556873356046171619gmail-m_6077973053988611529gmail-m_5169817330970536202gmail-m_-6945911958741551355gmail-m_-1657390673076249714gmail-m_-3188658297228096346gmail-m_5767020761850768500gmail-m_9173063153275566578gmail-m_-8878884332252521502gmail-m_796811872599655286gmail-il"><span class="gmail-m_8452836149434112396gmail-m_2038649208801028506gmail-m_-3191619973981293488gmail-il"><span class="gmail-m_8452836149434112396gmail-m_2038649208801028506gmail-il"><span class="gmail-il">Young</span></span></span></span> <span class="gmail-m_8452836149434112396gmail-m_2038649208801028506gmail-m_-3191619973981293488gmail-m_-8851913240295298494gmail-m_-2556873356046171619gmail-m_6077973053988611529gmail-m_5169817330970536202gmail-m_-6945911958741551355gmail-m_-1657390673076249714gmail-m_-3188658297228096346gmail-m_5767020761850768500gmail-m_9173063153275566578gmail-m_-8878884332252521502gmail-m_796811872599655286gmail-il"><span class="gmail-m_8452836149434112396gmail-m_2038649208801028506gmail-m_-3191619973981293488gmail-il"><span class="gmail-m_8452836149434112396gmail-m_2038649208801028506gmail-il"><span class="gmail-il">Researcher</span></span></span></span> <span class="gmail-il">Seminar</span> <span class="gmail-m_8452836149434112396gmail-m_2038649208801028506gmail-m_-3191619973981293488gmail-m_-8851913240295298494gmail-m_-2556873356046171619gmail-m_6077973053988611529gmail-m_5169817330970536202gmail-m_-6945911958741551355gmail-m_-1657390673076249714gmail-m_-3188658297228096346gmail-m_5767020761850768500gmail-m_9173063153275566578gmail-m_-8878884332252521502gmail-m_796811872599655286gmail-il"><wbr>Series</span> (<a href="http://www.ttic.edu/young-researcher.php" target="_blank">http://www.ttic.edu/yo<wbr>ung-<span class="gmail-m_8452836149434112396gmail-m_2038649208801028506gmail-m_-3191619973981293488gmail-m_-8851913240295298494gmail-m_-2556873356046171619gmail-m_6077973053988611529gmail-m_5169817330970536202gmail-m_-6945911958741551355gmail-m_-1657390673076249714gmail-m_-3188658297228096346gmail-m_5767020761850768500gmail-m_9173063153275566578gmail-m_-8878884332252521502gmail-m_796811872599655286gmail-il"><span class="gmail-m_8452836149434112396gmail-m_2038649208801028506gmail-m_-3191619973981293488gmail-il"><span class="gmail-m_8452836149434112396gmail-m_2038649208801028506gmail-il"><span class="gmail-il">researcher</span></span></span></span>.php</a>) features talks by Ph.D. students and postdocs whose <span class="gmail-m_8452836149434112396gmail-m_2038649208801028506gmail-m_-3191619973981293488gmail-m_-8851913240295298494gmail-m_-2556873356046171619gmail-m_6077973053988611529gmail-m_5169817330970536202gmail-m_-6945911958741551355gmail-m_-1657390673076249714gmail-m_-3188658297228096346gmail-m_5767020761850768500gmail-m_9173063153275566578gmail-m_-8878884332252521502gmail-m_796811872599655286gmail-il"><span class="gmail-m_8452836149434112396gmail-m_2038649208801028506gmail-m_-3191619973981293488gmail-il"><span class="gmail-m_8452836149434112396gmail-m_2038649208801028506gmail-il">research</span></span></span> is of broad interest to the computer science community. The <span class="gmail-m_8452836149434112396gmail-m_2038649208801028506gmail-m_-3191619973981293488gmail-m_-8851913240295298494gmail-m_-2556873356046171619gmail-m_6077973053988611529gmail-m_5169817330970536202gmail-m_-6945911958741551355gmail-m_-1657390673076249714gmail-m_-3188658297228096346gmail-m_5767020761850768500gmail-m_9173063153275566578gmail-m_-8878884332252521502gmail-m_796811872599655286gmail-il">series</span> provides an opportunity for early-career <span class="gmail-m_8452836149434112396gmail-m_2038649208801028506gmail-m_-3191619973981293488gmail-m_-8851913240295298494gmail-m_-2556873356046171619gmail-m_6077973053988611529gmail-m_5169817330970536202gmail-m_-6945911958741551355gmail-m_-1657390673076249714gmail-m_-3188658297228096346gmail-m_5767020761850768500gmail-m_9173063153275566578gmail-m_-8878884332252521502gmail-m_796811872599655286gmail-il"><span class="gmail-m_8452836149434112396gmail-m_2038649208801028506gmail-m_-3191619973981293488gmail-il"><span class="gmail-m_8452836149434112396gmail-m_2038649208801028506gmail-il">researchers</span></span></span> to present recent work to and meet with students and faculty at TTIC and nearby universities.</font></p><p class="MsoNormal" style="margin-bottom:0.0001pt;font-size:12.8px"><font face="arial, helvetica, sans-serif"><br>The seminars are typically held on Wednesdays at 11:00am in TTIC Room 526.<br><br>For additional information, please contact Matthew Walter (<a href="mailto:mwalter@ttic.edu" target="_blank">mwalter@ttic.edu</a>).</font></p><p class="MsoNormal" style="margin-bottom:0.0001pt;font-size:12.8px"><font face="arial, helvetica, sans-serif"><br></font></p><p class="MsoNormal" style="margin-bottom:0.0001pt;font-size:12.8px"><font face="arial, helvetica, sans-serif"><br></font></p><p class="MsoNormal" style="margin-bottom:0.0001pt;font-size:12.8px"><font face="arial, helvetica, sans-serif"><br></font></p><p class="MsoNormal" style="margin-bottom:0.0001pt;font-size:12.8px"><font face="arial, helvetica, sans-serif"><br></font></p></div></div></div><div><div class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><font face="arial, helvetica, sans-serif">Mary C. Marre</font><div><font face="arial, helvetica, sans-serif">Administrative Assistant</font></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6"><b>Toyota Technological Institute</b></font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">6045 S. Kenwood Avenue</font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Room 504</font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Chicago, IL 60637</font></i></div><div><i><font face="arial, helvetica, sans-serif">p:(773) 834-1757</font></i></div><div><i><font face="arial, helvetica, sans-serif">f: (773) 357-6970</font></i></div><div><b><i><a href="mailto:mmarre@ttic.edu" target="_blank"><font face="arial, helvetica, sans-serif">mmarre@ttic.edu</font></a></i></b></div></div></div></div></div></div></div></div></div></div>
</div>