<div dir="ltr"><div dir="ltr"><div class="gmail_default" style="font-size:small"><div class="gmail_default"><p style="color:rgb(80,0,80);font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><font face="arial, sans-serif" color="#000000"><font style="vertical-align:inherit"><font style="vertical-align:inherit"><b>When:</b> </font></font><font style="vertical-align:inherit"><font style="vertical-align:inherit"> Monday, March 29th at<b> 11:10 am CT</b></font></font><br></font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> </font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font style="color:rgb(0,0,0);vertical-align:inherit"><font style="vertical-align:inherit"><b>Where:</b> </font></font><font color="#000000">Zoom Virtual Talk (</font><font color="#0000ff"><b><a href="https://uchicagogroup.zoom.us/webinar/register/WN_WE0hdZNgQGmC-3e5vde_zA" target="_blank">register in advance here</a></b></font><font color="#000000">)</font></font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> </font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font style="color:rgb(80,0,80);vertical-align:inherit"><font style="vertical-align:inherit"><b>Who: </b> </font></font><font color="#000000">Wei Hu, Princeton University</font></font></p></div><div class="gmail_default"><font face="arial, sans-serif" color="#000000"><br></font></div><div class="gmail_default"><font face="arial, sans-serif" color="#000000"><br></font></div><div class="gmail_default"><div><font face="arial, sans-serif" color="#000000"><b>Title</b>: Opening the Black Box: Towards Theoretical Understanding of Deep Learning<br><br><div><b style="text-align:justify;white-space:pre-wrap">Abstract</b><span style="text-align:justify;white-space:pre-wrap">: Despite the phenomenal empirical successes of deep learning in many application domains, its underlying mathematical mechanisms remain poorly understood. Mysteriously, deep neural networks in practice can often fit training data almost perfectly and generalize remarkably well to unseen test data, despite highly non-convex optimization landscapes and significant over-parameterization. A solid theory not only can help us understand such mysteries, but also will be the key to improving the practice of deep learning and making it more principled, reliable, and easy-to-use.</span></div><span id="gmail-m_8274392582987159353gmail-m_-2390014943886158741gmail-m_-5104972856951789363gmail-m_-8519987321097201641m_-4034394888817761410gmail-m_3558794617303584713gmail-m_-8454003908662032912gmail-m_3829690996579951010gmail-docs-internal-guid-82470440-7fff-d514-cfa5-d38df5ceb32e"><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">In this talk, I will present our recent progress on building the theoretical foundations of deep learning, </span><span style="white-space:pre-wrap">by opening the black box of the interactions among data, model architecture, and training </span>algorithm<span style="white-space:pre-wrap">. </span><span style="white-space:pre-wrap">First, I will study the effect of making the neural network deeper, and will show </span><span style="white-space:pre-wrap">that gradient descent on deep linear neural networks induces an implicit bias towards low-rank solutions, which leads to an improved method for the classical low-rank matrix completion problem. </span><span style="white-space:pre-wrap">Next, turning to nonlinear deep neural networks, I will talk about a line of studies on wide neural networks, where by drawing a connection to the neural tangent kernels, we can answer various questions such as how training loss is minimized, why trained network can generalize well, and why certain component in the network architecture is useful; we also use theoretical insights to design a new simple and effective method for training on noisily labeled datasets. In closing, I will discuss key questions going forward towards building practically relevant theoretical foundations of modern machine learning.</span></p></span><br><b>Bio</b>: Wei Hu is a PhD candidate in the Department of Computer Science at Princeton University, advised by Sanjeev Arora. Previously, he obtained his bachelor's degree in Computer Science from Tsinghua University. He has also spent time as a research intern at research labs of Google and Microsoft. His current research interest is broadly in the theoretical foundations of modern machine learning. In particular, his main focus is on obtaining solid theoretical understanding of deep learning, as well as using theoretical insights to design practical and principled machine learning methods. He is a recipient of the Siebel Scholarship Class of 2021.</font></div><div><font face="arial, sans-serif"><br></font></div><div style="color:rgb(80,0,80)"><font face="arial, sans-serif"><b>Host:</b> <b><a href="mailto:avrim@ttic.edu" target="_blank">Avrim Blum</a></b></font></div><div style="color:rgb(80,0,80)"><font face="arial, sans-serif"><br></font></div><div style="color:rgb(80,0,80)"><font face="arial, sans-serif"><br></font></div><div style="color:rgb(80,0,80)"><font face="arial, sans-serif"><br></font></div><div style="color:rgb(80,0,80)"><font face="arial, sans-serif"><br></font></div></div></div><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><font face="arial, helvetica, sans-serif">Mary C. Marre</font><div><font face="arial, helvetica, sans-serif">Faculty Administrative Support</font></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6"><b>Toyota Technological Institute</b></font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">6045 S. Kenwood Avenue</font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Room 517</font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Chicago, IL 60637</font></i></div><div><i><font face="arial, helvetica, sans-serif">p:(773) 834-1757</font></i></div><div><i><font face="arial, helvetica, sans-serif">f: (773) 357-6970</font></i></div><div><b><i><a href="mailto:mmarre@ttic.edu" target="_blank"><font face="arial, helvetica, sans-serif">mmarre@ttic.edu</font></a></i></b></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Mar 28, 2021 at 3:44 PM Mary Marre <<a href="mailto:mmarre@ttic.edu">mmarre@ttic.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div style="font-size:small"><div><p style="color:rgb(80,0,80);font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><font face="arial, sans-serif" color="#000000"><font style="vertical-align:inherit"><font style="vertical-align:inherit"><b>When:</b> </font></font><font style="vertical-align:inherit"><font style="vertical-align:inherit"> Monday, March 29th at<b> 11:10 am CT</b></font></font><br></font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> </font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font style="color:rgb(0,0,0);vertical-align:inherit"><font style="vertical-align:inherit"><b>Where:</b> </font></font><font color="#000000">Zoom Virtual Talk (</font><font color="#0000ff"><b><a href="https://uchicagogroup.zoom.us/webinar/register/WN_WE0hdZNgQGmC-3e5vde_zA" target="_blank">register in advance here</a></b></font><font color="#000000">)</font></font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> </font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font style="color:rgb(80,0,80);vertical-align:inherit"><font style="vertical-align:inherit"><b>Who: </b> </font></font><font color="#000000">Wei Hu, Princeton University</font></font></p></div><div><font face="arial, sans-serif" color="#000000"><br></font></div><div><font face="arial, sans-serif" color="#000000"><br></font></div><div><div><font face="arial, sans-serif" color="#000000"><b>Title</b>: Opening the Black Box: Towards Theoretical Understanding of Deep Learning<br><br><div><b style="text-align:justify;white-space:pre-wrap">Abstract</b><span style="text-align:justify;white-space:pre-wrap">: Despite the phenomenal empirical successes of deep learning in many application domains, its underlying mathematical mechanisms remain poorly understood. Mysteriously, deep neural networks in practice can often fit training data almost perfectly and generalize remarkably well to unseen test data, despite highly non-convex optimization landscapes and significant over-parameterization. A solid theory not only can help us understand such mysteries, but also will be the key to improving the practice of deep learning and making it more principled, reliable, and easy-to-use.</span></div><span id="gmail-m_8274392582987159353gmail-m_-2390014943886158741gmail-m_-5104972856951789363gmail-m_-8519987321097201641m_-4034394888817761410gmail-m_3558794617303584713gmail-m_-8454003908662032912gmail-m_3829690996579951010gmail-docs-internal-guid-82470440-7fff-d514-cfa5-d38df5ceb32e"><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">In this talk, I will present our recent progress on building the theoretical foundations of deep learning, </span><span style="white-space:pre-wrap">by opening the black box of the interactions among data, model architecture, and training </span>algorithm<span style="white-space:pre-wrap">. </span><span style="white-space:pre-wrap">First, I will study the effect of making the neural network deeper, and will show </span><span style="white-space:pre-wrap">that gradient descent on deep linear neural networks induces an implicit bias towards low-rank solutions, which leads to an improved method for the classical low-rank matrix completion problem. </span><span style="white-space:pre-wrap">Next, turning to nonlinear deep neural networks, I will talk about a line of studies on wide neural networks, where by drawing a connection to the neural tangent kernels, we can answer various questions such as how training loss is minimized, why trained network can generalize well, and why certain component in the network architecture is useful; we also use theoretical insights to design a new simple and effective method for training on noisily labeled datasets. In closing, I will discuss key questions going forward towards building practically relevant theoretical foundations of modern machine learning.</span></p></span><br><b>Bio</b>: Wei Hu is a PhD candidate in the Department of Computer Science at Princeton University, advised by Sanjeev Arora. Previously, he obtained his bachelor's degree in Computer Science from Tsinghua University. He has also spent time as a research intern at research labs of Google and Microsoft. His current research interest is broadly in the theoretical foundations of modern machine learning. In particular, his main focus is on obtaining solid theoretical understanding of deep learning, as well as using theoretical insights to design practical and principled machine learning methods. He is a recipient of the Siebel Scholarship Class of 2021.</font></div><div><font face="arial, sans-serif"><br></font></div><div style="color:rgb(80,0,80)"><font face="arial, sans-serif"><b>Host:</b> <b><a href="mailto:avrim@ttic.edu" target="_blank">Avrim Blum</a></b></font></div><div style="color:rgb(80,0,80)"><font face="arial, sans-serif"><br></font></div><div style="color:rgb(80,0,80)"><font face="arial, sans-serif"><br></font></div><div style="color:rgb(80,0,80)"><font face="arial, sans-serif"><br></font></div><div style="color:rgb(80,0,80)"><font face="arial, sans-serif"><br></font></div><div style="color:rgb(80,0,80)"><font face="arial, sans-serif"><br></font></div></div></div><div><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><font face="arial, helvetica, sans-serif">Mary C. Marre</font><div><font face="arial, helvetica, sans-serif">Faculty Administrative Support</font></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6"><b>Toyota Technological Institute</b></font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">6045 S. Kenwood Avenue</font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Room 517</font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Chicago, IL 60637</font></i></div><div><i><font face="arial, helvetica, sans-serif">p:(773) 834-1757</font></i></div><div><i><font face="arial, helvetica, sans-serif">f: (773) 357-6970</font></i></div><div><b><i><a href="mailto:mmarre@ttic.edu" target="_blank"><font face="arial, helvetica, sans-serif">mmarre@ttic.edu</font></a></i></b></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Mar 22, 2021 at 5:15 PM Mary Marre <<a href="mailto:mmarre@ttic.edu" target="_blank">mmarre@ttic.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div><p style="color:rgb(80,0,80);font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><font face="arial, sans-serif" color="#000000"><font style="vertical-align:inherit"><font style="vertical-align:inherit"><b>When:</b> </font></font><font style="vertical-align:inherit"><font style="vertical-align:inherit"> Monday, March 29th at<b> 11:10 am CT</b></font></font><br></font></p><p class="MsoNormal" style="color:rgb(80,0,80);margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> </font></p><p class="MsoNormal" style="color:rgb(80,0,80);margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font style="color:rgb(0,0,0);vertical-align:inherit"><font style="vertical-align:inherit"><b>Where:</b> </font></font><font color="#000000">Zoom Virtual Talk (</font><font color="#0000ff"><b><a href="https://uchicagogroup.zoom.us/webinar/register/WN_WE0hdZNgQGmC-3e5vde_zA" target="_blank">register in advance here</a></b></font><font color="#000000">)</font></font></p><p class="MsoNormal" style="color:rgb(80,0,80);margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> </font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font style="color:rgb(80,0,80);vertical-align:inherit"><font style="vertical-align:inherit"><b>Who: </b> </font></font><font color="#000000">Wei Hu, Princeton University</font></font></p></div><div><font face="arial, sans-serif" color="#000000"><br></font></div><div><font face="arial, sans-serif" color="#000000"><br></font></div><div><div><font face="arial, sans-serif" color="#000000"><b>Title</b>: Opening the Black Box: Towards Theoretical Understanding of Deep Learning<br><br><div style="text-align:left"><b style="text-align:justify;white-space:pre-wrap">Abstract</b><span style="text-align:justify;white-space:pre-wrap">: Despite the phenomenal empirical successes of deep learning in many application domains, its underlying mathematical mechanisms remain poorly understood. Mysteriously, deep neural networks in practice can often fit training data almost perfectly and generalize remarkably well to unseen test data, despite highly non-convex optimization landscapes and significant over-parameterization. A solid theory not only can help us understand such mysteries, but also will be the key to improving the practice of deep learning and making it more principled, reliable, and easy-to-use.</span></div><span id="gmail-m_8274392582987159353gmail-m_-2390014943886158741gmail-m_-5104972856951789363gmail-m_-8519987321097201641m_-4034394888817761410gmail-m_3558794617303584713gmail-m_-8454003908662032912gmail-m_3829690996579951010gmail-docs-internal-guid-82470440-7fff-d514-cfa5-d38df5ceb32e"><br><p dir="ltr" style="text-align:left;line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">In this talk, I will present our recent progress on building the theoretical foundations of deep learning, </span><span style="white-space:pre-wrap">by opening the black box of the interactions among data, model architecture, and training </span>algorithm<span style="white-space:pre-wrap">. </span><span style="white-space:pre-wrap">First, I will study the effect of making the neural network deeper, and will show </span><span style="white-space:pre-wrap">that gradient descent on deep linear neural networks induces an implicit bias towards low-rank solutions, which leads to an improved method for the classical low-rank matrix completion problem. </span><span style="white-space:pre-wrap">Next, turning to nonlinear deep neural networks, I will talk about a line of studies on wide neural networks, where by drawing a connection to the neural tangent kernels, we can answer various questions such as how training loss is minimized, why trained network can generalize well, and why certain component in the network architecture is useful; we also use theoretical insights to design a new simple and effective method for training on noisily labeled datasets. In closing, I will discuss key questions going forward towards building practically relevant theoretical foundations of modern machine learning.</span></p></span><br><b>Bio</b>: Wei Hu is a PhD candidate in the Department of Computer Science at Princeton University, advised by Sanjeev Arora. Previously, he obtained his bachelor's degree in Computer Science from Tsinghua University. He has also spent time as a research intern at research labs of Google and Microsoft. His current research interest is broadly in the theoretical foundations of modern machine learning. In particular, his main focus is on obtaining solid theoretical understanding of deep learning, as well as using theoretical insights to design practical and principled machine learning methods. He is a recipient of the Siebel Scholarship Class of 2021.</font></div><div><font face="arial, sans-serif"><br style="color:rgb(34,34,34)"></font></div><div style="color:rgb(80,0,80)"><font face="arial, sans-serif"><b>Host:</b> <a href="mailto:avrim@ttic.edu" target="_blank"><b>Avrim Blum</b></a></font></div><div style="color:rgb(80,0,80);font-size:small"><br></div><div style="color:rgb(80,0,80);font-size:small"><br></div></div><div style="font-size:small;color:rgb(80,0,80)"><br></div></div><div><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><font face="arial, helvetica, sans-serif">Mary C. Marre</font><div><font face="arial, helvetica, sans-serif">Faculty Administrative Support</font></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6"><b>Toyota Technological Institute</b></font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">6045 S. Kenwood Avenue</font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Room 517</font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Chicago, IL 60637</font></i></div><div><i><font face="arial, helvetica, sans-serif">p:(773) 834-1757</font></i></div><div><i><font face="arial, helvetica, sans-serif">f: (773) 357-6970</font></i></div><div><b><i><a href="mailto:mmarre@ttic.edu" target="_blank"><font face="arial, helvetica, sans-serif">mmarre@ttic.edu</font></a></i></b></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div>
</blockquote></div></div>
</blockquote></div></div>