<div dir="ltr"><div dir="ltr"><div dir="ltr"><div><div class="gmail_default" style="font-size:small"><font style="font-family:arial,sans-serif;color:rgb(0,0,0);vertical-align:inherit"><font style="vertical-align:inherit"><b>When:</b> </font></font><font style="vertical-align:inherit"><font style="vertical-align:inherit"><font style="font-family:arial,sans-serif;color:rgb(0,0,0)"> Wednes</font><span class="gmail_default" style="font-family:arial,sans-serif;color:rgb(0,0,0)">day, April 30,<span class="gmail_default"> </span>2025</span><font style="font-family:arial,sans-serif;color:rgb(0,0,0)"> at</font><b><font color="#000000" style="font-family:arial,sans-serif"> <u style="background-color:rgb(255,255,0)">11:00</u></font></b><font color="#000000"><u><b><font face="arial, sans-serif"><span style="background-color:rgb(255,255,0)"> am</span></font></b><b style="background-color:rgb(255,255,0)"><font face="arial, sans-serif"> CT </font></b></u></font></font></font></div><div><div class="gmail_default"><div class="gmail_default"><div class="gmail_default"><p style="color:rgb(80,0,80);font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><b><font color="#500050" face="arial, sans-serif"><br></font></b></p><p style="color:rgb(80,0,80);font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><font face="arial, sans-serif"><b><font color="#500050">Where: </font></b><font color="#000000">Talk will be given </font><font color="#000000" style="font-weight:bold"><u>live, in-person</u></font><font style="font-weight:bold"> </font>at<br></font></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font color="#500050"> </font><font color="#000000"> TTIC, 6045 S. Kenwood Avenue</font></font></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> 5th Floor, Room 530<b> </b></font></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><b><span style="color:black"><br></span></b></font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><b style="font-family:arial,sans-serif;color:rgb(60,64,67);letter-spacing:0.2px">Virtually:</b><span style="font-family:arial,sans-serif;color:rgb(60,64,67);letter-spacing:0.2px"> </span><span style="letter-spacing:0.2px"><font color="#0000ff" face="tahoma, sans-serif"><b> </b></font></span><span style="font-family:arial,sans-serif;letter-spacing:0.2px"><font color="#3c4043"> </font></span><i style="font-family:arial,sans-serif;letter-spacing:0.2px"><font color="#000000">livestream via </font><b style="color:rgb(0,0,255)"><a href="https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=62572752-fb5d-494b-bc0a-b2c80154063e" target="_blank">panopto</a></b></i></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><span style="letter-spacing:0.2px"><b style="font-style:italic;font-family:arial,sans-serif;color:rgb(0,0,255)"> </b></span><b style="letter-spacing:0.2px"><font size="1" face="tahoma, sans-serif"> </font></b><b style="color:rgb(60,64,67);letter-spacing:0.2px"><font face="arial, sans-serif"> </font></b></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font style="color:rgb(80,0,80);vertical-align:inherit"><font style="vertical-align:inherit"><b>Who: </b> <font color="#500050"> </font><font color="#000000"><font color="#500050"> </font></font></font></font></font><span>Dravyansh</span> Sharma, TTIC/IDEAL</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><br></p><div style="border-top:none;border-right:none;border-left:none;border-bottom:2.25pt solid rgb(11,118,159);padding:0in 0in 1pt"></div><div><font face="arial, sans-serif"><br></font></div></div></div><div class="gmail_default"><div dir="ltr"><p dir="ltr" id="m_-45798768406947022m_6945106302658679979m_-8079628325606074636m_722219328535240114m_-6334697159743454485m_4637356163276351851m_-2834646526354896859m_-3504761229940322681m_4938701260463822682m_5860868498644129419m_-8505693634201348731m_-8109962251780210301m_2767765040961654113m_4331931867628220036m_-4442932724059632450gmail-docs-internal-guid-5f578bda-7fff-41e8-2cfc-0aaba7b02842" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><font face="arial, sans-serif"><b style="color:rgb(31,31,31)">Title</b><span style="color:rgb(31,31,31)">: Provable tuning of deep learning model hyperparameters</span></font></p><p style="color:rgb(31,31,31)"><font face="arial, sans-serif"><b>Abstract</b>: Modern machine learning algorithms, especially deep learning-based techniques, typically involve careful hyperparameter tuning to achieve the best performance. Despite the surge of intense interest in practical techniques like Bayesian optimization and random search-based approaches to automating this laborious and compute-intensive task, the fundamental learning-theoretic complexity of tuning hyperparameters for deep neural networks is poorly understood. Inspired by this glaring gap, we initiate the formal study of hyperparameter tuning complexity in deep learning under a powerful data-driven paradigm. A major difficulty is that the utility function as a function of the hyperparameter is very volatile and furthermore, it is given implicitly by an optimization problem over the model parameters. To tackle this challenge, we employ subtle concepts from differential/algebraic geometry and constrained optimization to show that the learning-theoretic complexity of the corresponding family of utility functions is bounded. We instantiate our results and provide sample complexity bounds for concrete applications—tuning a hyperparameter that interpolates neural activation functions and setting the kernel parameter in graph neural networks.</font></p><p style="color:rgb(31,31,31)"><font face="arial, sans-serif">The talk is based on joint work with Nina Balcan and Anh Nguyen.</font></p><p style="color:rgb(31,31,31)"><font face="arial, sans-serif"><b>Bio</b>: <span>Dravyansh</span> (Dravy) Sharma is an IDEAL postdoctoral researcher, hosted by Avrim Blum at TTIC and Aravindan Vijayaraghavan at Northwestern University. He obtained his PhD at Carnegie Mellon University, advised by Nina Balcan. His research interests include machine learning theory and algorithms, with a focus on provable hyperparameter tuning, adversarial robustness, and learning in the presence of rational agents. His work develops principled techniques for tuning fundamental machine learning algorithms to domain-specific data, including decision trees, linear regression, graph-based learning and, most recently, deep networks. He has published several papers at top ML venues, including NeurIPS, ICML, COLT, JMLR, AISTATS, UAI and AAAI, has multiple papers awarded with Oral presentations, won the Outstanding Student Paper Award at UAI 2024, and has interned with Google Research and Microsoft Research.</font></p></div></div></div></div><div class="gmail_default"><font face="arial, sans-serif"><b style="color:rgb(31,31,31)">Host: </b><span style="color:rgb(31,31,31)"><a href="mailto:avrim@ttic.edu" target="_blank"><b>Avrim Blum</b></a></span></font></div><br><br clear="all"></div><div><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><span style="font-family:arial,helvetica,sans-serif;font-size:x-small">Mary C. Marre</span><br></div><div><div><font face="arial, helvetica, sans-serif" size="1">Faculty Administrative Support</font></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6" size="1"><b>Toyota Technological Institute</b></font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6" size="1">6045 S. Kenwood Avenue, Rm 517</font></i></div><div><font size="1"><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Chicago, IL 60637</font></i><br></font></div><div><font size="1"><i><font face="arial, helvetica, sans-serif" color="#3d85c6">773-834-1757</font></i></font></div><div><b><i><a href="mailto:mmarre@ttic.edu" target="_blank"><font face="arial, helvetica, sans-serif" size="1">mmarre@ttic.edu</font></a></i></b></div></div></div></div></div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Apr 23, 2025 at 4:31 PM Mary Marre <<a href="mailto:mmarre@ttic.edu" target="_blank">mmarre@ttic.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><div style="font-size:small"><font style="font-family:arial,sans-serif;color:rgb(0,0,0);vertical-align:inherit"><font style="vertical-align:inherit"><b>When:</b> </font></font><font style="vertical-align:inherit"><font style="vertical-align:inherit"><font style="font-family:arial,sans-serif;color:rgb(0,0,0)"> Wednes</font><span class="gmail_default" style="font-family:arial,sans-serif;color:rgb(0,0,0)">day, April 30,<span class="gmail_default"> </span>2025</span><font style="font-family:arial,sans-serif;color:rgb(0,0,0)"> at</font><b><font color="#000000" style="font-family:arial,sans-serif"> <u style="background-color:rgb(255,255,0)">11:00</u></font></b><font color="#000000"><u><b><font face="arial, sans-serif"><span style="background-color:rgb(255,255,0)"> am</span></font></b><b style="background-color:rgb(255,255,0)"><font face="arial, sans-serif"> CT </font></b></u></font></font></font></div><div><div><div><div><p style="color:rgb(80,0,80);font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><b><font color="#500050" face="arial, sans-serif"><br></font></b></p><p style="color:rgb(80,0,80);font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><font face="arial, sans-serif"><b><font color="#500050">Where: </font></b><font color="#000000"><span>Talk</span> will be given </font><font color="#000000" style="font-weight:bold"><u>live, in-person</u></font><font style="font-weight:bold"> </font>at<br></font></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font color="#500050"> </font><font color="#000000"> <span>TTIC</span>, 6045 S. Kenwood Avenue</font></font></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> 5th Floor, Room 530<b> </b></font></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><b><span style="color:black"><br></span></b></font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><b style="font-family:arial,sans-serif;color:rgb(60,64,67);letter-spacing:0.2px">Virtually:</b><span style="font-family:arial,sans-serif;color:rgb(60,64,67);letter-spacing:0.2px"> </span><span style="letter-spacing:0.2px"><font color="#0000ff" face="tahoma, sans-serif"><b> </b></font></span><span style="font-family:arial,sans-serif;letter-spacing:0.2px"><font color="#3c4043"> </font></span><i style="font-family:arial,sans-serif;letter-spacing:0.2px"><font color="#000000">livestream via </font><b style="color:rgb(0,0,255)"><a href="https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=62572752-fb5d-494b-bc0a-b2c80154063e" target="_blank">panopto</a></b></i></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><span style="letter-spacing:0.2px"><b style="font-style:italic;font-family:arial,sans-serif;color:rgb(0,0,255)"> </b></span><b style="letter-spacing:0.2px"><font size="1" face="tahoma, sans-serif"> </font></b><b style="color:rgb(60,64,67);letter-spacing:0.2px"><font face="arial, sans-serif"> </font></b></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font style="color:rgb(80,0,80);vertical-align:inherit"><font style="vertical-align:inherit"><b>Who: </b> <font color="#500050"> </font><font color="#000000"><font color="#500050"> </font></font></font></font></font>Dravyansh Sharma, TTIC/IDEAL</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><br></p><div style="border-top:none;border-right:none;border-left:none;border-bottom:2.25pt solid rgb(11,118,159);padding:0in 0in 1pt"></div><div><font face="arial, sans-serif"><br></font></div></div></div><div><div dir="ltr"><p dir="ltr" id="m_-45798768406947022m_6945106302658679979m_-8079628325606074636m_722219328535240114m_-6334697159743454485m_4637356163276351851m_-2834646526354896859m_-3504761229940322681m_4938701260463822682m_5860868498644129419m_-8505693634201348731m_-8109962251780210301m_2767765040961654113m_4331931867628220036m_-4442932724059632450gmail-docs-internal-guid-5f578bda-7fff-41e8-2cfc-0aaba7b02842" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><font face="arial, sans-serif" style="background-color:rgb(255,255,255)"><b style="color:rgb(31,31,31)">Title</b><span style="color:rgb(31,31,31)">: Provable tuning of deep learning model hyperparameters</span></font></p><p style="color:rgb(31,31,31)"><font face="arial, sans-serif" style="background-color:rgb(255,255,255)"><b>Abstract</b>: Modern machine learning algorithms, especially deep learning-based techniques, typically involve careful hyperparameter tuning to achieve the best performance. Despite the surge of intense interest in practical techniques like Bayesian optimization and random search-based approaches to automating this laborious and compute-intensive task, the fundamental learning-theoretic complexity of tuning hyperparameters for deep neural networks is poorly understood. Inspired by this glaring gap, we initiate the formal study of hyperparameter tuning complexity in deep learning under a powerful data-driven paradigm. A major difficulty is that the utility function as a function of the hyperparameter is very volatile and furthermore, it is given implicitly by an optimization problem over the model parameters. To tackle this challenge, we employ subtle concepts from differential/algebraic geometry and constrained optimization to show that the learning-theoretic complexity of the corresponding family of utility functions is bounded. We instantiate our results and provide sample complexity bounds for concrete applications—tuning a hyperparameter that interpolates neural activation functions and setting the kernel parameter in graph neural networks.</font></p><p style="color:rgb(31,31,31)"><font face="arial, sans-serif" style="background-color:rgb(255,255,255)">The talk is based on joint work with Nina Balcan and Anh Nguyen.</font></p><p style="color:rgb(31,31,31)"><font face="arial, sans-serif" style="background-color:rgb(255,255,255)"><b>Bio</b>: Dravyansh (Dravy) Sharma is an IDEAL postdoctoral researcher, hosted by Avrim Blum at TTIC and Aravindan Vijayaraghavan at Northwestern University. He obtained his PhD at Carnegie Mellon University, advised by Nina Balcan. His research interests include machine learning theory and algorithms, with a focus on provable hyperparameter tuning, adversarial robustness, and learning in the presence of rational agents. His work develops principled techniques for tuning fundamental machine learning algorithms to domain-specific data, including decision trees, linear regression, graph-based learning and, most recently, deep networks. He has published several papers at top ML venues, including NeurIPS, ICML, COLT, JMLR, AISTATS, UAI and AAAI, has multiple papers awarded with Oral presentations, won the Outstanding Student Paper Award at UAI 2024, and has interned with Google Research and Microsoft Research.</font></p></div></div></div></div><div><font face="arial, sans-serif" style="background-color:rgb(255,255,255)"><b style="color:rgb(31,31,31)">Host: </b><span style="color:rgb(31,31,31)"><a href="mailto:avrim@ttic.edu" target="_blank"><b>Avrim Blum</b></a></span></font></div><div><font face="arial, sans-serif" style="background-color:rgb(255,255,255)"><br></font></div><div><font face="arial, sans-serif" style="background-color:rgb(255,255,255)"><br></font></div><br><br clear="all"></div><div><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><span style="font-family:arial,helvetica,sans-serif;font-size:x-small">Mary C. Marre</span><br></div><div><div><font face="arial, helvetica, sans-serif" size="1">Faculty Administrative Support</font></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6" size="1"><b>Toyota Technological Institute</b></font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6" size="1">6045 S. Kenwood Avenue, Rm 517</font></i></div><div><font size="1"><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Chicago, IL 60637</font></i><br></font></div><div><font size="1"><i><font face="arial, helvetica, sans-serif" color="#3d85c6">773-834-1757</font></i></font></div><div><b><i><a href="mailto:mmarre@ttic.edu" target="_blank"><font face="arial, helvetica, sans-serif" size="1">mmarre@ttic.edu</font></a></i></b></div></div></div></div></div></div>
</div>
</div>
</blockquote></div></div>
</div>