[Theory] NOW: 4/30 Talks at TTIC: Dravyansh Sharma, TTIC/IDEAL
Mary Marre via Theory
theory at mailman.cs.uchicago.edu
Wed Apr 30 10:54:00 CDT 2025
*When:* Wednesday, April 30, 2025 at* 11:00** am CT *
*Where: *Talk will be given *live, in-person* at
TTIC, 6045 S. Kenwood Avenue
5th Floor, Room 530
*Virtually:* *livestream via panopto
<https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=62572752-fb5d-494b-bc0a-b2c80154063e>*
*Who: * Dravyansh Sharma, TTIC/IDEAL
*Title*: Provable tuning of deep learning model hyperparameters
*Abstract*: Modern machine learning algorithms, especially deep
learning-based techniques, typically involve careful hyperparameter tuning
to achieve the best performance. Despite the surge of intense interest in
practical techniques like Bayesian optimization and random search-based
approaches to automating this laborious and compute-intensive task, the
fundamental learning-theoretic complexity of tuning hyperparameters for
deep neural networks is poorly understood. Inspired by this glaring gap, we
initiate the formal study of hyperparameter tuning complexity in deep
learning under a powerful data-driven paradigm. A major difficulty is that
the utility function as a function of the hyperparameter is very volatile
and furthermore, it is given implicitly by an optimization problem over the
model parameters. To tackle this challenge, we employ subtle concepts from
differential/algebraic geometry and constrained optimization to show that
the learning-theoretic complexity of the corresponding family of utility
functions is bounded. We instantiate our results and provide sample
complexity bounds for concrete applications—tuning a hyperparameter that
interpolates neural activation functions and setting the kernel parameter
in graph neural networks.
The talk is based on joint work with Nina Balcan and Anh Nguyen.
*Bio*: Dravyansh (Dravy) Sharma is an IDEAL postdoctoral researcher, hosted
by Avrim Blum at TTIC and Aravindan Vijayaraghavan at Northwestern
University. He obtained his PhD at Carnegie Mellon University, advised by
Nina Balcan. His research interests include machine learning theory and
algorithms, with a focus on provable hyperparameter tuning, adversarial
robustness, and learning in the presence of rational agents. His work
develops principled techniques for tuning fundamental machine learning
algorithms to domain-specific data, including decision trees, linear
regression, graph-based learning and, most recently, deep networks. He has
published several papers at top ML venues, including NeurIPS, ICML, COLT,
JMLR, AISTATS, UAI and AAAI, has multiple papers awarded with Oral
presentations, won the Outstanding Student Paper Award at UAI 2024, and has
interned with Google Research and Microsoft Research.
*Host: **Avrim Blum* <avrim at ttic.edu>
Mary C. Marre
Faculty Administrative Support
*Toyota Technological Institute*
*6045 S. Kenwood Avenue, Rm 517*
*Chicago, IL 60637*
*773-834-1757*
*mmarre at ttic.edu <mmarre at ttic.edu>*
On Wed, Apr 30, 2025 at 10:00 AM Mary Marre <mmarre at ttic.edu> wrote:
> *When:* Wednesday, April 30, 2025 at* 11:00** am CT *
>
>
> *Where: *Talk will be given *live, in-person* at
>
> TTIC, 6045 S. Kenwood Avenue
>
> 5th Floor, Room 530
>
>
> *Virtually:* *livestream via panopto
> <https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=62572752-fb5d-494b-bc0a-b2c80154063e>*
>
>
>
> *Who: * Dravyansh Sharma, TTIC/IDEAL
>
>
>
> *Title*: Provable tuning of deep learning model hyperparameters
>
> *Abstract*: Modern machine learning algorithms, especially deep
> learning-based techniques, typically involve careful hyperparameter tuning
> to achieve the best performance. Despite the surge of intense interest in
> practical techniques like Bayesian optimization and random search-based
> approaches to automating this laborious and compute-intensive task, the
> fundamental learning-theoretic complexity of tuning hyperparameters for
> deep neural networks is poorly understood. Inspired by this glaring gap, we
> initiate the formal study of hyperparameter tuning complexity in deep
> learning under a powerful data-driven paradigm. A major difficulty is that
> the utility function as a function of the hyperparameter is very volatile
> and furthermore, it is given implicitly by an optimization problem over the
> model parameters. To tackle this challenge, we employ subtle concepts from
> differential/algebraic geometry and constrained optimization to show that
> the learning-theoretic complexity of the corresponding family of utility
> functions is bounded. We instantiate our results and provide sample
> complexity bounds for concrete applications—tuning a hyperparameter that
> interpolates neural activation functions and setting the kernel parameter
> in graph neural networks.
>
> The talk is based on joint work with Nina Balcan and Anh Nguyen.
>
> *Bio*: Dravyansh (Dravy) Sharma is an IDEAL postdoctoral researcher,
> hosted by Avrim Blum at TTIC and Aravindan Vijayaraghavan at Northwestern
> University. He obtained his PhD at Carnegie Mellon University, advised by
> Nina Balcan. His research interests include machine learning theory and
> algorithms, with a focus on provable hyperparameter tuning, adversarial
> robustness, and learning in the presence of rational agents. His work
> develops principled techniques for tuning fundamental machine learning
> algorithms to domain-specific data, including decision trees, linear
> regression, graph-based learning and, most recently, deep networks. He has
> published several papers at top ML venues, including NeurIPS, ICML, COLT,
> JMLR, AISTATS, UAI and AAAI, has multiple papers awarded with Oral
> presentations, won the Outstanding Student Paper Award at UAI 2024, and has
> interned with Google Research and Microsoft Research.
> *Host: **Avrim Blum* <avrim at ttic.edu>
>
>
>
>
>
> Mary C. Marre
> Faculty Administrative Support
> *Toyota Technological Institute*
> *6045 S. Kenwood Avenue, Rm 517*
> *Chicago, IL 60637*
> *773-834-1757*
> *mmarre at ttic.edu <mmarre at ttic.edu>*
>
>
> On Tue, Apr 29, 2025 at 3:18 PM Mary Marre <mmarre at ttic.edu> wrote:
>
>> *When:* Wednesday, April 30, 2025 at* 11:00** am CT *
>>
>>
>> *Where: *Talk will be given *live, in-person* at
>>
>> TTIC, 6045 S. Kenwood Avenue
>>
>> 5th Floor, Room 530
>>
>>
>> *Virtually:* *livestream via panopto
>> <https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=62572752-fb5d-494b-bc0a-b2c80154063e>*
>>
>>
>>
>> *Who: * Dravyansh Sharma, TTIC/IDEAL
>>
>>
>>
>> *Title*: Provable tuning of deep learning model hyperparameters
>>
>> *Abstract*: Modern machine learning algorithms, especially deep
>> learning-based techniques, typically involve careful hyperparameter tuning
>> to achieve the best performance. Despite the surge of intense interest in
>> practical techniques like Bayesian optimization and random search-based
>> approaches to automating this laborious and compute-intensive task, the
>> fundamental learning-theoretic complexity of tuning hyperparameters for
>> deep neural networks is poorly understood. Inspired by this glaring gap, we
>> initiate the formal study of hyperparameter tuning complexity in deep
>> learning under a powerful data-driven paradigm. A major difficulty is that
>> the utility function as a function of the hyperparameter is very volatile
>> and furthermore, it is given implicitly by an optimization problem over the
>> model parameters. To tackle this challenge, we employ subtle concepts from
>> differential/algebraic geometry and constrained optimization to show that
>> the learning-theoretic complexity of the corresponding family of utility
>> functions is bounded. We instantiate our results and provide sample
>> complexity bounds for concrete applications—tuning a hyperparameter that
>> interpolates neural activation functions and setting the kernel parameter
>> in graph neural networks.
>>
>> The talk is based on joint work with Nina Balcan and Anh Nguyen.
>>
>> *Bio*: Dravyansh (Dravy) Sharma is an IDEAL postdoctoral researcher,
>> hosted by Avrim Blum at TTIC and Aravindan Vijayaraghavan at Northwestern
>> University. He obtained his PhD at Carnegie Mellon University, advised by
>> Nina Balcan. His research interests include machine learning theory and
>> algorithms, with a focus on provable hyperparameter tuning, adversarial
>> robustness, and learning in the presence of rational agents. His work
>> develops principled techniques for tuning fundamental machine learning
>> algorithms to domain-specific data, including decision trees, linear
>> regression, graph-based learning and, most recently, deep networks. He has
>> published several papers at top ML venues, including NeurIPS, ICML, COLT,
>> JMLR, AISTATS, UAI and AAAI, has multiple papers awarded with Oral
>> presentations, won the Outstanding Student Paper Award at UAI 2024, and has
>> interned with Google Research and Microsoft Research.
>> *Host: **Avrim Blum* <avrim at ttic.edu>
>>
>>
>> Mary C. Marre
>> Faculty Administrative Support
>> *Toyota Technological Institute*
>> *6045 S. Kenwood Avenue, Rm 517*
>> *Chicago, IL 60637*
>> *773-834-1757*
>> *mmarre at ttic.edu <mmarre at ttic.edu>*
>>
>>
>> On Wed, Apr 23, 2025 at 4:31 PM Mary Marre <mmarre at ttic.edu> wrote:
>>
>>> *When:* Wednesday, April 30, 2025 at* 11:00** am CT *
>>>
>>>
>>> *Where: *Talk will be given *live, in-person* at
>>>
>>> TTIC, 6045 S. Kenwood Avenue
>>>
>>> 5th Floor, Room 530
>>>
>>>
>>> *Virtually:* *livestream via panopto
>>> <https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=62572752-fb5d-494b-bc0a-b2c80154063e>*
>>>
>>>
>>>
>>> *Who: * Dravyansh Sharma, TTIC/IDEAL
>>>
>>>
>>>
>>> *Title*: Provable tuning of deep learning model hyperparameters
>>>
>>> *Abstract*: Modern machine learning algorithms, especially deep
>>> learning-based techniques, typically involve careful hyperparameter tuning
>>> to achieve the best performance. Despite the surge of intense interest in
>>> practical techniques like Bayesian optimization and random search-based
>>> approaches to automating this laborious and compute-intensive task, the
>>> fundamental learning-theoretic complexity of tuning hyperparameters for
>>> deep neural networks is poorly understood. Inspired by this glaring gap, we
>>> initiate the formal study of hyperparameter tuning complexity in deep
>>> learning under a powerful data-driven paradigm. A major difficulty is that
>>> the utility function as a function of the hyperparameter is very volatile
>>> and furthermore, it is given implicitly by an optimization problem over the
>>> model parameters. To tackle this challenge, we employ subtle concepts from
>>> differential/algebraic geometry and constrained optimization to show that
>>> the learning-theoretic complexity of the corresponding family of utility
>>> functions is bounded. We instantiate our results and provide sample
>>> complexity bounds for concrete applications—tuning a hyperparameter that
>>> interpolates neural activation functions and setting the kernel parameter
>>> in graph neural networks.
>>>
>>> The talk is based on joint work with Nina Balcan and Anh Nguyen.
>>>
>>> *Bio*: Dravyansh (Dravy) Sharma is an IDEAL postdoctoral researcher,
>>> hosted by Avrim Blum at TTIC and Aravindan Vijayaraghavan at Northwestern
>>> University. He obtained his PhD at Carnegie Mellon University, advised by
>>> Nina Balcan. His research interests include machine learning theory and
>>> algorithms, with a focus on provable hyperparameter tuning, adversarial
>>> robustness, and learning in the presence of rational agents. His work
>>> develops principled techniques for tuning fundamental machine learning
>>> algorithms to domain-specific data, including decision trees, linear
>>> regression, graph-based learning and, most recently, deep networks. He has
>>> published several papers at top ML venues, including NeurIPS, ICML, COLT,
>>> JMLR, AISTATS, UAI and AAAI, has multiple papers awarded with Oral
>>> presentations, won the Outstanding Student Paper Award at UAI 2024, and has
>>> interned with Google Research and Microsoft Research.
>>> *Host: **Avrim Blum* <avrim at ttic.edu>
>>>
>>>
>>>
>>>
>>> Mary C. Marre
>>> Faculty Administrative Support
>>> *Toyota Technological Institute*
>>> *6045 S. Kenwood Avenue, Rm 517*
>>> *Chicago, IL 60637*
>>> *773-834-1757*
>>> *mmarre at ttic.edu <mmarre at ttic.edu>*
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/theory/attachments/20250430/6202e73b/attachment-0001.html>
More information about the Theory
mailing list