[Theory] NOW: 3/7 Talks at TTIC: Guodong Zhang, University of Toronto

[Mary Marre]

*When:*        Monday, March 7th at* 11:30 am CT*


*Where:*       Zoom Virtual Talk (*register in advance here
<https://www.google.com/url?q=https://uchicagogroup.zoom.us/webinar/register/WN_CzxpKP4XTwySfk-iwHSeiw&sa=D&source=calendar&ust=1646520803994961&usg=AOvVaw0S8Uw1D6ATrcuea9EUnXl1>*
)


*Who: *         Guodong Zhang, University of Toronto


*Title:*          Scalable and Multiagent Deep Learning

*Abstract: *Deep learning has achieved huge successes over the last few
years, largely due to three important ideas: deep models with residual
connections, parallelism, and gradient-based learning. However, it was
shown that (1) deep ResNets behave like ensembles of shallow networks; (2)
naively increasing the scale of data parallelism leads to diminishing
return; (3) gradient-based learning could converge to spurious fixed points
in the multi-agent setting.

In this talk, I will present some of my works on understanding and
addressing these issues. First, I will give a general recipe for training
very deep neural networks without shortcuts. Second, I will present a noisy
quadratic model for neural network optimization, which qualitatively
predicts scaling properties of a variety of optimizers and in particular
suggests that second-order algorithms would benefit more from data
parallelism. Third, I will describe a novel algorithm that finds desired
equilibria and saves us from converging to spurious fixed points in
multi-agent games. In the end, I will conclude with future directions
towards building intelligent machines that can learn from experience
efficiently, reason about their own decisions, and act in our interests.


*Bio:* Guodong Zhang is a final-year PhD candidate at the University of
Toronto, advised by Roger Grosse. His research lies at the intersection
between machine learning, optimization, and Bayesian statistics. In
particular, his research focuses on understanding and developing algorithms
for optimization, Bayesian inference, and multi-agent games in the context
of deep learning. He has been recognized through the Apple PhD fellowship,
Borealis AI fellowship, and many other scholarships. In the past, he has
also interned at DeepMind, Google Brain, and Microsoft Research.


*Host:* *David McAllester <mcallester at ttic.edu>*


Mary C. Marre
Faculty Administrative Support
*Toyota Technological Institute*
*6045 S. Kenwood Avenue*
*Chicago, IL  60637*
*mmarre at ttic.edu <mmarre at ttic.edu>*


On Sun, Mar 6, 2022 at 2:00 PM Mary Marre <mmarre at ttic.edu> wrote:

> *When:*        Monday, March 7th at* 11:30 am CT*
>
>
> *Where:*       Zoom Virtual Talk (*register in advance here
> <https://www.google.com/url?q=https://uchicagogroup.zoom.us/webinar/register/WN_CzxpKP4XTwySfk-iwHSeiw&sa=D&source=calendar&ust=1646520803994961&usg=AOvVaw0S8Uw1D6ATrcuea9EUnXl1>*
> )
>
>
> *Who: *         Guodong Zhang, University of Toronto
>
>
> *Title:*          Scalable and Multiagent Deep Learning
>
> *Abstract: *Deep learning has achieved huge successes over the last few
> years, largely due to three important ideas: deep models with residual
> connections, parallelism, and gradient-based learning. However, it was
> shown that (1) deep ResNets behave like ensembles of shallow networks; (2)
> naively increasing the scale of data parallelism leads to diminishing
> return; (3) gradient-based learning could converge to spurious fixed
> points in the multi-agent setting.
>
> In this talk, I will present some of my works on understanding and
> addressing these issues. First, I will give a general recipe for training
> very deep neural networks without shortcuts. Second, I will present a noisy
> quadratic model for neural network optimization, which qualitatively
> predicts scaling properties of a variety of optimizers and in particular
> suggests that second-order algorithms would benefit more from data
> parallelism. Third, I will describe a novel algorithm that finds desired
> equilibria and saves us from converging to spurious fixed points in
> multi-agent games. In the end, I will conclude with future directions
> towards building intelligent machines that can learn from experience
> efficiently, reason about their own decisions, and act in our interests.
>
>
> *Bio:* Guodong Zhang is a final-year PhD candidate at the University of
> Toronto, advised by Roger Grosse. His research lies at the intersection
> between machine learning, optimization, and Bayesian statistics. In
> particular, his research focuses on understanding and developing algorithms
> for optimization, Bayesian inference, and multi-agent games in the context
> of deep learning. He has been recognized through the Apple PhD fellowship,
> Borealis AI fellowship, and many other scholarships. In the past, he has
> also interned at DeepMind, Google Brain, and Microsoft Research.
>
>
> *Host:* *David McAllester <mcallester at ttic.edu>*
>
>
>
>
> Mary C. Marre
> Faculty Administrative Support
> *Toyota Technological Institute*
> *6045 S. Kenwood Avenue*
> *Chicago, IL  60637*
> *mmarre at ttic.edu <mmarre at ttic.edu>*
>
>
> On Mon, Feb 28, 2022 at 5:01 PM Mary Marre <mmarre at ttic.edu> wrote:
>
>> *When:*        Monday, March 7th at* 11:30 am CT*
>>
>>
>> *Where:*       Zoom Virtual Talk (*register in advance here
>> <https://www.google.com/url?q=https://uchicagogroup.zoom.us/webinar/register/WN_CzxpKP4XTwySfk-iwHSeiw&sa=D&source=calendar&ust=1646520803994961&usg=AOvVaw0S8Uw1D6ATrcuea9EUnXl1>*
>> )
>>
>>
>> *Who: *         Guodong Zhang, University of Toronto
>>
>>
>> *Title:*          Scalable and Multiagent Deep Learning
>>
>> *Abstract: *Deep learning has achieved huge successes over the last few
>> years, largely due to three important ideas: deep models with residual
>> connections, parallelism, and gradient-based learning. However, it was
>> shown that (1) deep ResNets behave like ensembles of shallow networks; (2)
>> naively increasing the scale of data parallelism leads to diminishing
>> return; (3) gradient-based learning could converge to spurious fixed points
>> in the multi-agent setting.
>>
>> In this talk, I will present some of my works on understanding and
>> addressing these issues. First, I will give a general recipe for training
>> very deep neural networks without shortcuts. Second, I will present a noisy
>> quadratic model for neural network optimization, which qualitatively
>> predicts scaling properties of a variety of optimizers and in particular
>> suggests that second-order algorithms would benefit more from data
>> parallelism. Third, I will describe a novel algorithm that finds desired
>> equilibria and saves us from converging to spurious fixed points in
>> multi-agent games. In the end, I will conclude with future directions
>> towards building intelligent machines that can learn from experience
>> efficiently, reason about their own decisions, and act in our interests.
>>
>>
>> *Bio:* Guodong Zhang is a final-year PhD candidate at the University of
>> Toronto, advised by Roger Grosse. His research lies at the intersection
>> between machine learning, optimization, and Bayesian statistics. In
>> particular, his research focuses on understanding and developing algorithms
>> for optimization, Bayesian inference, and multi-agent games in the context
>> of deep learning. He has been recognized through the Apple PhD fellowship,
>> Borealis AI fellowship, and many other scholarships. In the past, he has
>> also interned at DeepMind, Google Brain, and Microsoft Research.
>>
>>
>> *Host:* *David McAllester <mcallester at ttic.edu>*
>>
>>
>> Mary C. Marre
>> Faculty Administrative Support
>> *Toyota Technological Institute*
>> *6045 S. Kenwood Avenue*
>> *Chicago, IL  60637*
>> *mmarre at ttic.edu <mmarre at ttic.edu>*
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/theory/attachments/20220307/960847a6/attachment-0001.html>