[Theory] 3/7 Talks at TTIC: Guodong Zhang, University of Toronto

[Mary Marre]

*When:*        Monday, March 7th at* 11:30 am CT*


*Where:*       Zoom Virtual Talk (*register in advance here
<https://www.google.com/url?q=https://uchicagogroup.zoom.us/webinar/register/WN_CzxpKP4XTwySfk-iwHSeiw&sa=D&source=calendar&ust=1646520803994961&usg=AOvVaw0S8Uw1D6ATrcuea9EUnXl1>*
)


*Who: *         Guodong Zhang, University of Toronto


*Title:*          Scalable and Multiagent Deep Learning

*Abstract: *Deep learning has achieved huge successes over the last few
years, largely due to three important ideas: deep models with residual
connections, parallelism, and gradient-based learning. However, it was
shown that (1) deep ResNets behave like ensembles of shallow networks; (2)
naively increasing the scale of data parallelism leads to diminishing
return; (3) gradient-based learning could converge to spurious fixed points
in the multi-agent setting.

In this talk, I will present some of my works on understanding and
addressing these issues. First, I will give a general recipe for training
very deep neural networks without shortcuts. Second, I will present a noisy
quadratic model for neural network optimization, which qualitatively
predicts scaling properties of a variety of optimizers and in particular
suggests that second-order algorithms would benefit more from data
parallelism. Third, I will describe a novel algorithm that finds desired
equilibria and saves us from converging to spurious fixed points in
multi-agent games. In the end, I will conclude with future directions
towards building intelligent machines that can learn from experience
efficiently, reason about their own decisions, and act in our interests.


*Bio:* Guodong Zhang is a final-year PhD candidate at the University of
Toronto, advised by Roger Grosse. His research lies at the intersection
between machine learning, optimization, and Bayesian statistics. In
particular, his research focuses on understanding and developing algorithms
for optimization, Bayesian inference, and multi-agent games in the context
of deep learning. He has been recognized through the Apple PhD fellowship,
Borealis AI fellowship, and many other scholarships. In the past, he has
also interned at DeepMind, Google Brain, and Microsoft Research.


*Host:* *David McAllester <mcallester at ttic.edu>*


Mary C. Marre
Faculty Administrative Support
*Toyota Technological Institute*
*6045 S. Kenwood Avenue*
*Chicago, IL  60637*
*mmarre at ttic.edu <mmarre at ttic.edu>*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/theory/attachments/20220228/1b18d519/attachment-0001.html>