[Theory] REMINDER: 2/21 Talks at TTIC: Yuanzhi Li, Stanford University

[Mary Marre via Theory]

When:     Thursday, February 21st at *11:00 am*

Where:    TTIC, 6045 S Kenwood Avenue, 5th Floor, Room 526

Who:       Yuanzhi Li, Stanford University


*Title:       *Towards deeper understandings of deep learning


*Abstract:*

Learning through highly complicated and non-convex systems plays an
important rule in machine learning. Recently, a vast amount of empirical
works have demonstrated the success of these methods, especially in deep
learning. However, the formal study of the principles behind them is much
less developed.


This talk will cover a few recent results towards developing such
principles. Firstly, we focus on the principle of
``over-parameterization''. We show that for neural networks such as CNNs,
ResNet and RNNs, as long as enough over-parameterization is given,
algorithms such as stochastic gradient descent (SGD) provably finds the
global optimal on the training data set. Moreover, the solution also
generalizes to test data set as long as the training labels are realizable
by certain teacher networks.

The second result will cover the principle of ``noisy computation''. We
show how, for certain data sets, the neural network found by SGD with a
large learning rate (i.e. step size) at the begining follow by a learning
rate decay generalizes better than the one found by SGD with a small
learning rate, even when both case have the same training loss.


*Bio: *Yuanzhi Li is a postdoctoral researcher at the Computer Science
Department of Stanford University. Previously, he obtained his Ph.D. at
Princeton (2014-2018) under the advice of Sanjeev Arora. His research
interests include topics in deep learning, non-convex optimization,
algorithms, and online learning.


Host: Nathan Srebro <nati at ttic.edu>

Mary C. Marre
Administrative Assistant
*Toyota Technological Institute*
*6045 S. Kenwood Avenue*
*Room 517*
*Chicago, IL  60637*
*p:(773) 834-1757*
*f: (773) 357-6970*
*mmarre at ttic.edu <mmarre at ttic.edu>*


On Fri, Feb 15, 2019 at 4:23 PM Mary Marre <mmarre at ttic.edu> wrote:

> When:     Thursday, February 21st at *11:00 am*
>
> Where:    TTIC, 6045 S Kenwood Avenue, 5th Floor, Room 526
>
> Who:       Yuanzhi Li, Stanford University
>
>
> *Title:       *Towards deeper understandings of deep learning
>
>
> *Abstract:*
>
> Learning through highly complicated and non-convex systems plays an
> important rule in machine learning. Recently, a vast amount of empirical
> works have demonstrated the success of these methods, especially in deep
> learning. However, the formal study of the principles behind them is much
> less developed.
>
>
> This talk will cover a few recent results towards developing such
> principles. Firstly, we focus on the principle of
> ``over-parameterization''. We show that for neural networks such as CNNs,
> ResNet and RNNs, as long as enough over-parameterization is given,
> algorithms such as stochastic gradient descent (SGD) provably finds the
> global optimal on the training data set. Moreover, the solution also
> generalizes to test data set as long as the training labels are realizable
> by certain teacher networks.
>
> The second result will cover the principle of ``noisy computation''. We
> show how, for certain data sets, the neural network found by SGD with a
> large learning rate (i.e. step size) at the begining follow by a learning
> rate decay generalizes better than the one found by SGD with a small
> learning rate, even when both case have the same training loss.
>
>
> *Bio: *Yuanzhi Li is a postdoctoral researcher at the Computer Science
> Department of Stanford University. Previously, he obtained his Ph.D. at
> Princeton (2014-2018) under the advice of Sanjeev Arora. His research
> interests include topics in deep learning, non-convex optimization,
> algorithms, and online learning.
>
>
> Host: Nathan Srebro <nati at ttic.edu>
>
>
>
>
> Mary C. Marre
> Administrative Assistant
> *Toyota Technological Institute*
> *6045 S. Kenwood Avenue*
> *Room 517*
> *Chicago, IL  60637*
> *p:(773) 834-1757*
> *f: (773) 357-6970*
> *mmarre at ttic.edu <mmarre at ttic.edu>*
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/theory/attachments/20190220/934e9f23/attachment-0002.html>