[Colloquium] REMINDER: 2/2 Talks at TTIC: Yang Yuan, Cornell University

[Mary Marre via Colloquium]

 When:     Friday, February 2nd at *10:30 am*

Where:    TTIC, 6045 S Kenwood Avenue, 5th Floor, Room 526

Who:       Yang Yuan, Cornell University


Title: Provable and Practical Algorithms for Non-convex Problems in Machine
Learning
Abstract: Machine learning has become one of the most exciting research
areas in the world, with various applications. However, there exists a
noticeable gap between theory and practice. On one hand, simple algorithm
like stochastic gradient descent (SGD) works very well in practice, without
satisfactory theoretical explanations. On the other hand, the algorithms
from the theory community, although with solid guarantees, tend to be less
efficient compared with the techniques widely used in practice, which are
usually hand tuned or ad hoc based on intuition. In this talk, I would like
to discuss my effort to bridge theory and practice from two directions. The
first direction is “practice to theory”, i.e., to explain and analyze the
existing algorithms and empirical observations in machine learning. I will
first briefly talk about how SGD escapes saddle points, and then present a
two-phase convergence analysis of SGD for the two-layer neural network with
ReLU activation. The other direction is “theory to practice”, i.e., using
deep theory tools to obtain new, better and practical algorithms. Along
this direction, I will introduce our new algorithm Harmonica that uses
Fourier analysis and compressed sensing for tuning hyperparameters.
Harmonica supports parallel sampling and works well for tuning neural
networks with 30+ hyperparameters.


Host: Nathan Srebro <nati at ttic.edu>



Mary C. Marre
Administrative Assistant
*Toyota Technological Institute*
*6045 S. Kenwood Avenue*
*Room 504*
*Chicago, IL  60637*
*p:(773) 834-1757*
*f: (773) 357-6970*
*mmarre at ttic.edu <mmarre at ttic.edu>*

On Thu, Feb 1, 2018 at 2:40 PM, Mary Marre <mmarre at ttic.edu> wrote:

> When:     Friday, February 2nd at *10:30 am*
>
> Where:    TTIC, 6045 S Kenwood Avenue, 5th Floor, Room 526
>
> Who:       Yang Yuan, Cornell University
>
>
> Title: Provable and Practical Algorithms for Non-convex Problems in
> Machine Learning
> Abstract: Machine learning has become one of the most exciting research
> areas in the world, with various applications. However, there exists a
> noticeable gap between theory and practice. On one hand, simple algorithm
> like stochastic gradient descent (SGD) works very well in practice, without
> satisfactory theoretical explanations. On the other hand, the algorithms
> from the theory community, although with solid guarantees, tend to be less
> efficient compared with the techniques widely used in practice, which are
> usually hand tuned or ad hoc based on intuition. In this talk, I would like
> to discuss my effort to bridge theory and practice from two directions. The
> first direction is “practice to theory”, i.e., to explain and analyze the
> existing algorithms and empirical observations in machine learning. I will
> first briefly talk about how SGD escapes saddle points, and then present a
> two-phase convergence analysis of SGD for the two-layer neural network with
> ReLU activation. The other direction is “theory to practice”, i.e., using
> deep theory tools to obtain new, better and practical algorithms. Along
> this direction, I will introduce our new algorithm Harmonica that uses
> Fourier analysis and compressed sensing for tuning hyperparameters.
> Harmonica supports parallel sampling and works well for tuning neural
> networks with 30+ hyperparameters.
>
>
> Host: Nathan Srebro <nati at ttic.edu>
>
>
> Mary C. Marre
> Administrative Assistant
> *Toyota Technological Institute*
> *6045 S. Kenwood Avenue*
> *Room 504*
> *Chicago, IL  60637*
> *p:(773) 834-1757 <(773)%20834-1757>*
> *f: (773) 357-6970 <(773)%20357-6970>*
> *mmarre at ttic.edu <mmarre at ttic.edu>*
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/colloquium/attachments/20180202/fcbb8132/attachment.html>