[Colloquium] REMINDER: 2/7 Talks at TTIC: Brian Bullins, Princeton University

[Mary Marre via Colloquium]

When:     Thursday, February 7th at *11:00 am*

Where:    TTIC, 6045 S Kenwood Avenue, 5th Floor, Room 526

Who:       Brian Bullins, Princeton University


*Title:*       Efficient Higher-Order Optimization for Machine Learning

*Abstract: *In recent years, stochastic gradient descent (SGD) has taken
center stage for training large-scale models in machine learning. Although
some higher-order methods have improved iteration complexity in theory, the
per-iteration costs render them unusable when faced with millions of
parameters and training examples.

In this talk, I will present several of my works which enable higher-order
optimization to be as scalable as first-order methods. The first method is
a stochastic second-order algorithm for convex optimization, called LiSSA,
which uses Hessian information to construct an unbiased Newton step in time
linear in the dimension. To bypass the typical efficiency barriers for
second-order methods, we harness the ERM structure in standard machine
learning tasks.

While convex problems allow for global convergence, recent state-of-the-art
models, such as deep neural networks, highlight the importance of
developing a better understanding of non-convex guarantees. In order to
handle this challenging setting, I will present FastCubic, a Hessian-based
method which provably converges to first-order critical points faster than
gradient descent, while additionally converging to second-order critical
points. Finally, I will show how our algorithm FastQuartic combines even
higher-order information with “highly smooth acceleration” to guarantee
even faster convergence for a large class of convex quartic problems.


*Host:* Nathan Srebro <nati at ttic.edu>



Mary C. Marre
Administrative Assistant
*Toyota Technological Institute*
*6045 S. Kenwood Avenue*
*Room 517*
*Chicago, IL  60637*
*p:(773) 834-1757*
*f: (773) 357-6970*
*mmarre at ttic.edu <mmarre at ttic.edu>*


On Fri, Feb 1, 2019 at 7:19 PM Mary Marre <mmarre at ttic.edu> wrote:

>
> When:     Thursday, February 7th at *11:00 am*
>
> Where:    TTIC, 6045 S Kenwood Avenue, 5th Floor, Room 526
>
> Who:       Brian Bullins, Princeton University
>
>
> *Title:* Efficient Higher-Order Optimization for Machine Learning
>
> *Abstract: *In recent years, stochastic gradient descent (SGD) has taken
> center stage for training large-scale models in machine learning. Although
> some higher-order methods have improved iteration complexity in theory, the
> per-iteration costs render them unusable when faced with millions of
> parameters and training examples.
>
> In this talk, I will present several of my works which enable higher-order
> optimization to be as scalable as first-order methods. The first method is
> a stochastic second-order algorithm for convex optimization, called LiSSA,
> which uses Hessian information to construct an unbiased Newton step in time
> linear in the dimension. To bypass the typical efficiency barriers for
> second-order methods, we harness the ERM structure in standard machine
> learning tasks.
>
> While convex problems allow for global convergence, recent
> state-of-the-art models, such as deep neural networks, highlight the
> importance of developing a better understanding of non-convex guarantees.
> In order to handle this challenging setting, I will present FastCubic, a
> Hessian-based method which provably converges to first-order critical
> points faster than gradient descent, while additionally converging to
> second-order critical points. Finally, I will show how our algorithm
> FastQuartic combines even higher-order information with “highly smooth
> acceleration” to guarantee even faster convergence for a large class of
> convex quartic problems.
>
>
> *Host:* Nathan Srebro <nati at ttic.edu>
>
>
>
>
> Mary C. Marre
> Administrative Assistant
> *Toyota Technological Institute*
> *6045 S. Kenwood Avenue*
> *Room 517*
> *Chicago, IL  60637*
> *p:(773) 834-1757*
> *f: (773) 357-6970*
> *mmarre at ttic.edu <mmarre at ttic.edu>*
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/colloquium/attachments/20190206/e97b2b12/attachment.html>