[Colloquium] 5/22 Talks at TTIC: Yasaman Bahri, Google Brain

[Mary Marre via Colloquium]

 When:     Tuesday, May 22nd at *11:00 am*

Where:    TTIC, 6045 S Kenwood Avenue, 5th Floor, Room 526

Who:       Yasaman Bahri, Google Brain


Title:        Wide, Deep Neural Networks are Gaussian Processes

Abstract: One means of better understanding seemingly complicated models
such as deep neural networks is to connect them to other objects we already
understand. For instance, Gaussian processes are well-studied models with
well-controlled analytic properties. In his seminal work, Radford Neal
suggested thinking about inference in function space, rather than parameter
space, and in doing so established a correspondence between single-layer
fully-connected neural networks with an i.i.d prior over parameters and
certain Gaussian processes (GPs), in the limit of infinite network width.
The correspondence was, however, restricted to a single-hidden layer.

We develop this line of work and build an exact correspondence between
deep, infinitely wide neural networks and Gaussian processes.
Algorithmically, this mapping also enables a route towards Bayesian
inference with deep neural networks, without needing to instantiate a
network, which we implement on MNIST and CIFAR-10. We compare to the
performance of finite-width networks trained with standard stochastic
optimization. We find that performance increases as finite-width trained
networks are made wider and more similar to a GP, and thus that GP
predictions typically outperform those of finite-width networks.

Time permitting, I will also give some brief highlights of our related
work, studying the propagation of signals through random neural networks.
This analysis informs initializations for training ultra-deep networks with
tens of thousands of layers.

Links:
Jeffrey Pennington and Yasaman Bahri. “Geometry of Neural Network Loss
Surfaces via Random Matrix Theory.” ICML 2017. http://proceedings.mlr.p
ress/v70/pennington17a

Jaehoon Lee*, Yasaman Bahri*, Roman Novak, Samuel S. Schoenholz, Jeffrey
Pennington, Jascha Sohl-Dickstein. “Deep Neural Networks as Gaussian
Processes.” ICLR 2018. https://arxiv.org/abs/1711.00165.

Bio: Yasaman Bahri is a researcher at Google Brain working on deep
learning. The goal of her research is to advance a scientific, principled
understanding of deep learning, with an eye towards theoretical analysis
informed by careful empirical work. She got a PhD in Theoretical Condensed
Matter physics from UC Berkeley, specializing in many body physics; working
on symmetry-protected topological phases, many-body localization, non-Fermi
liquids, and topological mechanics. She is also interested in the
connections between condensed matter, theoretical physics and machine
learning.


Host: Mesrob Ohannessian <mesrob at ttic.edu>


Mary C. Marre
Administrative Assistant
*Toyota Technological Institute*
*6045 S. Kenwood Avenue*
*Room 504*
*Chicago, IL  60637*
*p:(773) 834-1757*
*f: (773) 357-6970*
*mmarre at ttic.edu <mmarre at ttic.edu>*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/colloquium/attachments/20180516/5a67d8a2/attachment-0001.html>