[Colloquium] Zhang/Dissertation Defense/Apr 30, 2018

[Margaret Jaffey via Colloquium]

       Department of Computer Science/The University of Chicago

                     *** Dissertation Defense ***


Candidate:  Liwen Zhang

Date:  Monday, April 30, 2018

Time:  3:00 PM

Place:  Ryerson 276

Title: Tropical Geometry, Neural Networks, and Low-Coherence Frames

Abstract:
This dissertation consists of three pieces of work. The first work
aims to set up the connection between tropical geometry and
feedforward neural network. We discovered that, mathematically, a
feedforward neural network equipped with rectified linear units (ReLU)
is a tropical rational function. This connection provides a new
approach to understand and analyze deep neural networks. Among other
things, we show that the decision boundary derived from an ReLU neural
network is contained by a tropical hypersurface of a tropical
polynomial in companion with the network. Moreover, we associate
functions represented by feedforward neural networks with polytopes
and show that a two layer network can be fully characterized by
zonotopes which also serve as the building blocks for deeper networks.
Also, the number of vertices on the polytopes provides an upper bound
on the number of linear regions of the function expressed by the
network. We show that this upper bound grows exponentially with the
number of layers but only polynomially with respect to number of
hidden nodes in each layer.

In the second work, we propose an attention model in continuous vector
space for content-based neural memory access. Our model represents
knowledge graph entities as low-dimensional vectors while expressing
context-dependent attention as a Gaussian scoring function over the
vector space. We apply such a model to perform tasks such as knowledge
graph completion and complex question answering. The proposed
attention model can handle both the propagation of the uncertainty
when following a series of relations and also the conjunction of
conditions in a natural way. On a dataset of soccer players who
participated in the FIFA World Cup 2014, we demonstrate that our model
can handle both path queries and conjunctive queries well.

The third work focus on building finite complex frames generated by
cyclic vectors under the action of non-commutative groups. We inspect
group frames in the space of operators associated with the group's von
Neumann algebra. The searching for a proper cyclic vector is then
transformed to finding the intersection of a convex set that
prescribes the coherence constraints and a subset of Hermitian
rank-one operators. An alternating projection algorithm is employed to
search for their intersection and an heuristic extrapolation technique
is adapted to accelerate the computation. In the experiments, we
applied our model to Heisenberg groups and finite affine groups. In
the case of Heisenberg group, our method is able to find cyclic
vectors that generate equiangular tight frames up to numerical
precisions.

Liwen's advisors are Prof. Risi Kondor and Prof. Lek-Heng Lim

Login to the Computer Science Department website for details,
including a draft copy of the dissertation:

 https://www.cs.uchicago.edu/phd/phd_announcements#liwenz

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Margaret P. Jaffey            margaret at cs.uchicago.edu
Department of Computer Science
Student Support Rep (Ry 156)               (773) 702-6011
The University of Chicago      http://www.cs.uchicago.edu
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=