[Theory] REMINDER: [Talks at TTIC] 5/5 TTIC Colloquium:​ Mert Pilanci, Stanford University

[Brandie Jones via Theory]

*When:*        Monday, May 5th at *11:30am CT*

*Where:       *Talk will be given *live, in-person* at

                       TTIC, 6045 S. Kenwood Avenue

                       5th Floor, Room 530


*Virtually:*  via Panopto (livestream
<https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=ea4f88ec-182b-4a68-b4e6-b2ce01105d8d>
)


*Who: *         Mert Pilanci, Stanford University

*Title:*          Deep Networks as Convex Models: Exact Lasso Formulations
via Geometric Algebra
*Abstract:  *In this talk, we introduce an analysis of deep neural networks
through convex optimization and geometric (Clifford) algebra. We begin by
introducing exact convex optimization formulations for ReLU neural
networks. This approach demonstrates that deep networks can be globally
trained through convex programs, offering a globally optimal solution. Our
results further establish an equivalent characterization of neural networks
as high-dimensional convex Lasso models. These models employ a discrete set
of wedge product features and apply sparsity-inducing convex regularization
to fit data. This framework provides an intuitive geometric interpretation
where the optimal neurons represent signed volumes of parallelotopes formed
by data vectors.

Specifically, we show that the Lasso dictionary is constructed from a
discrete set of wedge products of input samples, with deeper network
architectures leading to geometric reflections of these features. This
analysis also reveals that standard convolutional neural networks can be
globally optimized in fully polynomial time. Numerical simulations validate
our claims, illustrating that the proposed convex approach is faster and
more reliable than standard local search heuristics, such as stochastic
gradient descent and its variants. We show a layerwise convex optimization
scheme whose performance is comparable to non-convex end-to-end
optimization. We also discuss extensions to batch normalization, generative
adversarial networks, transformers, and diffusion models.


*Host: Nati Srebro <nati at ttic.edu>*

-- 
*Brandie Jones *
*Executive **Administrative Assistant*
Toyota Technological Institute
6045 S. Kenwood Avenue
Chicago, IL  60637
www.ttic.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/theory/attachments/20250502/41e5b2bf/attachment.html>