[Theory] NOW: [TTIC Talks] 4/19 Research at TTIC: Theodor Misiakiewicz , TTIC

[Brandie Jones]

*When:*         April 19th* at 12:30pm CT  *


*Where:*        Talk will be given *live, in-person* at

                       TTIC, 6045 S. Kenwood Avenue

                        5th Floor, Room 530


*Virtually:*    via Panopto (Livestream
<https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=479b1837-8c4c-40b5-b6ef-b09a00fef8c7>
)



*Who:*           Theodor Misiakiewicz , TTIC



*Title*:            Deterministic equivalents for kernel regression


*Abstract:*   In this talk, we revisit kernel ridge regression (KRR) which
has attracted renewed interest recently due to its connections to deep
learning. A large body of work has sought to precisely characterize the
performance of KRR in toy settings, either through novel uniform
convergence bounds or by leveraging RMT in asymptotic regimes. In a
parallel—and mostly disjoint—line of work dating back 20 years, physicists
have derived closed-form estimates for the risk of KRR using statistical
physics heuristics.  These estimates show remarkably good agreement with
the learning curves obtained numerically for both synthetic and real data,
and complex kernels, including pre-trained neural networks.

In this talk, we prove that these deterministic estimates hold for a
general class of problems satisfying some spectral properties. Our proofs
rely on a careful derivation of deterministic equivalents for random matrix
functionals in the dimension-free regime pioneered by Cheng and Montanari
(2022).

If time permits, we will discuss the case of kernel methods with general
loss functions. Gaussian universality no longer holds. We show instead that
the model belongs to another universality class defined in terms of a
Gaussian model to which we append a low-dimensional non-gaussian part.

***********************************************************************************************

*Masks are optional in all common areas. **Full visitor guidance is
available at ttic.edu/visitors <http://ttic.edu/visitors>.*

***********************************************************************************************

*Research at TTIC Seminar Series*



TTIC is hosting a weekly seminar series presenting the research currently
underway at the Institute. Every week a different TTIC faculty member will
present their research.  The lectures are intended for students seeking
research topics and advisors and for the general TTIC and University of
Chicago communities interested in hearing what their colleagues are up to.

--
*Brandie Jones *
*Executive **Administrative Assistant*
Toyota Technological Institute
6045 S. Kenwood Avenue
Chicago, IL  60637
www.ttic.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/theory/attachments/20240419/174a6bc8/attachment.html>