[Theory] [TTIC Talks] 2/24 Talks at TTIC: Jamie Simon, UC Berkeley

[Brandie Jones]

*When: *Friday, January 24th, *10:30AM CT*

*Where: *Talk will be given *live, in-person* at

                   TTIC, 6045 S. Kenwood Avenue

                    5th Floor, Room 530


*Virtually: *via Panopto (Livestream
<https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=96b1e439-8b48-40c3-bd33-afac00e9cf35>
)

*Who: * Jamie Simon, UC Berkeley

*Title: *The Eigenlearning Framework: A Conservation Law Perspective on
Kernel Regression and Wide Neural Networks

*Abstract: *In a remarkable recent development spanning many works, the
generalization of kernel regression -- a classical algorithm equivalent to
wide neural nets -- has (in some sense) been solved. We now have a set of
simple, closed-form equations which accurately predict things like the
expected test risk of the algorithm in terms of the kernel eigenstructure
and the target function. In this talk I'll tell this story, with a focus on
our own contribution, which is to derive these equations in a simpler
manner (no replica calculation or random matrix theory needed, just linear
algebra!), give them a nice interpretation in terms of a conserved
quantity, and start to apply this powerful theory to knock down problems of
interest. Time permitting, I'll talk about how this theory lets us

   1. explain the "deep bootstrap" phenomenon of Nakkiran et al. (2020),
   2. give a new result on the hardness of the classic parity problem for
   rotation-invariant kernels,
   3. fashion a theoretical tool for the study of adversarial examples, and
   4. draw a tight analogy between kernel regression and the classic free
   Fermi gas from statistical physics.


*Bio: *Jamie is a 4th-year PhD student in the physics department at UC
Berkeley, aiming to use tools from theoretical physics to build a
fundamental understanding of deep neural networks. He is advised by Mike
DeWeese.

For more information about him and his work, please visit his personal
webpage. <https://james-simon.github.io/>



-- 
*Brandie Jones *
*Executive **Administrative Assistant*
Toyota Technological Institute
6045 S. Kenwood Avenue
Chicago, IL  60637
www.ttic.edu
Working Remotely on Tuesdays
Out of Office: Feb 9th - Feb 16th. Remote: Feb 17th - March 3rd.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/theory/attachments/20230220/c5b89c9c/attachment.html>