[Theory] TODAY: [Talks at TTIC] 5/7 Young Researcher Seminar Series: Ekdeep Lubana, Harvard

[Brandie Jones via Theory]

*When:    *Wednesday, May 7th* at **11AM CT*

*Where:   *Talk will be given *live, in-person* at

                    TTIC, 6045 S. Kenwood Avenue

                    5th Floor, Room 530


*Virtually: *via Panopto (Livestream
<https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=c095d3ad-c487-4898-9ee1-b26e0117679e>
)

*Who:      *Ekdeep Lubana, Harvard

*Title:*      Dynamics of Concept Learning and Emergent Abilities in Neural
Networks

*Abstract: *Neural networks' scaling has been argued to yield sudden
learning of capabilities (a.k.a. emergent abilities). In this talk, I will
first summarize our recent work on formal models that help explain the
mechanisms underlying such sudden learning via data scaling, implicating
the compositional nature of a task and formation of structured
representations that are shared across several tasks involved in the
broader data composition.

Then, focusing on in-context learning (ICL)---one such suddenly learned
capability---I will demonstrate the precise configurations used for
training can lead to learning of fundamentally different algorithms for
performing an ICL task. This indicates the phenomenology of ICL established
in past work may not be universal.

Further, I will discuss how merely scaling the context size can lead to a
crossover between different ICL algorithms used by the model. This can be
explained via a competition of algorithms lens, which also yields a new
theory on the transient nature of ICL.

The talk will be based on the following series of papers:
https://arxiv.org/abs/2310.09336, https://arxiv.org/abs/2406.19370,
https://arxiv.org/abs/2408.12578, https://arxiv.org/abs/2410.08309,
https://arxiv.org/abs/2412.01003, https://arxiv.org/abs/2501.00070.

*Host: Nati Srebro <nati at ttic.edu>*

--
*Brandie Jones *
*Executive **Administrative Assistant*
Toyota Technological Institute
6045 S. Kenwood Avenue
Chicago, IL  60637
www.ttic.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/theory/attachments/20250507/9a35d0ed/attachment.html>