[Theory] NOW: [Talks at TTIC] 10/22 Young Researcher Seminar Series: ​Keyon Vafa, Harvard University

[Brandie Jones via Theory]

*When:*        Wednesday, October 22nd at *11am CT*

*Where:       *Talk will be given *live, in-person* at

                       TTIC, 6045 S. Kenwood Avenue

                       5th Floor, Room 530


*Virtually:*  via Panopto (livestream
<https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=fa7cab9f-5be7-40c1-b739-b34d0127a3af>
)


*Who: *         Keyon Vafa, Harvard University

*Title:*          Evaluating the Implicit World Models of Generative Models
*Abstract:  *The challenge of evaluation is making conclusions about a
model's capabilities from a small amount of data. While there are many
benchmarks that allow us to quantify a model's performance on different
types of tasks, it is unclear how to turn these results into robust
conclusions about a model's understanding or its capabilities. This talk
will propose theoretically-grounded definitions and metrics that test for a
model's implicit understanding, or its world model. We will focus on two
settings: one where models are designed to perform a single task, and
another where a foundation model is intended to perform many tasks. These
exercises demonstrate that models can make highly accurate predictions with
incoherent world models, revealing their fragility.

*Bio*: Keyon Vafa is a postdoctoral fellow at Harvard University. His
research focuses on developing new evaluation methodology in order to
evaluate and improve generative models in AI. Keyon completed his PhD in
computer science from Columbia University, where he was an NSF GRFP Fellow
and the recipient of the Morton B. Friedman Memorial Prize for excellence
in engineering. He organized the NeurIPS 2024 Workshop on Behavioral
Machine Learning and the ICML 2025 Workshop on Assessing World Models, and
he is a member of the early career board of the Harvard Data Science Review.

*Host: Shiry Ginosar <shiry at ttic.edu>*


*Brandie Jones *
*Executive **Administrative Assistant*
*Outreach Administrator *
Toyota Technological Institute
6045 S. Kenwood Avenue
Chicago, IL  60637
www.ttic.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/theory/attachments/20251022/12076c0e/attachment.html>