[Theory] TIME CHANGE: 1/8 Talks at TTIC: Siddharth Karamcheti, Stanford University

[Mary Marre via Theory]

PLEASE NOTE: new talk time below

*When:*        Wednesday, January 8, 2025 at* 11:30** am** CT   *


*Where:       *Talk will be given *live, in-person* at

                   TTIC, 6045 S. Kenwood Avenue

                   5th Floor, Room 530


*Virtually:*   *via panopto: **livestream*
<https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=46925b13-5fd1-4f39-8654-b25a01327547>





*Who: *        Siddharth Karamcheti, Stanford University



*Title:         *Language-Driven Learning for Interactive Robotics

*Abstract*: Building and deploying broadly capable robots requires systems
that can efficiently learn from and work with people. To achieve this,
robots must balance *capability* — the skills necessary to enable
real-world deployment — and *sustainability* — the ability to grow and
adapt through human feedback. In this talk, I will motivate *language-driven
learning* to tackle these axes, providing robots with better abstractions
for perception, action, and human-robot interaction. First, towards
capability, I will present our approach for using language to learn
flexible visual representations that can be efficiently adapted for diverse
robotics tasks. Building on these ideas, I will then discuss and
investigate methods for training visually-conditioned language models and
vision-language-action policies at scale, harnessing pretrained language
models for robust and generalizable reasoning. Finally, towards
sustainability, I will introduce a new framework that integrates these
foundation models to develop collaborative robots that can work alongside
human partners, using language to express uncertainty and learn new
behaviors from real-time interactions. I will conclude with open challenges
for enhancing both the capability and sustainability of modern robots, with
directions for future work.

*Bio: *Siddharth Karamcheti is a final year PhD student at Stanford
University advised by Dorsa Sadigh and Percy Liang, and a robotics intern
at the Toyota Research Institute. His research focuses on robot learning,
natural language processing, and human-robot interaction, with a goal of
developing scalable approaches for human-robot collaboration. Prior to the
PhD, he earned his bachelor’s degree in Computer Science and Literary Arts
at Brown University, where he worked with Eugene Charniak and Stefanie
Tellex. He is a recipient of the Open Philanthropy AI Fellowship (2018), is
an RSS Pioneer (2024), and his research has won paper awards at conferences
such as RSS, CoRL, ICRA, and ACL.

*Host: **Matthew Walter* <mwalter at ttic.edu>



Mary C. Marre
Faculty Administrative Support
*Toyota Technological Institute*
*6045 S. Kenwood Avenue, Rm 517*
*Chicago, IL  60637*
*773-834-1757*
*mmarre at ttic.edu <mmarre at ttic.edu>*


On Fri, Jan 3, 2025 at 12:45 PM Mary Marre <mmarre at ttic.edu> wrote:

> *When:*        Wednesday, January 8, 2025 at* 11:00** am** CT   *
>
>
> *Where:       *Talk will be given *live, in-person* at
>
>                    TTIC, 6045 S. Kenwood Avenue
>
>                    5th Floor, Room 530
>
>
> *Virtually:*   *via panopto: **livestream*
> <https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=46925b13-5fd1-4f39-8654-b25a01327547>
>
>
>
>
>
> *Who: *        Siddharth Karamcheti, Stanford University
>
>
>
> *Title:         *Language-Driven Learning for Interactive Robotics
>
> *Abstract*: Building and deploying broadly capable robots requires
> systems that can efficiently learn from and work with people. To achieve
> this, robots must balance *capability* — the skills necessary to enable
> real-world deployment — and *sustainability* — the ability to grow and
> adapt through human feedback. In this talk, I will motivate *language-driven
> learning* to tackle these axes, providing robots with better abstractions
> for perception, action, and human-robot interaction. First, towards
> capability, I will present our approach for using language to learn
> flexible visual representations that can be efficiently adapted for diverse
> robotics tasks. Building on these ideas, I will then discuss and
> investigate methods for training visually-conditioned language models and
> vision-language-action policies at scale, harnessing pretrained language
> models for robust and generalizable reasoning. Finally, towards
> sustainability, I will introduce a new framework that integrates these
> foundation models to develop collaborative robots that can work alongside
> human partners, using language to express uncertainty and learn new
> behaviors from real-time interactions. I will conclude with open challenges
> for enhancing both the capability and sustainability of modern robots, with
> directions for future work.
>
> *Bio: *Siddharth Karamcheti is a final year PhD student at Stanford
> University advised by Dorsa Sadigh and Percy Liang, and a robotics intern
> at the Toyota Research Institute. His research focuses on robot learning,
> natural language processing, and human-robot interaction, with a goal of
> developing scalable approaches for human-robot collaboration. Prior to the
> PhD, he earned his bachelor’s degree in Computer Science and Literary Arts
> at Brown University, where he worked with Eugene Charniak and Stefanie
> Tellex. He is a recipient of the Open Philanthropy AI Fellowship (2018), is
> an RSS Pioneer (2024), and his research has won paper awards at conferences
> such as RSS, CoRL, ICRA, and ACL.
>
> *Host: **Matthew Walter* <mwalter at ttic.edu>
>
>
>
> Mary C. Marre
> Faculty Administrative Support
> *Toyota Technological Institute*
> *6045 S. Kenwood Avenue, Rm 517*
> *Chicago, IL  60637*
> *773-834-1757*
> *mmarre at ttic.edu <mmarre at ttic.edu>*
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/theory/attachments/20250103/54e8bb95/attachment.html>