[Theory] 9/24 Thesis Defense: Shengjie Lin, TTIC

[Mary Marre via Theory]

*When*:     Wednesday, September 24th from* 2:30 - 3:30p**m CT*

*Virtually*: via *Zoom*
<https://uchicagogroup.zoom.us/j/95222100221?pwd=Mdam0MtHKq34h0ipKA1bNx3DivTMww.1>


*Who:  *    Shengjie Lin, TTIC


*Title: *Scalable 3D Scene Understanding and Embodied Reasoning for Robotics
*Abstract: *For robots to become truly useful partners in our world, they
must first build a rich, actionable understanding of it. This thesis
introduces a complete framework to make that happen, enabling an agent to
perceive its environment, understand how objects function, and act on
complex human instructions.
The work first tackles perception at scale. We present a novel method for
building large, high-fidelity 3D maps by seamlessly stitching together
multiple, independently-captured scene representations. This provides the
robot with a foundational spatial awareness. Building on this static map,
we then address object dynamics. From just two sparse observations—like a
cabinet door open and closed—our system can reconstruct an articulated
object and infer its kinematics, creating a world model that is not just
descriptive, but functional.

With this rich world model in place, we use a Large Language Model (LLM) as
the robot's cognitive engine. This system empowers the agent to robustly
ground complex, free-form human commands within its 3D map. Crucially, it
also maintains an explicit "memory" of the world's state, allowing it to
track changes as it acts and successfully perform long-horizon, multi-step
tasks.

Together, these contributions form a complete pipeline from pixels to
actions, paving the way for more capable and collaborative robots that can
operate effectively in human-centric environments.

Thesis* Committee: *Chair: Matthew Walter <mwalter at ttic.edu>  Members:Greg
Shakhnarovich, Vitor Guizilini




Mary C. Marre
Faculty Administrative Support
*Toyota Technological Institute*
*6045 S. Kenwood Avenue, Rm 517*
*Chicago, IL  60637*
*773-834-1757*
*mmarre at ttic.edu <mmarre at ttic.edu>*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/theory/attachments/20250911/9245b42f/attachment.html>