[Colloquium] Re: REMINDER: 11/11 Research at TTIC: Matthew Walter, TTIC

[Mary Marre via Colloquium]

When:     Friday, November 11th at noon

Where:    TTIC, 6045 S Kenwood Avenue, 5th Floor, Room 526

Who:       Matthew Walter; TTIC


Title:       Following Natural Language Instructions in Unknown Environments

Abstract:
Natural language promises an efficient and flexible means for humans to
communicate with robots, whether they are assisting the physically or
cognitively impaired, or performing disaster mitigation tasks as our
surrogates. Recent advancements have given rise to robots that are able to
interpret natural language commands that direct object manipulation and
spatial navigation. However, most methods require prior knowledge of the
metric and semantic properties of the objects and places that comprise the
robot's environment.

In this talk, I will present our work that enables robots to successfully
follow natural language navigation instructions within novel, unknown
environments. I will first describe a method that treats language as a
sensor, exploiting information implicit and explicit in the user's command
to learn distributions over the latent spatial and semantic properties of
the environment and over the robot's intended behavior. The method then
learns a belief space policy that reasons over these distributions to
identify suitable navigation actions. In the second part of the talk, I
will present an alternative formulation that represents language
understanding as a multi-view sequence-to-sequence learning problem. I will
introduce an alignment-based neural encoder-decoder architecture that
translates free-form instructions to action sequences based on images of
the observable world. Unlike previous methods, this architecture uses
no specialized linguistic resources and can be trained in a weakly
supervised, end-to-end fashion, which allows for generalization to
new domains. I will evaluate the efficacy of these methods on a combination
of benchmark navigation datasets and through demonstrations on a
voice-commandable wheelchair.


************************************************************
****************************************************

*Research at TTIC Seminar Series*

TTIC is hosting a weekly seminar series presenting the research currently
underway at the Institute. Every week a different TTIC faculty member will
present their research.  The lectures are intended both for students
seeking research topics and adviser, and for the general TTIC and
University of Chicago communities interested in hearing what their
colleagues are up to.

To receive announcements about the seminar series, please subscribe to the
mailing list: https://groups.google.com/a/ttic.edu/group/talks/subscribe

Speaker details can be found at: http://www.ttic.edu/tticseminar.php.

For additional questions, please contact Nathan Srebro at nati at ttic.edu
<mcallester at ttic.edu>



Mary C. Marre
Administrative Assistant
*Toyota Technological Institute*
*6045 S. Kenwood Avenue*
*Room 504*
*Chicago, IL  60637*
*p:(773) 834-1757*
*f: (773) 357-6970*
*mmarre at ttic.edu <mmarre at ttic.edu>*

On Thu, Nov 10, 2016 at 12:48 PM, Mary Marre <mmarre at ttic.edu> wrote:

> When:     Friday, November 11th at noon
>
> Where:    TTIC, 6045 S Kenwood Avenue, 5th Floor, Room 526
>
> Who:       Matthew Walter; TTIC
>
>
> Title:       Following Natural Language Instructions in Unknown
> Environments
>
> Abstract:
> Natural language promises an efficient and flexible means for humans to
> communicate with robots, whether they are assisting the physically or
> cognitively impaired, or performing disaster mitigation tasks as our
> surrogates. Recent advancements have given rise to robots that are able to
> interpret natural language commands that direct object manipulation and
> spatial navigation. However, most methods require prior knowledge of the
> metric and semantic properties of the objects and places that comprise the
> robot's environment.
>
> In this talk, I will present our work that enables robots to successfully
> follow natural language navigation instructions within novel, unknown
> environments. I will first describe a method that treats language as a
> sensor, exploiting information implicit and explicit in the user's command
> to learn distributions over the latent spatial and semantic properties of
> the environment and over the robot's intended behavior. The method then
> learns a belief space policy that reasons over these distributions to
> identify suitable navigation actions. In the second part of the talk, I
> will present an alternative formulation that represents language
> understanding as a multi-view sequence-to-sequence learning problem. I will
> introduce an alignment-based neural encoder-decoder architecture that
> translates free-form instructions to action sequences based on images of
> the observable world. Unlike previous methods, this architecture uses
> no specialized linguistic resources and can be trained in a weakly
> supervised, end-to-end fashion, which allows for generalization to
> new domains. I will evaluate the efficacy of these methods on a combination
> of benchmark navigation datasets and through demonstrations on a
> voice-commandable wheelchair.
>
>
> ************************************************************
> ****************************************************
>
> *Research at TTIC Seminar Series*
>
> TTIC is hosting a weekly seminar series presenting the research currently
> underway at the Institute. Every week a different TTIC faculty member will
> present their research.  The lectures are intended both for students
> seeking research topics and adviser, and for the general TTIC and
> University of Chicago communities interested in hearing what their
> colleagues are up to.
>
> To receive announcements about the seminar series, please subscribe to the
> mailing list: https://groups.google.com/a/ttic.edu/group/talks/subscribe
>
> Speaker details can be found at: http://www.ttic.edu/tticseminar.php.
>
> For additional questions, please contact Nathan Srebro at nati at ttic.edu
> <mcallester at ttic.edu>
>
>
>
>
> Mary C. Marre
> Administrative Assistant
> *Toyota Technological Institute*
> *6045 S. Kenwood Avenue*
> *Room 504*
> *Chicago, IL  60637*
> *p:(773) 834-1757 <%28773%29%20834-1757>*
> *f: (773) 357-6970 <%28773%29%20357-6970>*
> *mmarre at ttic.edu <mmarre at ttic.edu>*
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/colloquium/attachments/20161111/3c7f274f/attachment-0001.html>