[Theory] REMINDER: 3/14 Talks at TTIC: Mark Yatskar, AI2

[Mary Marre]

When:     Thursday, March 14th at *11:00 am*

Where:    TTIC, 6045 S Kenwood Avenue, 5th Floor, Room 526

Who:      Mark Yatskar, AI2


*Title:*      Language as a Scaffold for Grounded Intelligence

*Abstract:*
Natural language can be used to construct rich, compositional descriptions
of the world, highlighting for example entities (nouns), events (verbs),
and the interactions between them (simple sentences). In this talk, I show
how compositional structure around verbs and nouns can be repurposed to
build computer vision systems that scale to recognize hundreds of thousands
of visual concepts in images. I introduce the task of situation
recognition, where the goal is to map an image to a language-inspired
structured representation of the main activity it depicts. The problem is
challenging because it requires recognition systems to identify not only
what entities are present, but also how they are participating within an
event (e.g. not only that there are scissors but they are they are being
used to cut). I also describe new deep learning models that better capture
compositionality in situation recognition and leverage the close connection
to language ‘to know what we don’t know’ and cheaply mine new training
data. Although these methods work well, I show that they have a tendency to
amplify underlying societal biases in the training data (including over
predicting stereotypical activities based on gender), and introduce a new
dual decomposition method that significantly reduces this amplification
without sacrificing classification accuracy. Finally, I propose new
directions for expanding what visual recognition systems can see and ways
to minimize the encoding of negative social biases in our learned models.

*Bio:*
Mark Yatskar is a post-doc at the Allen Institute for Artificial
Intelligence and recipient of their Young Investigator Award. His primary
research is in the intersection of natural language processing and computer
vision and fairness in machine learning.  He received his Ph.D. from the
University of Washington with Luke Zettlemoyer and Ali Farhadi, received
the EMNLP best paper award in 2017, and his work has been featured in Wired
and the New York Times.


Host: Karen Livescu <klivescu at ttic.edu>

Mary C. Marre
Administrative Assistant
*Toyota Technological Institute*
*6045 S. Kenwood Avenue*
*Room 517*
*Chicago, IL  60637*
*p:(773) 834-1757*
*f: (773) 357-6970*
*mmarre at ttic.edu <mmarre at ttic.edu>*


On Fri, Mar 8, 2019 at 3:57 PM Mary Marre <mmarre at ttic.edu> wrote:

> When:     Thursday, March 14th at *11:00 am*
>
> Where:    TTIC, 6045 S Kenwood Avenue, 5th Floor, Room 526
>
> Who:      Mark Yatskar, AI2
>
>
> *Title:*      Language as a Scaffold for Grounded Intelligence
>
> *Abstract:*
> Natural language can be used to construct rich, compositional descriptions
> of the world, highlighting for example entities (nouns), events (verbs),
> and the interactions between them (simple sentences). In this talk, I show
> how compositional structure around verbs and nouns can be repurposed to
> build computer vision systems that scale to recognize hundreds of thousands
> of visual concepts in images. I introduce the task of situation
> recognition, where the goal is to map an image to a language-inspired
> structured representation of the main activity it depicts. The problem is
> challenging because it requires recognition systems to identify not only
> what entities are present, but also how they are participating within an
> event (e.g. not only that there are scissors but they are they are being
> used to cut). I also describe new deep learning models that better capture
> compositionality in situation recognition and leverage the close connection
> to language ‘to know what we don’t know’ and cheaply mine new training
> data. Although these methods work well, I show that they have a tendency to
> amplify underlying societal biases in the training data (including over
> predicting stereotypical activities based on gender), and introduce a new
> dual decomposition method that significantly reduces this amplification
> without sacrificing classification accuracy. Finally, I propose new
> directions for expanding what visual recognition systems can see and ways
> to minimize the encoding of negative social biases in our learned models.
>
> *Bio:*
> Mark Yatskar is a post-doc at the Allen Institute for Artificial
> Intelligence and recipient of their Young Investigator Award. His primary
> research is in the intersection of natural language processing and computer
> vision and fairness in machine learning.  He received his Ph.D. from the
> University of Washington with Luke Zettlemoyer and Ali Farhadi, received
> the EMNLP best paper award in 2017, and his work has been featured in Wired
> and the New York Times.
>
>
> Host: Karen Livescu <klivescu at ttic.edu>
>
>
>
> Mary C. Marre
> Administrative Assistant
> *Toyota Technological Institute*
> *6045 S. Kenwood Avenue*
> *Room 517*
> *Chicago, IL  60637*
> *p:(773) 834-1757*
> *f: (773) 357-6970*
> *mmarre at ttic.edu <mmarre at ttic.edu>*
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/theory/attachments/20190313/4c2a2dd0/attachment.html>