[Colloquium] 2/22 Talks at TTIC: Carina Silberer, University of Edinburgh

[Mary Marre]

When:    Monday, February 22nd at 11:00 am

Where:    TTIC, 6045 S Kenwood Avenue, 5th Floor, Room 526

Who:      Carina Silberer, University of Edinburgh


Title:       Using Visual and Linguistic Information to Connect Language to
the World

Abstract: There is a growing body of research in natural language
processing (NLP) and computer vision which uses cross-modal methods and
information to advance research within one area or to address cross-modal
tasks, such as image description generation. In this talk I will explore
the problem of grounding language in the visual world and address two
instantiations of this problem, semantic spaces and natural language image
descriptions. Central in this work is the question of how to best represent
the information provided by the two modalities.
I will first describe my work on integrating visual information from images
to address the classical NLP problem of inducing vector-based
representations of words from text corpora. My approach learns meaning
representations of words in a stacked autoencoder architecture which is
trained on information automatically extracted from text corpora and image
data.
I will then present my work on learning to give natural language
descriptions of the interactions between objects shown in real-world
images. The approach uses spatial and linguistic features from images and
text to determine the relevant objects depicted in an image. Exploiting
insights from parsing and structural representations, the method then
creates representations which capture the objects of an image and their
spatial relations, and uses statistical machine translation techniques to
transfer these spatial representations to natural language. I will discuss
different types of spatial representations and their influence on the
generation quality.


Host: Greg Shakhnarovich,  greg at ttic.edu



Mary C. Marre
Administrative Assistant
*Toyota Technological Institute*
*6045 S. Kenwood Avenue*
*Room 504*
*Chicago, IL  60637*
*p:(773) 834-1757*
*f: (773) 357-6970*
*mmarre at ttic.edu <mmarre at ttic.edu>*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.cs.uchicago.edu/pipermail/colloquium/attachments/20160216/1f6d5793/attachment.htm