[Colloquium] TTI-C Talk: Abhinav Gupta (University of Maryland, College Park)

[Julia MacGlashan]

REMINDER

When:             Wednesday, April 1st @ 11:00am (lunch will be provided
after talk)

Where:            6045 S Kenwood Ave, TTI-C Conference Room #526 (5th Floor)

Who:               Abhinav Gupta (University of Maryland, College Park)

Title:                Beyond Nouns and Verbs: Learning Visually Grounded
Stories of Images and Videos using Language and Vision


In this talk, I will present our recent work on exploring synergy between
language and vision for learning visually grounded contextual structures.
Our work departs from the traditional view to visual and semantic learning
where individual detectors and relationships are learned separately. Our
work focuses on simultaneous learning of visual appearance and contextual
models from richly annotated, weakly labeled datasets.

In the first part of the talk, I will show how rich annotations can be
utilized to constrain the learning of visually grounded models of nouns,
prepositions and comparative adjectives from weakly labeled data. I will
also show how visually grounded models of prepositions and comparative
adjectives can be utilized as contextual models for scene analysis.

In the second part, I will present storyline models for interpretation of
videos. Storyline models go beyond pair-wise contextual models and represent
higher order constraints by allowing only a few and finite number of
possible action sequences (stories). Visual inference using storyline models
involve inferring the "plot" of the video (sequence of actions) and
recognizing individual activities in the plot. I will also present an
iterative approach to learn visually grounded storyline models from video
and linguistic information provided in captions.

Contact:          Greg Shakhnarovich, TTI-C		greg at tti-c.org
834-2572




-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.cs.uchicago.edu/pipermail/colloquium/attachments/20090331/e3eb93ca/attachment.htm