[Colloquium] Kevin Suk MS Presentation - May 2, 2024

[Jessica Garza]

This is an announcement of Kevin Suk's MS Presentation

===============================================
Candidate: Kevin Suk

Date: Thursday, May 2, 2024

Time:  3 pm CT

Location: JCL 298

Title: Probabilistic Contrastive Language-Image Pretraining

Abstract: CLIP utilizes cosine similarities to perform contrastive learning. Although it is efficient, the cosine only captures the angle between two vectors, enforcing all vectors onto an unit hypersphere. This could limit the expressive power of the model as it might not optimally capture the relationships between different modalities in the vector space. Moreover, the cosine similarity is symmetric, failing to discriminate between classification of images with their corresponding captions and classification of captions with images. Hence, we aim to improve performance by directly utilizing the conditional probabilities $\mathbb{P}(\text{Image}|\text{Text})$, $\mathbb{P}(\text{Text}|\text{Image})$ to form a new asymmetric contrastive loss. Testing on multiple new and old benchmarks, we show that under resource constrained training conditions, Probabilistic CLIP is able to provide better or at-par performance compared to CLIP with a higher confidence level.

Advisors: Greg Shakhnarovich

Committee Members: Greg Shakhnarovich, Michael Maire, Yuxin Chen


Jessica Garza
Assistant Director of Undergraduate Studies
Department of Computer Science
The University of Chicago
John Crerar Library 374
Office: (773) 702-2336

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/colloquium/attachments/20240419/03df2e48/attachment-0001.html>