[CS] Xiaoan Ding Candidacy Exam/April 9, 2021

[pbaclawski at uchicago.edu]

This is an announcement of Xiaoan Ding's Candidacy Exam.
===============================================
Date: Friday, April 9, 2021

Time: 3:00PM CST

Location: Via Zoom

https://uchicago.zoom.us/j/91393344794?pwd=OEdDYkVEOE9COEE2dEw5YjUrT2EyQT09
Meeting ID: 913 9334 4794
Passcode: 398561

Candidacy Candidate: Xiaoan Ding

Title: Latent-Variable Generative Modeling for Natural Language Processing

Abstract: Human excels at discovering patterns, extracting knowledge, and performing complex reasoning based on the data they observe. One natural motivation in building AI systems is to develop models and algorithms that can analyze, understand, reason, and learn from tremendous real-world raw data including images, text, graphs, videos. Generative models are one of the most promising approaches towards this goal.

In our work, we explore generative modeling on both classification and generation tasks in the context of NLP. For classification tasks, we show advantages of generative classifiers in terms of data efficiency and model robustness compared to discriminative counterparts. In particular, we introduce discrete latent variables into generative story which has shown to capture auxiliary information that could be helpful in classification. For generation tasks, we focus on learning and sampling from a continuous latent-variable model, i.e., the sentence-VAE model. In particular, we focus on variations that learn expressive prior distributions over the latent variable by adapting normalizing flow to the sentence-VAE. We add the importance-sampled log marginal likelihood as a second term to the standard VAE objective to help when learning the prior. Our results show a substantial improvement in language modeling and sampling.

Under the line of generative modeling, we propose two works: (1) scoring sentences with generative pretrained language model, in which we treat sentence score as energy acquired from downstream classification tasks; (2) improving model robustness to spurious correlation with pretrained generative classifiers.

Advisor: Kevin Gimpel and Janos Simon 

Committee Members: Kevin Gimpel, Janos Simon, Chenhao Tan, Sam Wiseman