[Theory] 2/20 Talks at TTIC: Yuntian Deng, Harvard University

[Mary Marre]

*When:*        Monday, February 20, 2023 at* 11:30** a**m CT   *


*Where:       *Talk will be given *live, in-person* at

                   TTIC, 6045 S. Kenwood Avenue

                   5th Floor, Room 530


*Virtually:*  *via* Panopto (*livestream
<https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=5b6b593d-06dd-4b63-b3a6-afa8017c3c02>*
)


*Who: *         Yuntian Deng, Harvard University


------------------------------

*Title:*          Structural Coherence in Text Generation



*Abstract:* The field of text generation has seen significant progress in
recent years. We are approaching a future where ubiquitous text generation
technologies will allow us to generate long-form texts that are not only
fluent at a surface level, but also coherent in their overall structure. To
enable this future, my research focuses on evaluating and improving
structure modeling in language models.

In the first part of this talk, I will introduce a method for quantifying
structural coherence in language models. This method extracts structures by
projecting data into a latent space of interest and then compares the
structures in model generations to human-written text. This quantitative
measure of structural coherence enables us to identify structural issues in
language models and reveals that structural coherence does not fully
correlate with surface fluency.

In the second part of the talk, I will present my research on improving
structure modeling in language models. I will introduce a global model that
scores the overall structure of text, in addition to the traditional
language model that scores text by scoring each local word. The traditional
language model excels at surface-level modeling, while the introduced
global model specializes in structure modeling. I will demonstrate that the
proposed model has a simple training and sampling procedure and leads to
improvements in both local fluency and structural coherence.

To conclude, I will outline my future plans to extend my research into
different types of sequence modeling problems that can benefit from
structure modeling.



*Bio:* Yuntian Deng is a PhD student at Harvard University, advised by
Professors Alexander Rush and Stuart Shieber. His research focuses on
developing long-form text generation methods that are coherent,
transparent, and efficient. He is also a key contributor to several
open-source projects, including OpenNMT, image-to-LaTeX, and LaTeX-to-image.

Yuntian is the recipient of an Nvidia Fellowship, a Baidu Fellowship, and
multiple awards for his research, including the University of Chicago
Rising Stars in Data Science, the ACL 2017 Best Demo Paper Runner-Up, the
ACM Gordon Bell Special Prize for Covid Research, the Impact Award from
Argonne National Lab, and the DAC 2020 Best Paper.

*Host:* Karen Livescu <klivescu at ttic.edu>



Mary C. Marre
Faculty Administrative Support
*Toyota Technological Institute*
*6045 S. Kenwood Avenue, Rm 517*
*Chicago, IL  60637*
*773-834-1757*
*mmarre at ttic.edu <mmarre at ttic.edu>*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/theory/attachments/20230213/941c3e9e/attachment.html>