<div dir="ltr"><div dir="ltr"><div class="gmail_default" style="font-size:small"><div class="gmail_default"><font face="arial, sans-serif"><font style="color:rgb(0,0,0);vertical-align:inherit"><font style="vertical-align:inherit"><b>When:</b> </font></font><font style="vertical-align:inherit"><font style="vertical-align:inherit"><font style="color:rgb(0,0,0)"> Monday</font><span class="gmail_default" style="color:rgb(0,0,0)">, February 20, 2023</span><font style="color:rgb(0,0,0)"> at</font><b style="color:rgb(0,0,0)"> <u>11:30</u></b><b><u><font color="#000000"> a</font></u></b><b><u><font color="#000000">m CT</font></u><font color="#000000"> </font></b></font></font><br></font></div><span style="color:rgb(80,0,80)"><div class="gmail_default"><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><font face="arial, sans-serif" color="#000000"><font style="vertical-align:inherit"><font style="vertical-align:inherit"><b><span style="background-color:rgb(255,255,0)"><br></span></b></font></font></font></p><div class="gmail_default"><font face="arial, sans-serif"><b><font color="#500050">Where: </font></b><font color="#000000">Talk will be given </font><font color="#000000" style="font-weight:bold"><u>live, in-person</u></font><font style="font-weight:bold"> </font>at</font></div><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font color="#500050"> </font><font color="#000000"> TTIC, 6045 S. Kenwood Avenue</font></font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> 5th Floor, Room 530<b> </b></font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><b><span style="color:black"><br></span></b></font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><b style="color:rgb(60,64,67);letter-spacing:0.2px;white-space:pre-wrap">Virtually:</b><span style="color:rgb(60,64,67);letter-spacing:0.2px;white-space:pre-wrap"> <i>via</i> Panopto </span>(<b><a href="https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=5b6b593d-06dd-4b63-b3a6-afa8017c3c02" target="_blank">livestream</a></b><span style="color:rgb(60,64,67);letter-spacing:0.2px;white-space:pre-wrap">)</span><br></font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><br></font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font style="vertical-align:inherit"><font style="vertical-align:inherit"><b>Who: </b> <font color="#500050"> </font><font color="#000000"><font color="#500050"> </font></font></font></font></font><span class="gmail-il">Yuntian</span> <span class="gmail-il">Deng</span>, Harvard University</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><br></font></p><div class="MsoNormal" align="center" style="margin:0in 0in 8pt;text-align:center;line-height:15.6933px"><hr size="2" width="100%" align="center"></div><div><p class="MsoNormal" style="margin:0in;line-height:normal"><font face="arial, sans-serif"><b>Title:</b> Structural Coherence in Text Generation</font></p><p class="MsoNormal" style="margin:0in;line-height:normal"><font face="arial, sans-serif"> </font></p><p class="MsoNormal" style="margin:0in;line-height:normal"><font face="arial, sans-serif"><b>Abstract:</b> The field of text generation has seen significant progress in recent years. We are approaching a future where ubiquitous text generation technologies will allow us to generate long-form texts that are not only fluent at a surface level, but also coherent in their overall structure. To enable this future, my research focuses on evaluating and improving structure modeling in language models.<br><br>In the first part of this talk, I will introduce a method for quantifying structural coherence in language models. This method extracts structures by projecting data into a latent space of interest and then compares the structures in model generations to human-written text. This quantitative measure of structural coherence enables us to identify structural issues in language models and reveals that structural coherence does not fully correlate with surface fluency.<br><br>In the second part of the talk, I will present my research on improving structure modeling in language models. I will introduce a global model that scores the overall structure of text, in addition to the traditional language model that scores text by scoring each local word. The traditional language model excels at surface-level modeling, while the introduced global model specializes in structure modeling. I will demonstrate that the proposed model has a simple training and sampling procedure and leads to improvements in both local fluency and structural coherence.<br><br>To conclude, I will outline my future plans to extend my research into different types of sequence modeling problems that can benefit from structure modeling.</font></p><p class="MsoNormal" style="margin:0in;line-height:normal"><font face="arial, sans-serif"> </font></p><p class="MsoNormal" style="margin:0in;line-height:normal"><font face="arial, sans-serif"><b style="color:rgb(60,64,67)"><span style="line-height:13.91px">Bio:</span></b><span style="color:rgb(60,64,67);line-height:13.91px"> <span class="gmail-il">Yuntian</span> <span class="gmail-il">Deng</span> is a PhD student at Harvard University, advised by Professors Alexander Rush and Stuart Shieber. His research focuses on developing long-form text generation methods that are coherent, transparent, and efficient. He is also a key contributor to several open-source projects, including OpenNMT, image-to-LaTeX, and LaTeX-to-image.</span></font></p><p style="color:rgb(60,64,67)"><font face="arial, sans-serif"><span style="line-height:13.91px"><span class="gmail-il">Yuntian</span> is the recipient of an Nvidia Fellowship, a Baidu Fellowship, and multiple awards for his research, including the University of Chicago Rising Stars in Data Science, the ACL 2017 Best Demo Paper Runner-Up, the ACM Gordon Bell Special Prize for Covid Research, the Impact Award from Argonne National Lab, and the DAC 2020 Best Paper.</span><br></font></p><p style="color:rgb(60,64,67);letter-spacing:0.2px;white-space:pre-wrap"><font face="arial, sans-serif"><b style="letter-spacing:normal;color:rgb(34,34,34)">Host:</b><span style="letter-spacing:normal;color:rgb(34,34,34)"> </span><a href="mailto:klivescu@ttic.edu" target="_blank" style="letter-spacing:normal">Karen Livescu</a><br></font></p></div></div></span><br class="gmail-Apple-interchange-newline"></div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small"><br></div><div><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><span style="font-family:arial,helvetica,sans-serif;font-size:x-small">Mary C. Marre</span><br></div><div><div><font face="arial, helvetica, sans-serif" size="1">Faculty Administrative Support</font></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6" size="1"><b>Toyota Technological Institute</b></font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6" size="1">6045 S. Kenwood Avenue, Rm 517</font></i></div><div><font size="1"><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Chicago, IL 60637</font></i><br></font></div><div><font size="1"><i><font face="arial, helvetica, sans-serif" color="#3d85c6">773-834-1757</font></i></font></div><div><b><i><a href="mailto:mmarre@ttic.edu" target="_blank"><font face="arial, helvetica, sans-serif" size="1">mmarre@ttic.edu</font></a></i></b></div></div></div></div></div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Feb 13, 2023 at 5:54 PM Mary Marre <<a href="mailto:mmarre@ttic.edu">mmarre@ttic.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div style="font-size:small"><font face="arial, sans-serif"><font style="color:rgb(0,0,0);vertical-align:inherit"><font style="vertical-align:inherit"><b>When:</b> </font></font><font style="vertical-align:inherit"><font style="vertical-align:inherit"><font style="color:rgb(0,0,0)"> Monday</font><span class="gmail_default" style="color:rgb(0,0,0)">, February 20, 2023</span><font style="color:rgb(0,0,0)"> at</font><b style="color:rgb(0,0,0)"> <u>11:30</u></b><b><u><font color="#000000"> a</font></u></b><b><u><font color="#000000">m CT</font></u><font color="#000000"> </font></b></font></font><br></font></div><span style="color:rgb(80,0,80)"><div><p style="font-size:small;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><font face="arial, sans-serif" color="#000000"><font style="vertical-align:inherit"><font style="vertical-align:inherit"><b><span style="background-color:rgb(255,255,0)"><br></span></b></font></font></font></p><div style="font-size:small"><font face="arial, sans-serif"><b><font color="#500050">Where: </font></b><font color="#000000">Talk will be given </font><font color="#000000" style="font-weight:bold"><u>live, in-person</u></font><font style="font-weight:bold"> </font>at</font></div><p class="MsoNormal" style="font-size:small;margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font color="#500050"> </font><font color="#000000"> TTIC, 6045 S. Kenwood Avenue</font></font></p><p class="MsoNormal" style="font-size:small;margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> 5th Floor, Room 530<b> </b></font></p><p class="MsoNormal" style="font-size:small;margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><b><span style="color:black"><br></span></b></font></p><p class="MsoNormal" style="font-size:small;margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><b style="color:rgb(60,64,67);letter-spacing:0.2px;white-space:pre-wrap">Virtually:</b><span style="color:rgb(60,64,67);letter-spacing:0.2px;white-space:pre-wrap"> <i>via</i> Panopto </span>(<b><a href="https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=5b6b593d-06dd-4b63-b3a6-afa8017c3c02" target="_blank">livestream</a></b><span style="color:rgb(60,64,67);letter-spacing:0.2px;white-space:pre-wrap">)</span><br></font></p><p class="MsoNormal" style="font-size:small;margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><br></font></p><p class="MsoNormal" style="font-size:small;margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font style="vertical-align:inherit"><font style="vertical-align:inherit"><b>Who: </b> <font color="#500050"> </font><font color="#000000"><font color="#500050"> </font></font></font></font></font>Yuntian Deng, Harvard University</p><p class="MsoNormal" style="font-size:small;margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><br></font></p><div class="MsoNormal" align="center" style="font-size:small;margin:0in 0in 8pt;text-align:center;line-height:15.6933px"><hr size="2" width="100%" align="center"></div><div><p class="MsoNormal" style="margin:0in;line-height:normal"><font face="arial, sans-serif"><b>Title:</b> Structural Coherence in Text
Generation</font></p><p class="MsoNormal" style="margin:0in;line-height:normal"><font face="arial, sans-serif"> </font></p><p class="MsoNormal" style="margin:0in;line-height:normal"><font face="arial, sans-serif"><b>Abstract:</b> The field of text generation has seen
significant progress in recent years. We are approaching a future where
ubiquitous text generation technologies will allow us to generate long-form
texts that are not only fluent at a surface level, but also coherent in their
overall structure. To enable this future, my research focuses on evaluating and
improving structure modeling in language models.<br>
<br>
In the first part of this talk, I will introduce a method for quantifying
structural coherence in language models. This method extracts structures by
projecting data into a latent space of interest and then compares the
structures in model generations to human-written text. This quantitative
measure of structural coherence enables us to identify structural issues in language
models and reveals that structural coherence does not fully correlate with
surface fluency.<br>
<br>
In the second part of the talk, I will present my research on improving
structure modeling in language models. I will introduce a global model that
scores the overall structure of text, in addition to the traditional language
model that scores text by scoring each local word. The traditional language
model excels at surface-level modeling, while the introduced global model
specializes in structure modeling. I will demonstrate that the proposed model
has a simple training and sampling procedure and leads to improvements in both
local fluency and structural coherence.<br>
<br>
To conclude, I will outline my future plans to extend my research into
different types of sequence modeling problems that can benefit from structure
modeling.</font></p><p class="MsoNormal" style="margin:0in;line-height:normal"><font face="arial, sans-serif"> </font></p><p class="MsoNormal" style="margin:0in;line-height:normal"><font face="arial, sans-serif"><b style="color:rgb(60,64,67)"><span style="line-height:107%">Bio:</span></b><span style="color:rgb(60,64,67);line-height:107%"> Yuntian Deng is
a PhD student at Harvard University, advised by Professors Alexander Rush and
Stuart Shieber. His research focuses on developing long-form text generation
methods that are coherent, transparent, and efficient. He is also a key
contributor to several open-source projects, including OpenNMT, image-to-LaTeX,
and LaTeX-to-image.</span></font></p><p style="color:rgb(60,64,67)"><font face="arial, sans-serif"><span style="line-height:107%">Yuntian is the recipient of an Nvidia Fellowship, a Baidu Fellowship, and
multiple awards for his research, including the University of Chicago Rising
Stars in Data Science, the ACL 2017 Best Demo Paper Runner-Up, the ACM Gordon
Bell Special Prize for Covid Research, the Impact Award from Argonne National
Lab, and the DAC 2020 Best Paper.</span><br></font></p><p style="color:rgb(60,64,67);letter-spacing:0.2px;white-space:pre-wrap"><font face="arial, sans-serif"><b style="letter-spacing:normal;color:rgb(34,34,34)">Host:</b><span style="letter-spacing:normal;color:rgb(34,34,34)"> </span><a href="mailto:klivescu@ttic.edu" style="letter-spacing:normal" target="_blank">Karen Livescu</a><br></font></p></div></div></span><br></div><div><br></div><div><br></div><div><div dir="ltr"><div dir="ltr"><div><span style="font-family:arial,helvetica,sans-serif;font-size:x-small">Mary C. Marre</span><br></div><div><div><font face="arial, helvetica, sans-serif" size="1">Faculty Administrative Support</font></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6" size="1"><b>Toyota Technological Institute</b></font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6" size="1">6045 S. Kenwood Avenue, Rm 517</font></i></div><div><font size="1"><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Chicago, IL 60637</font></i><br></font></div><div><font size="1"><i><font face="arial, helvetica, sans-serif" color="#3d85c6">773-834-1757</font></i></font></div><div><b><i><a href="mailto:mmarre@ttic.edu" target="_blank"><font face="arial, helvetica, sans-serif" size="1">mmarre@ttic.edu</font></a></i></b></div></div></div></div></div></div>
</blockquote></div></div>