<div dir="ltr"><div><div class="gmail_default" style="font-family:georgia,serif;font-size:small"><div class="gmail_default" style="font-family:Arial,Helvetica,sans-serif"><font face="georgia, serif" color="#000000"><font style="vertical-align:inherit"><font style="vertical-align:inherit"><b>When:</b>    </font></font><font style="vertical-align:inherit"><font style="vertical-align:inherit">    Thursday, January 23rd at <b style="background-color:rgb(255,255,0)">10</b><b style="background-color:rgb(255,255,0)">AM CT</b><b> </b></font></font></font></div><div style="font-family:Arial,Helvetica,sans-serif"><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><font color="#000000"><font style="vertical-align:inherit"><font face="georgia, serif" style="vertical-align:inherit"><b><span style="background-color:rgb(255,255,0)"><br></span></b></font></font></font></p><div class="gmail_default"><font face="georgia, serif" color="#000000"><b>Where:       </b>Talk will be given <font style="font-weight:bold"><u>live, in-person</u></font><font style="font-weight:bold"> </font>at</font></div><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000">                       TTIC, 6045 S. Kenwood Avenue</font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000" face="georgia, serif">                       5th Floor, Room 530<b> </b></font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><b><font face="georgia, serif" color="#000000"><br></font></b></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"><b style="letter-spacing:0.2px">Virtually:</b><span style="letter-spacing:0.2px">  via Panopto </span>(<a href="https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=de3ed27c-aea1-474e-ace4-b2610184c023" target="_blank">livestream</a><span style="letter-spacing:0.2px">)</span><br></font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"><br></font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"><font style="vertical-align:inherit"><font style="vertical-align:inherit"><b>Who: </b>         </font></font><span class="gmail_default"></span><span class="gmail_default"></span><span class="gmail_default"></span><span class="gmail_default">Will Merrill</span><span class="gmail_default">, New York University</span></font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"></p><div><p style="letter-spacing:0.2px"><font face="georgia, serif" color="#000000"><span style="letter-spacing:normal"><b>Title:</b>         </span><span style="letter-spacing:normal">Theoretical Computer Science as a Lens to Understand and Improve Large Language Models</span></font></p><div><font face="georgia, serif" color="#000000"><b>Abstract:  </b>Scaling up large language models has enabled tremendous progress in NLP and deep learning, but how far can this paradigm be pushed? In this talk, I will discuss my body of theoretical results on the expressive power of language modeling architectures, and how these results bear on this question. I will start with my theoretical result that transformers (without chain of thought) can only express problems in the complexity class uniform TC0 and thus cannot express many simple computational problems including state tracking, evaluating compositional formulas, and graph connectivity. I will then discuss my work characterizing how chain of thought approaches can expand the expressive power of transformers, as well as my work comparing the expressive power of state-space models and transformers. Overall, these findings reveal a fundamental tradeoff between parallelism and expressive power: the parallelism so essential for scaling up transformer language models also precludes them from expressing many simple computational problems. These insights let us more precisely understand the limitations of transformers and also provide a strong foundation upon which to develop novel language modeling architectures and inference methods, forming a key part of my future research agenda.</font></div><div><font face="georgia, serif" color="#000000"><br></font></div><div><font face="georgia, serif" color="#000000"><b>Short Bio</b>: Will is a PhD student at the Center for Data Science at NYU advised by Tal Linzen and is funded by an NSF Graduate Research Fellowship and a Two Sigma PhD Fellowship. Will has also worked and interned at the Allen Institute for AI and Google Research. A major focus of Will’s research has been to characterize the computational power and limitations of transformers, with an eye towards understanding how transformer language models represent linguistic structure and solve reasoning problems. He has also worked on understanding the foundations of distributional semantics and helped train OLMo: one of the best fully open large language models.</font></div><font face="georgia, serif" color="#000000"><br><b>Host: <a href="mailto:zhiyuanli@ttic.edu" target="_blank"><span class="gmail_default"></span></a><a href="mailto:nati@ttic.edu" target="_blank"><span class="gmail_default"></span></a><a href="mailto:greg@ttic.edu" target="_blank"><span class="gmail_default"></span></a><a href="http://n/" target="_blank">Nati Srebro</a></b></font></div></div></div><br clear="all"></div><div><br></div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><b style="background-color:rgb(255,255,255)"><font color="#3d85c6">Brandie Jones </font></b><div><div><div><font color="#3d85c6"><b><i>Executive </i></b></font><b style="color:rgb(61,133,198)"><i>Administrative Assistant</i></b></div></div><div><span style="background-color:rgb(255,255,255)"><font color="#3d85c6">Toyota Technological Institute</font></span></div><div><span style="background-color:rgb(255,255,255)"><font color="#3d85c6">6045 S. Kenwood Avenue</font></span></div><div><span style="background-color:rgb(255,255,255)"><font color="#3d85c6">Chicago, IL  60637</font></span></div></div><div><span style="background-color:rgb(255,255,255)"><font color="#3d85c6"><a href="http://www.ttic.edu" target="_blank">www.ttic.edu</a> </font></span></div><div><span style="background-color:rgb(255,255,255)"><font color="#3d85c6"><br></font></span></div></div></div></div>