<div dir="ltr"><div dir="ltr"><div class="gmail_default" style="font-size:small"><div class="gmail_default"><div class="gmail_default"><div class="gmail_default" style="color:rgb(80,0,80)"><font style="font-family:arial,sans-serif;color:rgb(0,0,0);vertical-align:inherit"><font style="vertical-align:inherit"><b>When:</b> </font></font><font style="font-family:arial,sans-serif;vertical-align:inherit"><font style="vertical-align:inherit"><font style="color:rgb(0,0,0)"> </font><span class="gmail_default" style="color:rgb(0,0,0)">Tuesday, March 19<span class="gmail_default">, </span>2024</span><font style="color:rgb(0,0,0)"> at</font><b style="color:rgb(0,0,0)"> <u><font style="background-color:rgb(255,255,0)">11:00</font></u></b><b><u><font color="#000000" style="background-color:rgb(255,255,0)"> a</font></u></b><b><u><font color="#000000" style="background-color:rgb(255,255,0)">m CT</font></u><font color="#000000"> </font></b></font></font><br></div><div class="gmail_default"><p style="color:rgb(80,0,80);font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><b style="font-family:arial,sans-serif"><font color="#500050"><br></font></b></p><p style="color:rgb(80,0,80);font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><b style="font-family:arial,sans-serif"><font color="#500050">Where: </font></b><font color="#000000" style="font-family:arial,sans-serif">Talk will be given </font><font color="#000000" style="font-family:arial,sans-serif;font-weight:bold"><u>live, in-person</u></font><font style="font-family:arial,sans-serif;font-weight:bold"> </font><span style="font-family:arial,sans-serif">at</span><br></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font color="#500050"> </font><font color="#000000"> TTIC, 6045 S. Kenwood Avenue</font></font></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> 5th Floor, Room 530<b> </b></font></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><b><span style="color:black"><br></span></b></font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><b style="color:rgb(60,64,67);letter-spacing:0.2px">Virtually:</b><span style="color:rgb(60,64,67);letter-spacing:0.2px"> via Panopto (</span><i style="color:rgb(60,64,67);letter-spacing:0.2px"><a href="https://www.google.com/url?q=https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id%3Dc2b3423f-1057-4c5d-8ce1-b134017024ed&sa=D&source=calendar&ust=1710973448052118&usg=AOvVaw2Dc3qDb6aqjgEXUqKXGp5l" target="_blank"><b>livestream</b></a></i><span style="color:rgb(60,64,67);letter-spacing:0.2px">)</span></font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><span style="color:rgb(60,64,67);letter-spacing:0.2px"><font face="arial, sans-serif"></font><font face="georgia, serif"><b><font size="1"> </font></b></font></span></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font style="color:rgb(80,0,80);font-family:arial,sans-serif;vertical-align:inherit"><font style="vertical-align:inherit"><b>Who: </b> <font color="#500050"> </font><font color="#000000"><font color="#500050"> </font></font></font></font>Peter West, University of Washington</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><br></p></div></div><div class="gmail_default"><div dir="ltr"><div><div class="MsoNormal" align="center" style="margin:0in 0in 8pt;text-align:center;line-height:15.6933px;font-size:11pt;font-family:Calibri,sans-serif"><hr size="3" width="100%" noshade align="center" style="color:rgb(46,116,181)"></div></div><div><p><font face="arial, sans-serif"><strong>Title: </strong>Hidden Capabilities and Counterintuitive Limits in Large Language Models</font></p><p><font face="arial, sans-serif"><strong>Abstract: </strong>Massive scale has been a recent winning recipe in natural language processing and AI, with extreme-scale language models like GPT-4 receiving most attention. This is in spite of staggering energy and monetary costs, and further, the continuing struggle of even the largest models with concepts such as compositional problem solving and linguistic ambiguity. In this talk, I will propose my vision for a research landscape where compact language models share the forefront with extreme scale models, working in concert with many pieces besides scale, such as algorithms, knowledge, information theory, and more.</font></p><p><font face="arial, sans-serif">The first part of my talk will cover alternative ingredients to scale, including (1) an inference-time algorithm that combines language models with elements of discrete search and information theory and (2) a method for transferring useful knowledge from extreme-scale to compact language models with synthetically generated data. Next, I will discuss counterintuitive disparities in the capabilities of even extreme-scale models, which can meet or exceed human performance in some complex tasks while trailing behind humans in what seem to be much simpler tasks. Finally, I will discuss implications and next steps in scale-alternative methods.</font></p><p><font face="arial, sans-serif"><strong>Bio: </strong>Peter West is a PhD candidate in the Paul G. Allen School of Computer Science & Engineering at the University of Washington, working with Yejin Choi. His research is focused on natural language processing and language models, particularly combining language models with elements of knowledge, search algorithms, and information theory to equip compact models with new capabilities. In parallel, he studies the limits that even extreme-scale models have yet to solve. His work has received multiple awards, including best methods paper at NAACL 2022, and outstanding paper awards at ACL and EMNLP in 2023. His work has been supported in part by the NSERC PGS-D fellowship. Previously, Peter received a BSc in computer science from the University of British Columbia.</font></p></div></div><div><div id="m_348600480430748061m_6196770158209582839m_-752615248277548940m_-7248294781002049715m_6138816470193778399m_-5748196637763862687m_-3176780616110998963m_8681342134781089432m_285953687803334144m_5973546232189203326m_-6976229087647104413m_7917979885129397482m_-5423657134431402203m_-1337599008586739890m_8237382617653311322m_-1231130334284673048m_-1282025577005441955m_-1973358356214118865m_-5815637555669013367m_1779572315514282115m_-4485402625451270420m_1520697528942856564m_5948359943660736735m_-4789039193346764527m_3599676094611771654m_8264976978369198918m_7474850050874458051m_5107577024390010371m_253820422674989860m_3983419646637522536m_-7220900540036838011gmail-:qg" role="button" aria-label="Show trimmed content" aria-expanded="false"><font face="arial, sans-serif"><b>Host: </b><a href="mailto:mcallester@ttic.edu" target="_blank"><b>David McAllester</b></a></font></div></div></div></div><div class="gmail_default"><br></div><div class="gmail_default"><font face="arial, sans-serif" size="1"><font color="#0000ff">*</font>Access to the </font><font size="1" face="georgia, serif"><i><b>recorded</b></i></font><font face="arial, sans-serif" size="1"> talk is limited to TTIC / UChicago (press panopto link and sign in to your UChicago account with <span style="color:rgb(29,29,33);font-weight:600">CNetID</span>).</font><br></div><div class="gmail_default"><br></div><div class="gmail_default"><br></div><div class="gmail_default"><br></div><div class="gmail_default"><br></div></div><div><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><span style="font-family:arial,helvetica,sans-serif;font-size:x-small">Mary C. Marre</span><br></div><div><div><font face="arial, helvetica, sans-serif" size="1">Faculty Administrative Support</font></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6" size="1"><b>Toyota Technological Institute</b></font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6" size="1">6045 S. Kenwood Avenue, Rm 517</font></i></div><div><font size="1"><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Chicago, IL 60637</font></i><br></font></div><div><font size="1"><i><font face="arial, helvetica, sans-serif" color="#3d85c6">773-834-1757</font></i></font></div><div><b><i><a href="mailto:mmarre@ttic.edu" target="_blank"><font face="arial, helvetica, sans-serif" size="1">mmarre@ttic.edu</font></a></i></b></div></div></div></div></div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Mar 19, 2024 at 10:28 AM Mary Marre <<a href="mailto:mmarre@ttic.edu">mmarre@ttic.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div style="font-size:small"><div><div><div style="color:rgb(80,0,80)"><font style="font-family:arial,sans-serif;color:rgb(0,0,0);vertical-align:inherit"><font style="vertical-align:inherit"><b>When:</b> </font></font><font style="font-family:arial,sans-serif;vertical-align:inherit"><font style="vertical-align:inherit"><font style="color:rgb(0,0,0)"> </font><span class="gmail_default" style="color:rgb(0,0,0)">Tuesday, March 19<span class="gmail_default">, </span>2024</span><font style="color:rgb(0,0,0)"> at</font><b style="color:rgb(0,0,0)"> <u><font style="background-color:rgb(255,255,0)">11:00</font></u></b><b><u><font color="#000000" style="background-color:rgb(255,255,0)"> a</font></u></b><b><u><font color="#000000" style="background-color:rgb(255,255,0)">m CT</font></u><font color="#000000"> </font></b></font></font><br></div><div><p style="color:rgb(80,0,80);font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><b style="font-family:arial,sans-serif"><font color="#500050"><br></font></b></p><p style="color:rgb(80,0,80);font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><b style="font-family:arial,sans-serif"><font color="#500050">Where: </font></b><font color="#000000" style="font-family:arial,sans-serif">Talk will be given </font><font color="#000000" style="font-family:arial,sans-serif;font-weight:bold"><u>live, in-person</u></font><font style="font-family:arial,sans-serif;font-weight:bold"> </font><span style="font-family:arial,sans-serif">at</span><br></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font color="#500050"> </font><font color="#000000"> TTIC, 6045 S. Kenwood Avenue</font></font></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> 5th Floor, Room 530<b> </b></font></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><b><span style="color:black"><br></span></b></font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><b style="color:rgb(60,64,67);letter-spacing:0.2px">Virtually:</b><span style="color:rgb(60,64,67);letter-spacing:0.2px"> via Panopto (</span><i style="color:rgb(60,64,67);letter-spacing:0.2px"><a href="https://www.google.com/url?q=https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id%3Dc2b3423f-1057-4c5d-8ce1-b134017024ed&sa=D&source=calendar&ust=1710973448052118&usg=AOvVaw2Dc3qDb6aqjgEXUqKXGp5l" target="_blank"><b>livestream</b></a></i><span style="color:rgb(60,64,67);letter-spacing:0.2px">)</span></font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><span style="color:rgb(60,64,67);letter-spacing:0.2px"><font face="arial, sans-serif"></font><font face="georgia, serif"><b><font size="1"> </font></b></font></span></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font style="color:rgb(80,0,80);font-family:arial,sans-serif;vertical-align:inherit"><font style="vertical-align:inherit"><b>Who: </b> <font color="#500050"> </font><font color="#000000"><font color="#500050"> </font></font></font></font>Peter West, University of Washington</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><br></p></div></div><div><div dir="ltr"><div><div class="MsoNormal" align="center" style="margin:0in 0in 8pt;text-align:center;line-height:15.6933px;font-size:11pt;font-family:Calibri,sans-serif"><hr size="3" width="100%" noshade align="center" style="color:rgb(46,116,181)"></div></div><div><p><font face="arial, sans-serif"><strong>Title: </strong>Hidden Capabilities and Counterintuitive Limits in Large Language Models</font></p><p><font face="arial, sans-serif"><strong>Abstract: </strong>Massive scale has been a recent winning recipe in natural language processing and AI, with extreme-scale language models like GPT-4 receiving most attention. This is in spite of staggering energy and monetary costs, and further, the continuing struggle of even the largest models with concepts such as compositional problem solving and linguistic ambiguity. In this talk, I will propose my vision for a research landscape where compact language models share the forefront with extreme scale models, working in concert with many pieces besides scale, such as algorithms, knowledge, information theory, and more.</font></p><p><font face="arial, sans-serif">The first part of my talk will cover alternative ingredients to scale, including (1) an inference-time algorithm that combines language models with elements of discrete search and information theory and (2) a method for transferring useful knowledge from extreme-scale to compact language models with synthetically generated data. Next, I will discuss counterintuitive disparities in the capabilities of even extreme-scale models, which can meet or exceed human performance in some complex tasks while trailing behind humans in what seem to be much simpler tasks. Finally, I will discuss implications and next steps in scale-alternative methods.</font></p><p><font face="arial, sans-serif"><strong>Bio: </strong>Peter West is a PhD candidate in the Paul G. Allen School of Computer Science & Engineering at the University of Washington, working with Yejin Choi. His research is focused on natural language processing and language models, particularly combining language models with elements of knowledge, search algorithms, and information theory to equip compact models with new capabilities. In parallel, he studies the limits that even extreme-scale models have yet to solve. His work has received multiple awards, including best methods paper at NAACL 2022, and outstanding paper awards at ACL and EMNLP in 2023. His work has been supported in part by the NSERC PGS-D fellowship. Previously, Peter received a BSc in computer science from the University of British Columbia.</font></p></div></div><div><div id="m_348600480430748061m_6196770158209582839m_-752615248277548940m_-7248294781002049715m_6138816470193778399m_-5748196637763862687m_-3176780616110998963m_8681342134781089432m_285953687803334144m_5973546232189203326m_-6976229087647104413m_7917979885129397482m_-5423657134431402203m_-1337599008586739890m_8237382617653311322m_-1231130334284673048m_-1282025577005441955m_-1973358356214118865m_-5815637555669013367m_1779572315514282115m_-4485402625451270420m_1520697528942856564m_5948359943660736735m_-4789039193346764527m_3599676094611771654m_8264976978369198918m_7474850050874458051m_5107577024390010371m_253820422674989860m_3983419646637522536m_-7220900540036838011gmail-:qg" role="button" aria-label="Show trimmed content" aria-expanded="false"><font face="arial, sans-serif"><b>Host: </b><a href="mailto:mcallester@ttic.edu" target="_blank"><b>David McAllester</b></a></font></div></div></div></div><div><br></div><div><font face="arial, sans-serif" size="1"><font color="#0000ff">*</font>Access to the </font><font size="1" face="georgia, serif"><i><b>recorded</b></i></font><font face="arial, sans-serif" size="1"> talk is limited to TTIC / UChicago (press panopto link and sign in to your UChicago account with <span style="color:rgb(29,29,33);font-weight:600">CNetID</span>).</font><br></div><div><br></div><div><br></div></div><div><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><span style="font-family:arial,helvetica,sans-serif;font-size:x-small">Mary C. Marre</span><br></div><div><div><font face="arial, helvetica, sans-serif" size="1">Faculty Administrative Support</font></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6" size="1"><b>Toyota Technological Institute</b></font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6" size="1">6045 S. Kenwood Avenue, Rm 517</font></i></div><div><font size="1"><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Chicago, IL 60637</font></i><br></font></div><div><font size="1"><i><font face="arial, helvetica, sans-serif" color="#3d85c6">773-834-1757</font></i></font></div><div><b><i><a href="mailto:mmarre@ttic.edu" target="_blank"><font face="arial, helvetica, sans-serif" size="1">mmarre@ttic.edu</font></a></i></b></div></div></div></div></div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Mar 19, 2024 at 8:30 AM Mary Marre <<a href="mailto:mmarre@ttic.edu" target="_blank">mmarre@ttic.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div><div style="font-size:small"><div><div style="color:rgb(80,0,80)"><font style="font-family:arial,sans-serif;color:rgb(0,0,0);vertical-align:inherit"><font style="vertical-align:inherit"><b>When:</b> </font></font><font style="font-family:arial,sans-serif;vertical-align:inherit"><font style="vertical-align:inherit"><font style="color:rgb(0,0,0)"> </font><span class="gmail_default" style="color:rgb(0,0,0)">Tuesday, March 19<span class="gmail_default">, </span>2024</span><font style="color:rgb(0,0,0)"> at</font><b style="color:rgb(0,0,0)"> <u><font style="background-color:rgb(255,255,0)">11:00</font></u></b><b><u><font color="#000000" style="background-color:rgb(255,255,0)"> a</font></u></b><b><u><font color="#000000" style="background-color:rgb(255,255,0)">m CT</font></u><font color="#000000"> </font></b></font></font><br></div><div><p style="color:rgb(80,0,80);font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><b style="font-family:arial,sans-serif"><font color="#500050"><br></font></b></p><p style="color:rgb(80,0,80);font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><b style="font-family:arial,sans-serif"><font color="#500050">Where: </font></b><font color="#000000" style="font-family:arial,sans-serif">Talk will be given </font><font color="#000000" style="font-family:arial,sans-serif;font-weight:bold"><u>live, in-person</u></font><font style="font-family:arial,sans-serif;font-weight:bold"> </font><span style="font-family:arial,sans-serif">at</span><br></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font color="#500050"> </font><font color="#000000"> TTIC, 6045 S. Kenwood Avenue</font></font></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> 5th Floor, Room 530<b> </b></font></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><b><span style="color:black"><br></span></b></font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><b style="color:rgb(60,64,67);letter-spacing:0.2px">Virtually:</b><span style="color:rgb(60,64,67);letter-spacing:0.2px"> via Panopto (</span><i style="color:rgb(60,64,67);letter-spacing:0.2px"><a href="https://www.google.com/url?q=https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id%3Dc2b3423f-1057-4c5d-8ce1-b134017024ed&sa=D&source=calendar&ust=1710973448052118&usg=AOvVaw2Dc3qDb6aqjgEXUqKXGp5l" target="_blank"><b>livestream</b></a></i><span style="color:rgb(60,64,67);letter-spacing:0.2px">)</span></font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><span style="color:rgb(60,64,67);letter-spacing:0.2px"><font face="arial, sans-serif"></font><font face="georgia, serif"><b><font size="1"> </font></b></font></span></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font style="color:rgb(80,0,80);font-family:arial,sans-serif;vertical-align:inherit"><font style="vertical-align:inherit"><b>Who: </b> <font color="#500050"> </font><font color="#000000"><font color="#500050"> </font></font></font></font>Peter West, University of Washington</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><br></p></div></div><div><div dir="ltr"><div><div class="MsoNormal" align="center" style="margin:0in 0in 8pt;text-align:center;line-height:15.6933px;font-size:11pt;font-family:Calibri,sans-serif"><hr size="3" width="100%" noshade align="center" style="color:rgb(46,116,181)"></div></div><div><p><font face="arial, sans-serif"><strong>Title: </strong>Hidden Capabilities and Counterintuitive Limits in Large Language Models</font></p><p><font face="arial, sans-serif"><strong>Abstract: </strong>Massive scale has been a recent winning recipe in natural language processing and AI, with extreme-scale language models like GPT-4 receiving most attention. This is in spite of staggering energy and monetary costs, and further, the continuing struggle of even the largest models with concepts such as compositional problem solving and linguistic ambiguity. In this talk, I will propose my vision for a research landscape where compact language models share the forefront with extreme scale models, working in concert with many pieces besides scale, such as algorithms, knowledge, information theory, and more.</font></p><p><font face="arial, sans-serif">The first part of my talk will cover alternative ingredients to scale, including (1) an inference-time algorithm that combines language models with elements of discrete search and information theory and (2) a method for transferring useful knowledge from extreme-scale to compact language models with synthetically generated data. Next, I will discuss counterintuitive disparities in the capabilities of even extreme-scale models, which can meet or exceed human performance in some complex tasks while trailing behind humans in what seem to be much simpler tasks. Finally, I will discuss implications and next steps in scale-alternative methods.</font></p><p><font face="arial, sans-serif"><strong>Bio: </strong>Peter West is a PhD candidate in the Paul G. Allen School of Computer Science & Engineering at the University of Washington, working with Yejin Choi. His research is focused on natural language processing and language models, particularly combining language models with elements of knowledge, search algorithms, and information theory to equip compact models with new capabilities. In parallel, he studies the limits that even extreme-scale models have yet to solve. His work has received multiple awards, including best methods paper at NAACL 2022, and outstanding paper awards at ACL and EMNLP in 2023. His work has been supported in part by the NSERC PGS-D fellowship. Previously, Peter received a BSc in computer science from the University of British Columbia.</font></p></div></div><div><div id="m_348600480430748061m_6196770158209582839m_-752615248277548940m_-7248294781002049715m_6138816470193778399m_-5748196637763862687m_-3176780616110998963m_8681342134781089432m_285953687803334144m_5973546232189203326m_-6976229087647104413m_7917979885129397482m_-5423657134431402203m_-1337599008586739890m_8237382617653311322m_-1231130334284673048m_-1282025577005441955m_-1973358356214118865m_-5815637555669013367m_1779572315514282115m_-4485402625451270420m_1520697528942856564m_5948359943660736735m_-4789039193346764527m_3599676094611771654m_8264976978369198918m_7474850050874458051m_5107577024390010371m_253820422674989860m_3983419646637522536m_-7220900540036838011gmail-:qg" role="button" aria-label="Show trimmed content" aria-expanded="false"><font face="arial, sans-serif"><b>Host: </b><a href="mailto:mcallester@ttic.edu" target="_blank"><b>David McAllester</b></a></font></div></div></div></div><div style="font-size:small"><br></div><div><font face="arial, sans-serif" size="1"><font color="#0000ff">*</font>Access to the </font><font size="1" face="georgia, serif"><i><b>recorded</b></i></font><font face="arial, sans-serif" size="1"> talk is limited to TTIC / UChicago (press panopto link and sign in to your UChicago account with <span style="color:rgb(29,29,33);font-weight:600">CNetID</span>).</font><br></div><div style="font-size:small"><br></div><div style="font-size:small"><br></div><div style="font-size:small"><br></div><div style="font-size:small"><br></div><div style="font-size:small"><br></div></div><div><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><span style="font-family:arial,helvetica,sans-serif;font-size:x-small">Mary C. Marre</span><br></div><div><div><font face="arial, helvetica, sans-serif" size="1">Faculty Administrative Support</font></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6" size="1"><b>Toyota Technological Institute</b></font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6" size="1">6045 S. Kenwood Avenue, Rm 517</font></i></div><div><font size="1"><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Chicago, IL 60637</font></i><br></font></div><div><font size="1"><i><font face="arial, helvetica, sans-serif" color="#3d85c6">773-834-1757</font></i></font></div><div><b><i><a href="mailto:mmarre@ttic.edu" target="_blank"><font face="arial, helvetica, sans-serif" size="1">mmarre@ttic.edu</font></a></i></b></div></div></div></div></div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Mar 18, 2024 at 1:00 PM Mary Marre <<a href="mailto:mmarre@ttic.edu" target="_blank">mmarre@ttic.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div><div><div><div style="font-size:small;color:rgb(80,0,80)"><font style="font-family:arial,sans-serif;color:rgb(0,0,0);vertical-align:inherit"><font style="vertical-align:inherit"><b>When:</b> </font></font><font style="font-family:arial,sans-serif;vertical-align:inherit"><font style="vertical-align:inherit"><font style="color:rgb(0,0,0)"> </font><span class="gmail_default" style="color:rgb(0,0,0)">Tuesday, March 19<span class="gmail_default">, </span>2024</span><font style="color:rgb(0,0,0)"> at</font><b style="color:rgb(0,0,0)"> <u><font style="background-color:rgb(255,255,0)">11:00</font></u></b><b><u><font color="#000000" style="background-color:rgb(255,255,0)"> a</font></u></b><b><u><font color="#000000" style="background-color:rgb(255,255,0)">m CT</font></u><font color="#000000"> </font></b></font></font><br></div><div><p style="font-size:small;color:rgb(80,0,80);font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><b style="font-family:arial,sans-serif"><font color="#500050"><br></font></b></p><p style="font-size:small;color:rgb(80,0,80);font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><b style="font-family:arial,sans-serif"><font color="#500050">Where: </font></b><font color="#000000" style="font-family:arial,sans-serif">Talk will be given </font><font color="#000000" style="font-family:arial,sans-serif;font-weight:bold"><u>live, in-person</u></font><font style="font-family:arial,sans-serif;font-weight:bold"> </font><span style="font-family:arial,sans-serif">at</span><br></p><p class="MsoNormal" style="font-size:small;margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font color="#500050"> </font><font color="#000000"> TTIC, 6045 S. Kenwood Avenue</font></font></p><p class="MsoNormal" style="font-size:small;margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> 5th Floor, Room 530<b> </b></font></p><p class="MsoNormal" style="font-size:small;margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><b><span style="color:black"><br></span></b></font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><b style="font-size:small;color:rgb(60,64,67);letter-spacing:0.2px">Virtually:</b><span style="color:rgb(60,64,67);letter-spacing:0.2px"> via Panopto (</span><i style="color:rgb(60,64,67);letter-spacing:0.2px"><a href="https://www.google.com/url?q=https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id%3Dc2b3423f-1057-4c5d-8ce1-b134017024ed&sa=D&source=calendar&ust=1710973448052118&usg=AOvVaw2Dc3qDb6aqjgEXUqKXGp5l" target="_blank"><b>livestream</b></a></i><span style="color:rgb(60,64,67);letter-spacing:0.2px">)</span></font></p><p class="MsoNormal" style="font-size:small;margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><span style="color:rgb(60,64,67);letter-spacing:0.2px"><font face="arial, sans-serif"></font><font face="georgia, serif"><b><font size="1"> </font></b></font></span></p><p class="MsoNormal" style="font-size:small;margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font style="color:rgb(80,0,80);font-family:arial,sans-serif;vertical-align:inherit"><font style="vertical-align:inherit"><b>Who: </b> <font color="#500050"> </font><font color="#000000"><font color="#500050"> </font></font></font></font>Peter West, University of Washington</p><p class="MsoNormal" style="font-size:small;margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><br></p></div></div><div style="font-size:small"><div dir="ltr"><div><div class="MsoNormal" align="center" style="margin:0in 0in 8pt;text-align:center;line-height:15.6933px;font-size:11pt;font-family:Calibri,sans-serif"><hr size="3" width="100%" noshade align="center" style="color:rgb(46,116,181)"></div></div><div><p><font face="arial, sans-serif"><strong>Title: </strong>Hidden Capabilities and Counterintuitive Limits in Large Language Models</font></p><p><font face="arial, sans-serif"><strong>Abstract: </strong>Massive scale has been a recent winning recipe in natural language processing and AI, with extreme-scale language models like GPT-4 receiving most attention. This is in spite of staggering energy and monetary costs, and further, the continuing struggle of even the largest models with concepts such as compositional problem solving and linguistic ambiguity. In this talk, I will propose my vision for a research landscape where compact language models share the forefront with extreme scale models, working in concert with many pieces besides scale, such as algorithms, knowledge, information theory, and more.</font></p><p><font face="arial, sans-serif">The first part of my talk will cover alternative ingredients to scale, including (1) an inference-time algorithm that combines language models with elements of discrete search and information theory and (2) a method for transferring useful knowledge from extreme-scale to compact language models with synthetically generated data. Next, I will discuss counterintuitive disparities in the capabilities of even extreme-scale models, which can meet or exceed human performance in some complex tasks while trailing behind humans in what seem to be much simpler tasks. Finally, I will discuss implications and next steps in scale-alternative methods.</font></p><p><font face="arial, sans-serif"><strong>Bio: </strong>Peter West is a PhD candidate in the Paul G. Allen School of Computer Science & Engineering at the University of Washington, working with Yejin Choi. His research is focused on natural language processing and language models, particularly combining language models with elements of knowledge, search algorithms, and information theory to equip compact models with new capabilities. In parallel, he studies the limits that even extreme-scale models have yet to solve. His work has received multiple awards, including best methods paper at NAACL 2022, and outstanding paper awards at ACL and EMNLP in 2023. His work has been supported in part by the NSERC PGS-D fellowship. Previously, Peter received a BSc in computer science from the University of British Columbia.</font></p></div></div><div><div id="m_348600480430748061m_6196770158209582839m_-752615248277548940m_-7248294781002049715m_6138816470193778399m_-5748196637763862687m_-3176780616110998963m_8681342134781089432m_285953687803334144m_5973546232189203326m_-6976229087647104413m_7917979885129397482m_-5423657134431402203m_-1337599008586739890m_8237382617653311322m_-1231130334284673048m_-1282025577005441955m_-1973358356214118865m_-5815637555669013367m_1779572315514282115m_-4485402625451270420m_1520697528942856564m_5948359943660736735m_-4789039193346764527m_3599676094611771654m_8264976978369198918m_7474850050874458051m_5107577024390010371m_253820422674989860m_3983419646637522536m_-7220900540036838011gmail-:qg" role="button" aria-label="Show trimmed content" aria-expanded="false"><font face="arial, sans-serif"><b>Host: </b><a href="mailto:mcallester@ttic.edu" target="_blank"><b>David McAllester</b></a></font></div></div></div></div><div style="font-size:small"><br></div><div style="font-size:small"><br></div><div style="font-size:small"><br></div><div style="font-size:small"><br></div></div><div><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><span style="font-family:arial,helvetica,sans-serif;font-size:x-small">Mary C. Marre</span><br></div><div><div><font face="arial, helvetica, sans-serif" size="1">Faculty Administrative Support</font></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6" size="1"><b>Toyota Technological Institute</b></font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6" size="1">6045 S. Kenwood Avenue, Rm 517</font></i></div><div><font size="1"><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Chicago, IL 60637</font></i><br></font></div><div><font size="1"><i><font face="arial, helvetica, sans-serif" color="#3d85c6">773-834-1757</font></i></font></div><div><b><i><a href="mailto:mmarre@ttic.edu" target="_blank"><font face="arial, helvetica, sans-serif" size="1">mmarre@ttic.edu</font></a></i></b></div></div></div></div></div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Mar 14, 2024 at 6:19 PM Mary Marre <<a href="mailto:mmarre@ttic.edu" target="_blank">mmarre@ttic.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div><div style="font-size:small"><div style="color:rgb(80,0,80)"><font style="font-family:arial,sans-serif;color:rgb(0,0,0);vertical-align:inherit"><font style="vertical-align:inherit"><b>When:</b> </font></font><font style="font-family:arial,sans-serif;vertical-align:inherit"><font style="vertical-align:inherit"><font style="color:rgb(0,0,0)"> </font><span class="gmail_default" style="color:rgb(0,0,0)">Tuesday, March 19<span class="gmail_default">, </span>2024</span><font style="color:rgb(0,0,0)"> at</font><b style="color:rgb(0,0,0)"> <u><font style="background-color:rgb(255,255,0)">11:00</font></u></b><b><u><font color="#000000" style="background-color:rgb(255,255,0)"> a</font></u></b><b><u><font color="#000000" style="background-color:rgb(255,255,0)">m CT</font></u><font color="#000000"> </font></b></font></font><br></div><div><p style="color:rgb(80,0,80);font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><b style="font-family:arial,sans-serif"><font color="#500050"><br></font></b></p><p style="color:rgb(80,0,80);font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><b style="font-family:arial,sans-serif"><font color="#500050">Where: </font></b><font color="#000000" style="font-family:arial,sans-serif">Talk will be given </font><font color="#000000" style="font-family:arial,sans-serif;font-weight:bold"><u>live, in-person</u></font><font style="font-family:arial,sans-serif;font-weight:bold"> </font><span style="font-family:arial,sans-serif">at</span><br></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font color="#500050"> </font><font color="#000000"> TTIC, 6045 S. Kenwood Avenue</font></font></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> 5th Floor, Room 530<b> </b></font></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><b><span style="color:black"><br></span></b></font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><b style="color:rgb(60,64,67);letter-spacing:0.2px">Virtually:</b><span style="color:rgb(60,64,67);letter-spacing:0.2px"> <i>tba</i></span></font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><span style="color:rgb(60,64,67);letter-spacing:0.2px"><font face="arial, sans-serif"></font><font face="georgia, serif"><b><font size="1"> </font></b></font></span></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font style="color:rgb(80,0,80);font-family:arial,sans-serif;vertical-align:inherit"><font style="vertical-align:inherit"><b>Who: </b> <font color="#500050"> </font><font color="#000000"><font color="#500050"> </font></font></font></font>Peter West, University of Washington</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><br></p></div></div><div><div dir="ltr"><div style="font-size:small"><div class="MsoNormal" align="center" style="margin:0in 0in 8pt;text-align:center;line-height:15.6933px;font-size:11pt;font-family:Calibri,sans-serif"><hr size="3" width="100%" noshade align="center" style="color:rgb(46,116,181)"></div></div><div><p><font face="arial, sans-serif"><strong>Title: </strong>Hidden Capabilities and Counterintuitive Limits in Large Language Models</font></p><p><font face="arial, sans-serif"><strong>Abstract: </strong>Massive scale has been a recent winning recipe in natural language processing and AI, with extreme-scale language models like GPT-4 receiving most attention. This is in spite of staggering energy and monetary costs, and further, the continuing struggle of even the largest models with concepts such as compositional problem solving and linguistic ambiguity. In this talk, I will propose my vision for a research landscape where compact language models share the forefront with extreme scale models, working in concert with many pieces besides scale, such as algorithms, knowledge, information theory, and more.</font></p><p><font face="arial, sans-serif">The first part of my talk will cover alternative ingredients to scale, including (1) an inference-time algorithm that combines language models with elements of discrete search and information theory and (2) a method for transferring useful knowledge from extreme-scale to compact language models with synthetically generated data. Next, I will discuss counterintuitive disparities in the capabilities of even extreme-scale models, which can meet or exceed human performance in some complex tasks while trailing behind humans in what seem to be much simpler tasks. Finally, I will discuss implications and next steps in scale-alternative methods.</font></p><p><font face="arial, sans-serif"><strong>Bio: </strong>Peter West is a PhD candidate in the Paul G. Allen School of Computer Science & Engineering at the University of Washington, working with Yejin Choi. His research is focused on natural language processing and language models, particularly combining language models with elements of knowledge, search algorithms, and information theory to equip compact models with new capabilities. In parallel, he studies the limits that even extreme-scale models have yet to solve. His work has received multiple awards, including best methods paper at NAACL 2022, and outstanding paper awards at ACL and EMNLP in 2023. His work has been supported in part by the NSERC PGS-D fellowship. Previously, Peter received a BSc in computer science from the University of British Columbia.</font></p></div></div><div style="font-size:small"><div id="m_348600480430748061m_6196770158209582839m_-752615248277548940m_-7248294781002049715m_6138816470193778399m_-5748196637763862687m_-3176780616110998963m_8681342134781089432m_285953687803334144m_5973546232189203326m_-6976229087647104413m_7917979885129397482m_-5423657134431402203m_-1337599008586739890m_8237382617653311322m_-1231130334284673048m_-1282025577005441955m_-1973358356214118865m_-5815637555669013367m_1779572315514282115m_-4485402625451270420m_1520697528942856564m_5948359943660736735m_-4789039193346764527m_3599676094611771654m_8264976978369198918m_7474850050874458051m_5107577024390010371m_253820422674989860m_3983419646637522536m_-7220900540036838011gmail-:qg" role="button" aria-label="Show trimmed content" aria-expanded="false"><font face="arial, sans-serif"><b>Host: </b><a href="mailto:mcallester@ttic.edu" target="_blank"><b>David McAllester</b></a></font></div></div></div></div><div style="font-size:small"><br></div><div style="font-size:small"><div><br></div><div><br></div><div><br></div><div><br></div></div></div><div><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><span style="font-family:arial,helvetica,sans-serif;font-size:x-small">Mary C. Marre</span><br></div><div><div><font face="arial, helvetica, sans-serif" size="1">Faculty Administrative Support</font></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6" size="1"><b>Toyota Technological Institute</b></font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6" size="1">6045 S. Kenwood Avenue, Rm 517</font></i></div><div><font size="1"><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Chicago, IL 60637</font></i><br></font></div><div><font size="1"><i><font face="arial, helvetica, sans-serif" color="#3d85c6">773-834-1757</font></i></font></div><div><b><i><a href="mailto:mmarre@ttic.edu" target="_blank"><font face="arial, helvetica, sans-serif" size="1">mmarre@ttic.edu</font></a></i></b></div></div></div></div></div></div>
</blockquote></div>
</div>
</blockquote></div></div>
</blockquote></div></div>
</blockquote></div></div>