<div dir="ltr"><div dir="ltr"><div class="gmail_default" style=""><div class="gmail_default" style="font-size:small"><font style="font-family:arial,sans-serif;color:rgb(0,0,0);vertical-align:inherit"><font style="vertical-align:inherit"><b>When:</b> </font></font><font style="font-family:arial,sans-serif;vertical-align:inherit"><font style="vertical-align:inherit"><font style="color:rgb(0,0,0)"> Monday, September 18</font><span class="gmail_default" style="color:rgb(0,0,0)">, 2023</span><font style="color:rgb(0,0,0)"> at</font><b style="color:rgb(0,0,0)"> <u>11:30</u></b><b><u><font color="#000000"> a</font></u></b><b><u><font color="#000000">m CT</font></u><font color="#000000"> </font></b></font></font><br></div><div class="gmail_default" style=""><p style="font-size:small;color:rgb(80,0,80);font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><b style="font-family:arial,sans-serif"><font color="#500050"><br></font></b></p><p style="font-size:small;color:rgb(80,0,80);font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><b style="font-family:arial,sans-serif"><font color="#500050">Where: </font></b><font color="#000000" style="font-family:arial,sans-serif">Talk will be given </font><font color="#000000" style="font-family:arial,sans-serif;font-weight:bold"><u>live, in-person</u></font><font style="font-family:arial,sans-serif;font-weight:bold"> </font><span style="font-family:arial,sans-serif">at</span><br></p><p class="MsoNormal" style="font-size:small;margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font color="#500050"> </font><font color="#000000"> TTIC, 6045 S. Kenwood Avenue</font></font></p><p class="MsoNormal" style="font-size:small;margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> 5th Floor, Room 530<b> </b></font></p><p class="MsoNormal" style="font-size:small;margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"><b><br></b></font></p><p class="MsoNormal" style="font-size:small;margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><b style="color:rgb(60,64,67);font-family:arial,sans-serif;letter-spacing:0.2px">Virtually:</b><span style="font-family:arial,sans-serif;letter-spacing:0.2px"><font color="#3c4043"> <i> via</i> Panopto <a href="https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=f2ca8655-9ba3-4cb9-b45e-b07e003e833c" target="_blank"><b>livestream</b></a></font></span><font face="arial, sans-serif" style="color:rgb(80,0,80)"><b><br></b></font></p><p class="MsoNormal" style="font-size:small;margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><br></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font size="4"><font style="font-family:arial,sans-serif;vertical-align:inherit"><font style="vertical-align:inherit"><b style="">Who: </b> <font color="#500050" style=""> </font></font></font><span style="color:rgb(60,64,67);letter-spacing:0.2px"><font face="arial, sans-serif" style="">Kanishka Misra, <b style="background-color:rgb(255,255,0)">UT Austi</b><b style="background-color:rgb(255,255,0)">n</b></font></span></font><span style="color:rgb(0,0,0);font-size:16px;background-color:rgb(255,255,0)">(Linguistics)</span></p><p class="MsoNormal" style="font-size:small;margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><span style="color:rgb(34,34,34)"><br></span></p><div class="MsoNormal" align="center" style="font-size:small;margin:0in 0in 8pt;color:rgb(80,0,80);text-align:center;line-height:15.6933px"><hr size="2" width="100%" align="center"></div><div style="font-size:small"><span id="m_4134175150425154507m_-9092562243106946680m_4054032379435658787m_-9136831640199693879m_159455563385457554m_6781907569219318678m_2917498327190496145m_6658573597948212504m_6522692326762214755m_4171898695171381519m_-7622701669742356547m_-6967680496673197251m_8731442605182037865m_-6294525540622960617gmail-docs-internal-guid-fa76653a-7fff-1406-a32b-0d9b384fe884" style="color:rgb(0,0,0)"><font face="arial, sans-serif"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><b>Title: </b>Analyzing Robust Conceptual Knowledge and Property Inheritance in Language Models</p><div><br></div><div><b>Abstract: </b>A characteristic feature of human semantic cognition is the ability to not only store and retrieve the properties of concepts observed through experience, but to also facilitate the inheritance of properties (e.g., can breathe) from superordinate concepts (animal) to their subordinates (dog)---i.e. demonstrate property inheritance. In this talk, I will present COMPS, a collection of minimal pair sentences that jointly tests pre-trained language models (PLMs) on their ability to attribute properties to concepts and their ability to demonstrate property inheritance behavior. Analyses of 31 different PLMs on COMPS reveal that they can easily distinguish between concepts on the basis of a property when they are trivially different but find it relatively difficult when concepts are related on the basis of explicit knowledge representations. Further analyses find that PLMs can show behaviors suggesting successful property inheritance in simple contexts, but fail in the presence of distracting information, which decreases the performance of many models, sometimes even below chance. This lack of robustness in demonstrating simple reasoning raises important questions about PLMs’ capacity to make correct inferences even when they appear to possess the prerequisite knowledge. I will also discuss preliminary results on a potential few-shot setting of this task, complemented with additional controls for positional heuristics.</div><div><br></div><div><b>Bio: </b>Kanishka Misra is a postdoc at the department of linguistics at UT Austin with Kyle Mahowald. He received his PhD at Purdue University with Julia Rayz and Allyson Ettinger (UChicago). His research focuses on analyzing and evaluating language models and neural networks from the perspective of semantic cognition, analyzing their ability to extract conceptual meaning and demonstrate reasoning compatible with the nuances of human conceptual knowledge. He and his collaborators have been recognized with a best paper award at EACL 2023 and an outstanding paper award at ACL 2023.</div><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><br><br style="font-size:16px"></p></font></span></div><div style="font-size:small"><div><div><b style="color:rgb(0,0,0);font-family:arial,sans-serif">Host: </b><a href="mailto:avrim@ttic.edu" target="_blank" style="font-family:arial,sans-serif">Avrim Blum</a><br></div></div></div></div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small"><br></div></div><div><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><span style="font-family:arial,helvetica,sans-serif;font-size:x-small">Mary C. Marre</span><br></div><div><div><font face="arial, helvetica, sans-serif" size="1">Faculty Administrative Support</font></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6" size="1"><b>Toyota Technological Institute</b></font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6" size="1">6045 S. Kenwood Avenue, Rm 517</font></i></div><div><font size="1"><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Chicago, IL 60637</font></i><br></font></div><div><font size="1"><i><font face="arial, helvetica, sans-serif" color="#3d85c6">773-834-1757</font></i></font></div><div><b><i><a href="mailto:mmarre@ttic.edu" target="_blank"><font face="arial, helvetica, sans-serif" size="1">mmarre@ttic.edu</font></a></i></b></div></div></div></div></div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Sep 14, 2023 at 10:54 PM Mary Marre <<a href="mailto:mmarre@ttic.edu">mmarre@ttic.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div style="font-size:small"><div><font style="font-family:arial,sans-serif;color:rgb(0,0,0);vertical-align:inherit"><font style="vertical-align:inherit"><b>When:</b> </font></font><font style="font-family:arial,sans-serif;vertical-align:inherit"><font style="vertical-align:inherit"><font style="color:rgb(0,0,0)"> Monday, September 18</font><span class="gmail_default" style="color:rgb(0,0,0)">, 2023</span><font style="color:rgb(0,0,0)"> at</font><b style="color:rgb(0,0,0)"> <u>11:30</u></b><b><u><font color="#000000"> a</font></u></b><b><u><font color="#000000">m CT</font></u><font color="#000000"> </font></b></font></font><br></div><div><p style="color:rgb(80,0,80);font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><b style="font-family:arial,sans-serif"><font color="#500050"><br></font></b></p><p style="color:rgb(80,0,80);font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><b style="font-family:arial,sans-serif"><font color="#500050">Where: </font></b><font color="#000000" style="font-family:arial,sans-serif">Talk will be given </font><font color="#000000" style="font-family:arial,sans-serif;font-weight:bold"><u>live, in-person</u></font><font style="font-family:arial,sans-serif;font-weight:bold"> </font><span style="font-family:arial,sans-serif">at</span><br></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font color="#500050"> </font><font color="#000000"> TTIC, 6045 S. Kenwood Avenue</font></font></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> 5th Floor, Room 530<b> </b></font></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"><b><br></b></font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><b style="color:rgb(60,64,67);font-family:arial,sans-serif;letter-spacing:0.2px">Virtually:</b><span style="font-family:arial,sans-serif;letter-spacing:0.2px"><font color="#3c4043"> <i> via</i> Panopto <a href="https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=f2ca8655-9ba3-4cb9-b45e-b07e003e833c" target="_blank"><b>livestream</b></a></font></span><font face="arial, sans-serif" style="color:rgb(80,0,80)"><b><br></b></font></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><br></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font style="font-family:arial,sans-serif;vertical-align:inherit"><font style="vertical-align:inherit"><b>Who: </b> <font color="#500050"> </font><font color="#000000"><font color="#500050"> </font></font></font></font><span style="color:rgb(60,64,67);letter-spacing:0.2px"><font face="arial, sans-serif">Kanishka Misra, Purdue</font></span></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><span style="color:rgb(34,34,34)"><br></span></p><div class="MsoNormal" align="center" style="margin:0in 0in 8pt;color:rgb(80,0,80);text-align:center;line-height:15.6933px"><hr size="2" width="100%" align="center"></div><div><span id="m_4134175150425154507m_-9092562243106946680m_4054032379435658787m_-9136831640199693879m_159455563385457554m_6781907569219318678m_2917498327190496145m_6658573597948212504m_6522692326762214755m_4171898695171381519m_-7622701669742356547m_-6967680496673197251m_8731442605182037865m_-6294525540622960617gmail-docs-internal-guid-fa76653a-7fff-1406-a32b-0d9b384fe884" style="color:rgb(0,0,0)"><font face="arial, sans-serif"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><b>Title: </b>Analyzing Robust Conceptual Knowledge and Property Inheritance in Language Models</p><div><br></div><div><b>Abstract: </b>A characteristic feature of human semantic cognition is the ability to not only store and retrieve the properties of concepts observed through experience, but to also facilitate the inheritance of properties (e.g., can breathe) from superordinate concepts (animal) to their subordinates (dog)---i.e. demonstrate property inheritance. In this talk, I will present COMPS, a collection of minimal pair sentences that jointly tests pre-trained language models (PLMs) on their ability to attribute properties to concepts and their ability to demonstrate property inheritance behavior. Analyses of 31 different PLMs on COMPS reveal that they can easily distinguish between concepts on the basis of a property when they are trivially different but find it relatively difficult when concepts are related on the basis of explicit knowledge representations. Further analyses find that PLMs can show behaviors suggesting successful property inheritance in simple contexts, but fail in the presence of distracting information, which decreases the performance of many models, sometimes even below chance. This lack of robustness in demonstrating simple reasoning raises important questions about PLMs’ capacity to make correct inferences even when they appear to possess the prerequisite knowledge. I will also discuss preliminary results on a potential few-shot setting of this task, complemented with additional controls for positional heuristics.</div><div><br></div><div><b>Bio: </b>Kanishka Misra is a postdoc at the department of linguistics at UT Austin with Kyle Mahowald. He received his PhD at Purdue University with Julia Rayz and Allyson Ettinger (UChicago). His research focuses on analyzing and evaluating language models and neural networks from the perspective of semantic cognition, analyzing their ability to extract conceptual meaning and demonstrate reasoning compatible with the nuances of human conceptual knowledge. He and his collaborators have been recognized with a best paper award at EACL 2023 and an outstanding paper award at ACL 2023.</div><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><br><br style="font-size:16px"></p></font></span></div><div><div><div><b style="color:rgb(0,0,0);font-family:arial,sans-serif">Host: </b><a href="mailto:avrim@ttic.edu" style="font-family:arial,sans-serif" target="_blank">Avrim Blum</a><br></div></div></div></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div></div><div><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><span style="font-family:arial,helvetica,sans-serif;font-size:x-small">Mary C. Marre</span><br></div><div><div><font face="arial, helvetica, sans-serif" size="1">Faculty Administrative Support</font></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6" size="1"><b>Toyota Technological Institute</b></font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6" size="1">6045 S. Kenwood Avenue, Rm 517</font></i></div><div><font size="1"><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Chicago, IL 60637</font></i><br></font></div><div><font size="1"><i><font face="arial, helvetica, sans-serif" color="#3d85c6">773-834-1757</font></i></font></div><div><b><i><a href="mailto:mmarre@ttic.edu" target="_blank"><font face="arial, helvetica, sans-serif" size="1">mmarre@ttic.edu</font></a></i></b></div></div></div></div></div></div>
</blockquote></div></div>