<div dir="ltr"><div dir="ltr"><div class="gmail_default" style="font-size:small"><div class="gmail_default"><div class="gmail_default"><div class="gmail_default" style="color:rgb(80,0,80)"><font style="font-family:arial,sans-serif;color:rgb(0,0,0);vertical-align:inherit"><font style="vertical-align:inherit"><b>When:</b> </font></font><font style="font-family:arial,sans-serif;vertical-align:inherit"><font style="vertical-align:inherit"><font style="color:rgb(0,0,0)"> </font><span class="gmail_default" style="color:rgb(0,0,0)">Thursday, March 14<span class="gmail_default">, </span>2024</span><font style="color:rgb(0,0,0)"> at</font><b style="color:rgb(0,0,0)"> <u><font style="background-color:rgb(255,255,0)">11:00</font></u></b><b><u><font color="#000000" style="background-color:rgb(255,255,0)"> a</font></u></b><b><u><font color="#000000" style="background-color:rgb(255,255,0)">m CT</font></u><font color="#000000"> </font></b></font></font><br></div><div class="gmail_default"><p style="color:rgb(80,0,80);font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><b style="font-family:arial,sans-serif"><font color="#500050"><br></font></b></p><p style="color:rgb(80,0,80);font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><b style="font-family:arial,sans-serif"><font color="#500050">Where: </font></b><font color="#000000" style="font-family:arial,sans-serif">Talk will be given </font><font color="#000000" style="font-family:arial,sans-serif;font-weight:bold"><u>live, in-person</u></font><font style="font-family:arial,sans-serif;font-weight:bold"> </font><span style="font-family:arial,sans-serif">at</span><br></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font color="#500050"> </font><font color="#000000"> TTIC, 6045 S. Kenwood Avenue</font></font></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> 5th Floor, Room 530<b> </b></font></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><b><span style="color:black"><br></span></b></font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><b style="color:rgb(60,64,67);letter-spacing:0.2px">Virtually:</b><span style="color:rgb(60,64,67);letter-spacing:0.2px"> <i>via </i>Panopto </span>(<b><a href="https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=d9a754e1-4d1f-4d31-ace8-b12c01622b72" target="_blank">livestream</a></b><span style="color:rgb(60,64,67);letter-spacing:0.2px">)</span><br></font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><span style="color:rgb(60,64,67);letter-spacing:0.2px"><font face="arial, sans-serif"></font><font face="georgia, serif"><b><font size="1"> *limited access: see info below</font></b></font></span></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><span style="color:rgb(60,64,67);letter-spacing:0.2px"><font face="arial, sans-serif"></font><font face="georgia, serif"><b><font size="1"> </font></b></font></span></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font style="color:rgb(80,0,80);font-family:arial,sans-serif;vertical-align:inherit"><font style="vertical-align:inherit"><b>Who: </b> <font color="#500050"> </font><font color="#000000"><font color="#500050"> </font></font></font></font><font color="#000000">Shuyan Zhou, Carnegie Mellon University</font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><br></p></div></div><div class="gmail_default"><div dir="ltr"><div><div class="MsoNormal" align="center" style="margin:0in 0in 8pt;text-align:center;line-height:15.6933px;font-size:11pt;font-family:Calibri,sans-serif"><hr size="3" width="100%" noshade align="center" style="color:rgb(46,116,181)"></div></div><div><div><font face="arial, sans-serif"><b>Title: </b></font>Solving Real-World Tasks with AI Agents<br><font face="arial, sans-serif"><b><br></b></font></div><div><font face="arial, sans-serif"><b>Abstract: </b></font>For years, my dream has been to create autonomous AI agents capable of carrying out tedious procedural tasks (e.g., arranging conference travel), allowing me to focus on more creative and exciting tasks. Modern AI models, especially large language models (LLMs) like ChatGPT, have suddenly brought us much closer to achieving such AI agents. But, has my dream already come true? In this talk, I will answer this question by delving into my systematic evaluation of AI agents in realistic tasks. The evaluation uncovers many critical limitations of AI agents, such as tool use, abstract reasoning, and knowledge cutoff. It suggests that LLMs are crucial yet early steps towards AI autonomy. To address these challenges, I will introduce my research of a more suitable “language” for AIs, which overcomes the inherent limitations of using natural language for task solving. Then, I will discuss my work on teaching AI agents to learn new tools by reading the tool documentation rather than direct demonstrations. Finally, I will discuss my future plans for comprehensive AI agent evaluations, agent foundations, and the application of AI agents to critical sectors in the real world.</div><div><font face="arial, sans-serif"><b> </b></font> <font face="arial, sans-serif"><br><b>Bio: </b></font>Shuyan Zhou is a final-year PhD student at the Language Technologies Institute at CMU, advised by Graham Neubig. Her research in NLP and AI focuses on creating AI agents for real-world tasks, such as using computers and generating code. Her work has been recognized at top natural language processing and machine learning conferences and journals such as ICLR, ICML, ACL, EMNLP, and TACL. You can find more about her at <a href="https://shuyanzhou.com/" target="_blank">https://shuyanzhou.com</a> </div></div><div><font face="arial, sans-serif"><br></font></div></div><div><div id="m_5973546232189203326m_-6976229087647104413m_7917979885129397482m_-5423657134431402203m_-1337599008586739890m_8237382617653311322m_-1231130334284673048m_-1282025577005441955m_-1973358356214118865m_-5815637555669013367m_1779572315514282115m_-4485402625451270420m_1520697528942856564m_5948359943660736735m_-4789039193346764527m_3599676094611771654m_8264976978369198918m_7474850050874458051m_5107577024390010371m_253820422674989860m_3983419646637522536m_-7220900540036838011gmail-:qg" role="button" aria-label="Show trimmed content" aria-expanded="false"><font face="arial, sans-serif"><b>Host: </b><a href="mailto:mwalter@ttic.edu" target="_blank"><b>Matthew Walter</b></a></font></div></div></div></div><div class="gmail_default"><br></div><div class="gmail_default"><div><font color="#0000ff" face="arial black, sans-serif">*</font><font face="tahoma, sans-serif">Access to this livestream is limited to TTIC / UChicago (press panopto link and sign in to your UChicago account with </font><span style="font-family:tahoma,sans-serif;color:rgb(29,29,33);font-weight:600">CNetID</span><font face="tahoma, sans-serif">). </font><br></div><div><br></div><div><br></div></div></div><div><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><span style="font-family:arial,helvetica,sans-serif;font-size:x-small">Mary C. Marre</span><br></div><div><div><font face="arial, helvetica, sans-serif" size="1">Faculty Administrative Support</font></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6" size="1"><b>Toyota Technological Institute</b></font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6" size="1">6045 S. Kenwood Avenue, Rm 517</font></i></div><div><font size="1"><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Chicago, IL 60637</font></i><br></font></div><div><font size="1"><i><font face="arial, helvetica, sans-serif" color="#3d85c6">773-834-1757</font></i></font></div><div><b><i><a href="mailto:mmarre@ttic.edu" target="_blank"><font face="arial, helvetica, sans-serif" size="1">mmarre@ttic.edu</font></a></i></b></div></div></div></div></div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Mar 13, 2024 at 12:41 PM Mary Marre <<a href="mailto:mmarre@ttic.edu">mmarre@ttic.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div style="font-size:small"><div><div><div style="color:rgb(80,0,80)"><font style="font-family:arial,sans-serif;color:rgb(0,0,0);vertical-align:inherit"><font style="vertical-align:inherit"><b>When:</b> </font></font><font style="font-family:arial,sans-serif;vertical-align:inherit"><font style="vertical-align:inherit"><font style="color:rgb(0,0,0)"> </font><span class="gmail_default" style="color:rgb(0,0,0)">Thursday, March 14<span class="gmail_default">, </span>2024</span><font style="color:rgb(0,0,0)"> at</font><b style="color:rgb(0,0,0)"> <u><font style="background-color:rgb(255,255,0)">11:00</font></u></b><b><u><font color="#000000" style="background-color:rgb(255,255,0)"> a</font></u></b><b><u><font color="#000000" style="background-color:rgb(255,255,0)">m CT</font></u><font color="#000000"> </font></b></font></font><br></div><div><p style="color:rgb(80,0,80);font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><b style="font-family:arial,sans-serif"><font color="#500050"><br></font></b></p><p style="color:rgb(80,0,80);font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><b style="font-family:arial,sans-serif"><font color="#500050">Where: </font></b><font color="#000000" style="font-family:arial,sans-serif">Talk will be given </font><font color="#000000" style="font-family:arial,sans-serif;font-weight:bold"><u>live, in-person</u></font><font style="font-family:arial,sans-serif;font-weight:bold"> </font><span style="font-family:arial,sans-serif">at</span><br></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font color="#500050"> </font><font color="#000000"> TTIC, 6045 S. Kenwood Avenue</font></font></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> 5th Floor, Room 530<b> </b></font></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><b><span style="color:black"><br></span></b></font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><b style="color:rgb(60,64,67);letter-spacing:0.2px">Virtually:</b><span style="color:rgb(60,64,67);letter-spacing:0.2px"> <i>via </i>Panopto </span>(<b><a href="https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=d9a754e1-4d1f-4d31-ace8-b12c01622b72" target="_blank">livestream</a></b><span style="color:rgb(60,64,67);letter-spacing:0.2px">)</span><br></font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><span style="color:rgb(60,64,67);letter-spacing:0.2px"><font face="arial, sans-serif"></font><font face="georgia, serif"><b><font size="1"> *limited access: see info below</font></b></font></span></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><span style="color:rgb(60,64,67);letter-spacing:0.2px"><font face="arial, sans-serif"></font><font face="georgia, serif"><b><font size="1"> </font></b></font></span></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font style="color:rgb(80,0,80);font-family:arial,sans-serif;vertical-align:inherit"><font style="vertical-align:inherit"><b>Who: </b> <font color="#500050"> </font><font color="#000000"><font color="#500050"> </font></font></font></font><font color="#000000">Shuyan Zhou, Carnegie Mellon University</font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><br></p></div></div><div><div dir="ltr"><div><div class="MsoNormal" align="center" style="margin:0in 0in 8pt;text-align:center;line-height:15.6933px;font-size:11pt;font-family:Calibri,sans-serif"><hr size="3" width="100%" noshade align="center" style="color:rgb(46,116,181)"></div></div><div><div><font face="arial, sans-serif"><b>Title: </b></font>Solving Real-World Tasks with AI Agents<br><font face="arial, sans-serif"><b><br></b></font></div><div><font face="arial, sans-serif"><b>Abstract: </b></font>For years, my dream has been to create autonomous AI agents capable of carrying out tedious procedural tasks (e.g., arranging conference travel), allowing me to focus on more creative and exciting tasks. Modern AI models, especially large language models (LLMs) like ChatGPT, have suddenly brought us much closer to achieving such AI agents. But, has my dream already come true? In this talk, I will answer this question by delving into my systematic evaluation of AI agents in realistic tasks. The evaluation uncovers many critical limitations of AI agents, such as tool use, abstract reasoning, and knowledge cutoff. It suggests that LLMs are crucial yet early steps towards AI autonomy. To address these challenges, I will introduce my research of a more suitable “language” for AIs, which overcomes the inherent limitations of using natural language for task solving. Then, I will discuss my work on teaching AI agents to learn new tools by reading the tool documentation rather than direct demonstrations. Finally, I will discuss my future plans for comprehensive AI agent evaluations, agent foundations, and the application of AI agents to critical sectors in the real world.</div><div><font face="arial, sans-serif"><b> </b></font> <font face="arial, sans-serif"><br><b>Bio: </b></font>Shuyan Zhou is a final-year PhD student at the Language Technologies Institute at CMU, advised by Graham Neubig. Her research in NLP and AI focuses on creating AI agents for real-world tasks, such as using computers and generating code. Her work has been recognized at top natural language processing and machine learning conferences and journals such as ICLR, ICML, ACL, EMNLP, and TACL. You can find more about her at <a href="https://shuyanzhou.com/" target="_blank">https://shuyanzhou.com</a> </div></div><div><font face="arial, sans-serif"><br></font></div></div><div><div id="m_5973546232189203326m_-6976229087647104413m_7917979885129397482m_-5423657134431402203m_-1337599008586739890m_8237382617653311322m_-1231130334284673048m_-1282025577005441955m_-1973358356214118865m_-5815637555669013367m_1779572315514282115m_-4485402625451270420m_1520697528942856564m_5948359943660736735m_-4789039193346764527m_3599676094611771654m_8264976978369198918m_7474850050874458051m_5107577024390010371m_253820422674989860m_3983419646637522536m_-7220900540036838011gmail-:qg" role="button" aria-label="Show trimmed content" aria-expanded="false"><font face="arial, sans-serif"><b>Host: </b><a href="mailto:mwalter@ttic.edu" target="_blank"><b>Matthew Walter</b></a></font></div></div></div></div><div><br></div><div><div><font color="#0000ff" face="arial black, sans-serif">*</font><font face="tahoma, sans-serif">Access to this livestream is limited to TTIC / UChicago (press panopto link and sign in to your UChicago account with </font><span style="font-family:tahoma,sans-serif;color:rgb(29,29,33);font-weight:600">CNetID</span><font face="tahoma, sans-serif">). </font><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div></div></div><div><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><span style="font-family:arial,helvetica,sans-serif;font-size:x-small">Mary C. Marre</span><br></div><div><div><font face="arial, helvetica, sans-serif" size="1">Faculty Administrative Support</font></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6" size="1"><b>Toyota Technological Institute</b></font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6" size="1">6045 S. Kenwood Avenue, Rm 517</font></i></div><div><font size="1"><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Chicago, IL 60637</font></i><br></font></div><div><font size="1"><i><font face="arial, helvetica, sans-serif" color="#3d85c6">773-834-1757</font></i></font></div><div><b><i><a href="mailto:mmarre@ttic.edu" target="_blank"><font face="arial, helvetica, sans-serif" size="1">mmarre@ttic.edu</font></a></i></b></div></div></div></div></div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Mar 7, 2024 at 4:43 PM Mary Marre <<a href="mailto:mmarre@ttic.edu" target="_blank">mmarre@ttic.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div style="font-size:small"><div><div style="color:rgb(80,0,80)"><font style="font-family:arial,sans-serif;color:rgb(0,0,0);vertical-align:inherit"><font style="vertical-align:inherit"><b>When:</b> </font></font><font style="font-family:arial,sans-serif;vertical-align:inherit"><font style="vertical-align:inherit"><font style="color:rgb(0,0,0)"> </font><span class="gmail_default" style="color:rgb(0,0,0)">Thursday, March 14<span class="gmail_default">, </span>2024</span><font style="color:rgb(0,0,0)"> at</font><b style="color:rgb(0,0,0)"> <u><font style="background-color:rgb(255,255,0)">11:00</font></u></b><b><u><font color="#000000" style="background-color:rgb(255,255,0)"> a</font></u></b><b><u><font color="#000000" style="background-color:rgb(255,255,0)">m CT</font></u><font color="#000000"> </font></b></font></font><br></div><div><p style="color:rgb(80,0,80);font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><b style="font-family:arial,sans-serif"><font color="#500050"><br></font></b></p><p style="color:rgb(80,0,80);font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><b style="font-family:arial,sans-serif"><font color="#500050">Where: </font></b><font color="#000000" style="font-family:arial,sans-serif">Talk will be given </font><font color="#000000" style="font-family:arial,sans-serif;font-weight:bold"><u>live, in-person</u></font><font style="font-family:arial,sans-serif;font-weight:bold"> </font><span style="font-family:arial,sans-serif">at</span><br></p><p class="MsoNormal" style="color:rgb(80,0,80);margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font color="#500050"> </font><font color="#000000"> TTIC, 6045 S. Kenwood Avenue</font></font></p><p class="MsoNormal" style="color:rgb(80,0,80);margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> 5th Floor, Room 530<b> </b></font></p><p class="MsoNormal" style="color:rgb(80,0,80);margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><b><span style="color:black"><br></span></b></font></p><p class="MsoNormal" style="color:rgb(80,0,80);margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><b style="color:rgb(60,64,67);letter-spacing:0.2px">Virtually:</b><span style="color:rgb(60,64,67);letter-spacing:0.2px"> <i>via </i>Panopto </span>(<b><a href="https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=d9a754e1-4d1f-4d31-ace8-b12c01622b72" target="_blank">livestream</a></b><span style="color:rgb(60,64,67);letter-spacing:0.2px">)</span><br></font></p><p class="MsoNormal" style="color:rgb(80,0,80);margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><span style="color:rgb(60,64,67);letter-spacing:0.2px"><font face="arial, sans-serif"></font><font face="georgia, serif"><b><font size="1"> *limited access: see info below</font></b></font></span></p><p class="MsoNormal" style="color:rgb(80,0,80);margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><span style="color:rgb(60,64,67);letter-spacing:0.2px"><font face="arial, sans-serif"></font><font face="georgia, serif"><b><font size="1"> </font></b></font></span></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font style="color:rgb(80,0,80);font-family:arial,sans-serif;vertical-align:inherit"><font style="vertical-align:inherit"><b>Who: </b> <font color="#500050"> </font><font color="#000000"><font color="#500050"> </font></font></font></font><font color="#000000">Shuyan Zhou, Carnegie Mellon University</font></p><p class="MsoNormal" style="color:rgb(80,0,80);margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><br></p></div></div><div><div dir="ltr"><div><div class="MsoNormal" align="center" style="margin:0in 0in 8pt;text-align:center;line-height:15.6933px;font-size:11pt;font-family:Calibri,sans-serif"><hr size="3" width="100%" noshade align="center" style="color:rgb(46,116,181)"></div></div><div><div><font face="arial, sans-serif"><b>Title: </b></font>Solving Real-World Tasks with AI Agents<br><font face="arial, sans-serif"><b><br></b></font></div><div><font face="arial, sans-serif"><b>Abstract: </b></font>For years, my dream has been to create autonomous AI agents capable of carrying out tedious procedural tasks (e.g., arranging conference travel), allowing me to focus on more creative and exciting tasks. Modern AI models, especially large language models (LLMs) like ChatGPT, have suddenly brought us much closer to achieving such AI agents. But, has my dream already come true? In this talk, I will answer this question by delving into my systematic evaluation of AI agents in realistic tasks. The evaluation uncovers many critical limitations of AI agents, such as tool use, abstract reasoning, and knowledge cutoff. It suggests that LLMs are crucial yet early steps towards AI autonomy. To address these challenges, I will introduce my research of a more suitable “language” for AIs, which overcomes the inherent limitations of using natural language for task solving. Then, I will discuss my work on teaching AI agents to learn new tools by reading the tool documentation rather than direct demonstrations. Finally, I will discuss my future plans for comprehensive AI agent evaluations, agent foundations, and the application of AI agents to critical sectors in the real world.</div><div><font face="arial, sans-serif"><b> </b></font> <font face="arial, sans-serif"><br><b>Bio: </b></font><span>Shuyan</span> Zhou is a final-year PhD student at the Language Technologies Institute at CMU, advised by Graham Neubig. Her research in NLP and AI focuses on creating AI agents for real-world tasks, such as using computers and generating code. Her work has been recognized at top natural language processing and machine learning conferences and journals such as ICLR, ICML, ACL, EMNLP, and TACL. You can find more about her at <a href="https://shuyanzhou.com/" target="_blank">https://shuyanzhou.com</a> </div></div><div><font face="arial, sans-serif"><br></font></div></div><div><div id="m_5973546232189203326m_-6976229087647104413m_7917979885129397482m_-5423657134431402203m_-1337599008586739890m_8237382617653311322m_-1231130334284673048m_-1282025577005441955m_-1973358356214118865m_-5815637555669013367m_1779572315514282115m_-4485402625451270420m_1520697528942856564m_5948359943660736735m_-4789039193346764527m_3599676094611771654m_8264976978369198918m_7474850050874458051m_5107577024390010371m_253820422674989860m_3983419646637522536m_-7220900540036838011gmail-:qg" role="button" aria-label="Show trimmed content" aria-expanded="false"><font face="arial, sans-serif"><b>Host: </b><a href="mailto:mwalter@ttic.edu" target="_blank"><b>Matthew Walter</b></a></font></div></div></div></div><div style="font-size:small"><br></div><div style="font-size:small"><div><font color="#0000ff" face="arial black, sans-serif">*</font><font face="tahoma, sans-serif">Access to this livestream is limited to TTIC / UChicago (press panopto link and sign in to your UChicago account with </font><span style="font-family:tahoma,sans-serif;color:rgb(29,29,33);font-weight:600">CNetID</span><font face="tahoma, sans-serif">). </font><br></div><div><br></div></div><div style="font-size:small"><br></div><div style="font-size:small"><br></div><div style="font-size:small"><br></div><div><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><span style="font-family:arial,helvetica,sans-serif;font-size:x-small">Mary C. Marre</span><br></div><div><div><font face="arial, helvetica, sans-serif" size="1">Faculty Administrative Support</font></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6" size="1"><b>Toyota Technological Institute</b></font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6" size="1">6045 S. Kenwood Avenue, Rm 517</font></i></div><div><font size="1"><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Chicago, IL 60637</font></i><br></font></div><div><font size="1"><i><font face="arial, helvetica, sans-serif" color="#3d85c6">773-834-1757</font></i></font></div><div><b><i><a href="mailto:mmarre@ttic.edu" target="_blank"><font face="arial, helvetica, sans-serif" size="1">mmarre@ttic.edu</font></a></i></b></div></div></div></div></div></div>
</blockquote></div></div>
</blockquote></div></div>