<div dir="ltr"><div><div class="gmail_default" style="font-family:georgia,serif;font-size:small;color:rgb(0,0,0)"><div><div class="gmail_default"><font face="georgia, serif" color="#000000"><b><font style="vertical-align:inherit"><font style="vertical-align:inherit">When:    </font></font></b><font style="vertical-align:inherit"><font style="vertical-align:inherit"> Monday, May 12th at <b><font style="background-color:rgb(255,255,0)">11:30 AM CT</font></b></font></font></font></div><div><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"> </font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"><font style="vertical-align:inherit"><font style="vertical-align:inherit"><b>Where:    </b><span class="gmail_default"><b></b></span><span class="gmail_default">T</span></font></font>alk will be given <font style="font-weight:bold"><u style="background-color:rgb(255,255,0)">live, in-person</u></font><font style="font-weight:bold"> </font>at</font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000" face="georgia, serif">                     TTIC, 6045 S. Kenwood Avenue</font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000" face="georgia, serif">                     5th Floor, Room 530<b> </b></font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000" face="georgia, serif"><b><br></b></font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"><span class="gmail_default"><b>Virtually:  </b>via Panopto<b> </b>(<a href="https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=bc9cc75b-866b-4915-a52a-b2480121118b" target="_blank">Livestream</a>) </span><b><br></b></font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"> </font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"><b><font style="vertical-align:inherit"><font style="vertical-align:inherit">Who:      <span class="gmail_default"></span>   </font></font></b>Richard Zemel, Columbia University</font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"><br></font></p></div><div><font face="georgia, serif" color="#000000"><b>Title:        </b>Improving the Diversity and Evaluation of Large Models</font></div><div><p></p><div><font face="georgia, serif" color="#000000"><b>Abstract: </b>Given their increasing ubiquity it is of the utmost importance to ensure that AI systems can represent a diverse set of human values and perspectives. Meanwhile progress on this type of control of Large Language Models (LLMs) has been hampered by a fundamental underlying problem in generative models: the difficulty of evaluating their behavior. With their enormous range of inputs and responses and the wide variety of intended applications and dimensions along which to judge the responses, evaluation is extremely challenging.  I will describe two strands of research to help address these needs. </font></div><div><font face="georgia, serif" color="#000000"><br></font></div><div><font face="georgia, serif" color="#000000">The first aims to develop approaches that permit reliable and fine-grained control over LLM generations, which allow them to be steered towards a broad set of viewpoints. The second aims to rigorously combine model-generated labels of an LLM's responses with a small pool of ground-truth labels, to allow estimates of different aspects of the range of responses. I will describe quantitative and qualitative evaluations of these methods on a number of benchmark datasets, spanning public opinion data, educational tutoring, economic disparities, and LLM toxicity and summarization quality.<span class="gmail_default"></span></font></div></div><div><font face="georgia, serif" color="#000000"><br></font><div><font face="georgia, serif" color="#000000"><b><span class="gmail_default"></span>Bio:</b>  Richard Zemel is the Trianthe Dakolias Professor of Engineering and Applied Science in the Computer Science Department at Columbia University. He is the Director of the NSF AI Institute for Artificial and Natural Intelligence (ARNI). He was the Co-Founder and inaugural Research Director of the Vector Institute for Artificial Intelligence. He is a Canadian Institute for Advanced Research AI Chair, an Amazon Scholar, and is on the Advisory Board of the Neural Information Processing Society.  His research contributions include foundational work on systems that learn useful representations of data with little or no supervision; graph-based machine learning; and algorithms for fair and robust machine learning.</font></div><div><font face="georgia, serif" color="#000000"><br></font></div><div><p style="box-sizing:border-box;border-radius:0px;margin:0px 0px 10px"><font face="georgia, serif" color="#000000"><span style="box-sizing:border-box;border-radius:0px;font-weight:700">Hos</span><b><span style="box-sizing:border-box;border-radius:0px">t:<span class="gmail_default"> <a href="mailto:avrim@ttic.edu" target="_blank">Avrim Blum</a></span></span></b></font></p></div></div><br clear="all"></div><div><br></div><span class="gmail_signature_prefix">--</span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><b style="color:rgb(34,34,34);font-family:Arial,Helvetica,sans-serif"><font color="#3d85c6">Brandie Jones </font></b><div style="color:rgb(34,34,34);font-family:Arial,Helvetica,sans-serif"><div><div><font color="#3d85c6"><b><i>Executive </i></b></font><b style="color:rgb(61,133,198)"><i>Administrative Assistant</i></b></div></div><div><font color="#3d85c6">Toyota Technological Institute</font></div><div><font color="#3d85c6">6045 S. Kenwood Avenue</font></div><div><font color="#3d85c6">Chicago, IL  60637</font></div></div><div style="color:rgb(34,34,34);font-family:Arial,Helvetica,sans-serif"><font color="#3d85c6"><a href="http://www.ttic.edu/" target="_blank">www.ttic.edu</a> </font></div><br class="gmail-Apple-interchange-newline"></div></div></div></div></div>