<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div class=""><span class="" style="caret-color: rgb(60, 64, 67); letter-spacing: 0.20000000298023224px; white-space: pre-wrap;"><i class="">Department of Computer Science Seminar</i></span></div>
<div class=""><span class="" style="caret-color: rgb(60, 64, 67); letter-spacing: 0.20000000298023224px; white-space: pre-wrap;"><i class=""><br class="">
</i></span></div>
<div class=""><span class=""><span class="" style="caret-color: rgb(60, 64, 67); letter-spacing: 0.20000000298023224px; white-space: pre-wrap;"><b class="">Mina Lee
</b></span></span></div>
<div class=""><span class=""><span class="" style="caret-color: rgb(60, 64, 67); letter-spacing: 0.20000000298023224px; white-space: pre-wrap;"><b class="">PhD Candidate</b></span></span></div>
<div class=""><span class=""><span class="" style="caret-color: rgb(60, 64, 67); letter-spacing: 0.20000000298023224px; white-space: pre-wrap;"><b class="">Stanford University
</b></span></span></div>
<div class="">
<div class=""><b class=""><br class="">
</b></div>
<div class=""><b class="">Monday, February 20th </b></div>
<div class=""><b class="">3:00pm - 4:00pm </b></div>
<div class=""><b class="">In Person: John Crerar Library 390</b></div>
<div class=""><b class=""><br class="">
</b></div>
<div class=""><b class="">Zoom: </b></div>
<div class=""><b class=""><span class="">
<div class=""><a href="https://uchicagogroup.zoom.us/j/91201511461?pwd=N1BiR21WNUgrZzlZZXB5bDNTNXdnZz09" class="">https://uchicagogroup.zoom.us/j/91201511461?pwd=N1BiR21WNUgrZzlZZXB5bDNTNXdnZz09</a></div>
<div class=""><br class="">
</div>
<div class="">Meeting ID: 912 0151 1461</div>
<div class="">Passcode: 885562</div>
</span></b></div>
<div class=""><b class=""><i class=""><br class="">
</i></b></div>
<div class=""><b class="" style="caret-color: rgb(60, 64, 67); letter-spacing: 0.20000000298023224px; white-space: pre-wrap;">Title: Designing and evaluating language models for human interaction</b><br class="" style="caret-color: rgb(60, 64, 67); letter-spacing: 0.20000000298023224px; white-space: pre-wrap;">
<br class="" style="caret-color: rgb(60, 64, 67); letter-spacing: 0.20000000298023224px; white-space: pre-wrap;">
<b class="" style="caret-color: rgb(60, 64, 67); letter-spacing: 0.20000000298023224px; white-space: pre-wrap;">Abstract:</b><span class="" style="caret-color: rgb(60, 64, 67); letter-spacing: 0.20000000298023224px; white-space: pre-wrap; background-color: rgb(255, 255, 255);">
</span><br class="" style="caret-color: rgb(60, 64, 67); letter-spacing: 0.20000000298023224px; white-space: pre-wrap;">
<span class="" style="caret-color: rgb(60, 64, 67); letter-spacing: 0.20000000298023224px; white-space: pre-wrap; background-color: rgb(255, 255, 255);">In this talk, I will present my research on designing and evaluating language models (LMs) for human-LM
interaction. Despite the recent advancements in LMs, most LMs are not optimized for, nor are they evaluated on, real-world usage with human interaction. To address this, I will first describe how we can support human editing needs by enabling any LM to perform
text infilling at any position in a document (i.e., “fill in the blanks”). I will then introduce CoAuthor, a platform for capturing human-LM interaction in collaborative writing as rich, replayable, keystroke-level interaction traces. With the platform, I
demonstrate how collecting a large interaction dataset and analyzing the traces provide unique insights into LM capabilities regarding language, ideation, and collaboration. Lastly, I will propose a new framework, HALIE (Human-AI Language-based Interaction
Evaluation), that defines the components of interactive systems and evaluation metrics for human-LM interaction beyond writing. I will conclude by discussing open challenges and future directions in this field.</span><br class="" style="caret-color: rgb(60, 64, 67); letter-spacing: 0.20000000298023224px; white-space: pre-wrap;">
<span class="" style="caret-color: rgb(60, 64, 67); letter-spacing: 0.20000000298023224px; white-space: pre-wrap; background-color: rgb(255, 255, 255);"> </span><br class="" style="caret-color: rgb(60, 64, 67); letter-spacing: 0.20000000298023224px; white-space: pre-wrap;">
<b class="" style="caret-color: rgb(60, 64, 67); letter-spacing: 0.20000000298023224px; white-space: pre-wrap;">Bio:</b><br class="" style="caret-color: rgb(60, 64, 67); letter-spacing: 0.20000000298023224px; white-space: pre-wrap;">
<span class="" style="caret-color: rgb(60, 64, 67); letter-spacing: 0.20000000298023224px; white-space: pre-wrap; background-color: rgb(255, 255, 255);">Mina Lee is a final-year Ph.D. candidate at Stanford University, advised by Professor Percy Liang. Her research
goal is to leverage language models to enhance our productivity and creativity and understand how these models change the way we write. She has built various writing assistants, including an autocomplete system, a contextual thesaurus system, and a creative
story writing system, as well as evaluated language models based on their ability to interact with humans and augment human capabilities. She was named one of MIT Technology Review’s Korean Innovators under 35 in 2022, and her work has been published in top-tier
venues in natural language processing (e.g., ACL and NAACL), machine learning (e.g., NeurIPS), and human-computer interaction (e.g., CHI). Her recent work on human-AI collaborative writing received an Honorable Mention Award at CHI 2022 and was featured in
various media outlets including The Economist.</span></div>
</div>
<div class=""><br class="">
</div>
<div class=""><img apple-inline="yes" id="CEE4ABF2-EDE7-42AA-AED2-1C6F6EA09825" class="" src="cid:152A9259-E65C-4935-BDDE-BD0E16D968F1"></div>
<div class="">
<div class=""><br class="">
</div>
<div class="">---</div>
</div>
<div class="">
<div>Holly Santos<br class="">
Executive Assistant to Michael J. Franklin, Chairman<br class="">
Department of Computer Science<br class="">
The University of Chicago<br class="">
5730 S Ellis Ave-217 Chicago, IL 60637<br class="">
P: 773-834-8977<br class="">
<a href="mailto:hsantos@uchicago.edu" class="">hsantos@uchicago.edu</a><br class="">
</div>
</div>
<br class="">
</body>
</html>