<div dir="ltr"><div><div class="gmail_default" style="font-family:georgia,serif;font-size:small;color:rgb(0,0,0)"><div><div class="gmail_default"><font face="georgia, serif" color="#000000"><span style="letter-spacing:0.2px"><b>When:    </b>Wednesday, May 7th<b> at </b></span><b style="letter-spacing:0.2px"><span style="background-color:rgb(255,255,0)">11AM CT</span></b></font></div><div class="gmail_default"><div class="gmail_default"><b style="letter-spacing:0.2px"><font face="georgia, serif" color="#000000"><br></font></b></div><div class="gmail_default"><font face="georgia, serif" color="#000000"><b style="letter-spacing:0.2px">Where:   </b>Talk will be given<span style="background-color:rgb(255,255,0)"> </span><span style="background-color:rgb(255,255,0)"><font style="font-weight:bold"><u>live, in-person</u></font><font style="font-weight:bold"> </font></span>at</font></div><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000" face="georgia, serif">                    TTIC, 6045 S. Kenwood Avenue</font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000" face="georgia, serif">                    5th Floor, Room 530<b>  </b></font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><b style="letter-spacing:0.2px"><font face="georgia, serif" color="#000000"><br></font></b></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"><b style="letter-spacing:0.2px">Virtually: </b><span style="letter-spacing:0.2px">via Panopto (</span><a href="https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=c095d3ad-c487-4898-9ee1-b26e0117679e" target="_blank" style="letter-spacing:0.2px">Livestream</a><span style="letter-spacing:0.2px">)</span><br></font></p><div class="gmail_default"><b style="letter-spacing:0.2px"><font face="georgia, serif" color="#000000"><br></font></b></div><div class="gmail_default"><font face="georgia, serif" color="#000000"><span style="letter-spacing:0.2px"><b>Who:      </b></span>Ekdeep Lubana, Harvard</font></div><div class="gmail_default"><span style="letter-spacing:0.2px"><font face="georgia, serif" color="#000000"><br></font></span></div><div class="gmail_default"><font face="georgia, serif" color="#000000"><b style="letter-spacing:0.2px">Title:</b>      Dynamics of Concept Learning and Emergent Abilities in Neural Networks</font></div><div class="gmail_default"><font face="georgia, serif" color="#000000"><br><b style="letter-spacing:0.2px">Abstract: </b>Neural networks' scaling has been argued to yield sudden learning of capabilities (a.k.a. emergent abilities). In this talk, I will first summarize our recent work on formal models that help explain the mechanisms underlying such sudden learning via data scaling, implicating the compositional nature of a task and formation of structured representations that are shared across several tasks involved in the broader data composition. </font></div><div class="gmail_default"><font face="georgia, serif" color="#000000"><br></font></div><div class="gmail_default"><font face="georgia, serif" color="#000000">Then, focusing on in-context learning (ICL)---one such suddenly learned capability---I will demonstrate the precise configurations used for training can lead to learning of fundamentally different algorithms for performing an ICL task. This indicates the phenomenology of ICL established in past work may not be universal. </font></div><div class="gmail_default"><font face="georgia, serif" color="#000000"><br></font></div><div class="gmail_default"><font face="georgia, serif" color="#000000">Further, I will discuss how merely scaling the context size can lead to a crossover between different ICL algorithms used by the model. This can be explained via a competition of algorithms lens, which also yields a new theory on the transient nature of ICL. </font></div><div class="gmail_default"><font face="georgia, serif" color="#000000"><br></font></div><div class="gmail_default"><font face="georgia, serif" color="#000000">The talk will be based on the following series of papers: <a href="https://arxiv.org/abs/2310.09336" target="_blank">https://arxiv.org/abs/2310.09336</a>, <a href="https://arxiv.org/abs/2406.19370" target="_blank">https://arxiv.org/abs/2406.19370</a>, <a href="https://arxiv.org/abs/2408.12578" target="_blank">https://arxiv.org/abs/2408.12578</a>, <a href="https://arxiv.org/abs/2410.08309" target="_blank">https://arxiv.org/abs/2410.08309</a>, <a href="https://arxiv.org/abs/2412.01003" target="_blank">https://arxiv.org/abs/2412.01003</a>, <a href="https://arxiv.org/abs/2501.00070" target="_blank">https://arxiv.org/abs/2501.00070</a>.</font></div><div><font face="georgia, serif" color="#000000"><br></font></div><div><div class="gmail_default"><b><font face="georgia, serif" color="#000000">Host: <a href="mailto:nati@ttic.edu" target="_blank">Nati Srebro</a></font></b></div></div></div></div><div><br></div><span class="gmail_signature_prefix">--</span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><b style="color:rgb(34,34,34);font-family:Arial,Helvetica,sans-serif"><font color="#3d85c6">Brandie Jones </font></b><div style="color:rgb(34,34,34);font-family:Arial,Helvetica,sans-serif"><div><div><font color="#3d85c6"><b><i>Executive </i></b></font><b style="color:rgb(61,133,198)"><i>Administrative Assistant</i></b></div></div><div><font color="#3d85c6">Toyota Technological Institute</font></div><div><font color="#3d85c6">6045 S. Kenwood Avenue</font></div><div><font color="#3d85c6">Chicago, IL  60637</font></div></div><div style="color:rgb(34,34,34);font-family:Arial,Helvetica,sans-serif"><font color="#3d85c6"><a href="http://www.ttic.edu/" target="_blank">www.ttic.edu</a> </font></div><br class="gmail-Apple-interchange-newline"></div></div></div></div></div>