<div dir="ltr"><div><div class="gmail_default" style="font-family:georgia,serif;font-size:small;color:rgb(0,0,0)"><b>When:         </b>February<b> </b>14h <b>at <span style="background-color:rgb(255,255,0)">12:30pm CT  </span></b></div><div><div class="gmail_default"><div dir="ltr"><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000" face="georgia, serif"> <br><b>Where:</b><b>  </b>      Talk will be given<span style="background-color:rgb(255,255,0)"> </span><span style="background-color:rgb(255,255,0)"><font style="font-weight:bold"><u>live, in-person</u></font><font style="font-weight:bold"> </font></span>at</font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000" face="georgia, serif">                       TTIC, 6045 S. Kenwood Avenue</font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000" face="georgia, serif">                        5th Floor, Room 530<b>              </b>   </font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000" face="georgia, serif"><br></font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"><b><span class="gmail_default"></span>Virtually:</b>    via Panopto (<a href="https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=f41ea02d-3f4d-4e87-94a2-b1a901057aae" target="_blank">Livestream</a>) </font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000" face="georgia, serif"> </font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"><b>Who:</b><b> </b>          Zhiyuan Li, TTIC</font></p><table border="0" cellspacing="0" cellpadding="0" width="0" style="width:0in;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial;border-collapse:collapse"><tbody><tr><td style="padding:0in"></td></tr></tbody></table><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000" face="georgia, serif"> <span class="gmail_default"></span></font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"><b>Title</b>:           Pencil: Long Thoughts with Short Memory</font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"><br></font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"><b>Abstract:  </b> </font><font face="georgia, serif"> Recent works (e.g., Deepseek R1) shows that long CoT (Chain of Thought) greatly improves reasoning capability of Large Language Models (LLMs). However, it also poses significant challenges for memory efficiency —and consequently, time efficiency-- during inference, even for problems solvable with small space. This limitation stems from the non-erasable nature of standard CoT, which equals the space complexity (context length) to the time complexity (CoT length).</font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif"><br></font></p><font face="georgia, serif" color="#000000">In this talk, we will introduce a new method, PENCIL, to improve the efficiency of CoT by incorporating a reduction mechanism into the autoregressive generation process. PENCIL enables the model to actively discard obsolete tokens by outputting special tokens which triggers the reduction mechanism. We show PENCIL can perform universal space-efficient computation, that is,  PENCIL can simulate Turing machines with maximal context length matching its space complexity and total number of generated tokens matching its time complexity. By effectively reducing the maximal context length, PENCIL also decreases per-token generation time, enabling improved scalability compared to standard CoT. This efficiency gain translates into enhanced performance on complex reasoning tasks, including 97% accuracy on the challenging 5×5 Einstein’s puzzle, using a 25M-parameter transformer with a 2048-token context length.</font><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"><br></font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000"><br></font></p></div></div><div><div class="gmail_default"><b><font color="#000000" face="georgia, serif">*********************************************************************************************</font></b></div><p class="MsoNormal" style="margin:0in 0in 8pt;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial;line-height:15.6933px"><font face="georgia, serif" color="#000000"><b>Masks are optional in all common areas. </b><b>Full visitor guidance is available at <a href="http://ttic.edu/visitors" target="_blank">ttic.edu/visitors</a>.</b></font></p><p class="MsoNormal" style="margin:0in 0in 12pt;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial;line-height:15.6933px"><b><font color="#000000" face="georgia, serif">*********************************************************************************************</font></b></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><b><i><font color="#000000" face="georgia, serif">Research at TTIC Seminar Series</font></i></b></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000" face="georgia, serif"> </font></p><div class="gmail_default"><div dir="ltr"><div class="gmail_default"><div dir="ltr"><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="georgia, serif" color="#000000">TTIC is hosting a weekly seminar series presenting the research currently underway at the Institute. Every week a different TTIC faculty member will present their research.  The lectures are intended for students seeking research topics and advisors and for the general TTIC and University of Chicago communities interested in hearing what their colleagues are up to.</font></p></div></div></div></div></div></div><font color="#888888"><br clear="all"></font><br clear="all"></div><div><br></div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><b style="background-color:rgb(255,255,255)"><font color="#3d85c6">Brandie Jones </font></b><div><div><div><font color="#3d85c6"><b><i>Executive </i></b></font><b style="color:rgb(61,133,198)"><i>Administrative Assistant</i></b></div></div><div><span style="background-color:rgb(255,255,255)"><font color="#3d85c6">Toyota Technological Institute</font></span></div><div><span style="background-color:rgb(255,255,255)"><font color="#3d85c6">6045 S. Kenwood Avenue</font></span></div><div><span style="background-color:rgb(255,255,255)"><font color="#3d85c6">Chicago, IL  60637</font></span></div></div><div><span style="background-color:rgb(255,255,255)"><font color="#3d85c6"><a href="http://www.ttic.edu" target="_blank">www.ttic.edu</a> </font></span></div><div><span style="background-color:rgb(255,255,255)"><font color="#3d85c6"><br></font></span></div></div></div></div>