[Colloquium] 2/6 What's Next for Mamba? Toward More Expressive Recurrent Update Rules

[Markayla Epps via Colloquium]

UNIVERSITY OF CHICAGO
COMPUTER SCIENCE DEPARTMENT
PRESENTS

Songlin Yang
PhD student
Massachusetts Institute of Technology

Thursday, February 6th, 2025
1:00PM – 2:00PM CT . Food will be provided at 12:30PM CT
Location : TTIC 530
Zoom Link : Click Here<https://uchicago.zoom.us/j/97708787366?pwd=qD9KdtzwGCTPuciAfr6iaap3NubZgy.1>

Title: What's Next for Mamba? Toward More Expressive Recurrent Update Rules

Abstract: Recently, there has been rapid progress in the field of linear attention, including Lightning Attention (featured in the recent LLM MiniMax-01 with GPT-4.0-level performance), RetNet, Mamba2, xLSTM, GLA, DeltaNet, RWKV 7, TTT, and Titans, among others. In this talk, I will discuss the development and efficient training of linear attention models, explore the connections among these models, and culminate by deriving them in a principled manner through the lens of in-context meta-learning and gradient-based optimization.

Bio: Songlin is a second-year Ph.D. student at MIT CSAIL, advised by Prof. Yoon Kim. Her research focuses on hardware-aware algorithm design for sequence modeling, particularly in linear attention models.


[cid:image001.png at 01DB7623.5EB5D770]




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/colloquium/attachments/20250205/d2d9fa71/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 497490 bytes
Desc: image001.png
URL: <http://mailman.cs.uchicago.edu/pipermail/colloquium/attachments/20250205/d2d9fa71/attachment-0001.png>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: ATT00001.txt
URL: <http://mailman.cs.uchicago.edu/pipermail/colloquium/attachments/20250205/d2d9fa71/attachment-0002.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: ATT00002.txt
URL: <http://mailman.cs.uchicago.edu/pipermail/colloquium/attachments/20250205/d2d9fa71/attachment-0003.txt>