<div dir="ltr"><div><div style="font-size:12.8px"><font face="arial, helvetica, sans-serif"><span style="font-size:12.8px">When:</span><b style="font-size:12.8px"> </b><span style="font-size:12.8px"> </span><span style="font-size:12.8px">Monday, April 3rd at 11:00 a.m.</span><span style="font-size:12.8px"> </span><br></font></div><div dir="ltr" style="font-size:12.8px"><div style="font-size:12.8px"><font face="arial, helvetica, sans-serif"><br></font></div><div style="font-size:12.8px"><font face="arial, helvetica, sans-serif">Where: TTIC, 6045 S. Kenwood Avenue, 5th Floor, Room 526</font></div><div style="font-size:12.8px"><font face="arial, helvetica, sans-serif"><br></font></div><div><font face="arial, helvetica, sans-serif" style="font-size:12.8px">Who: Alexander Rush, Harvard</font></div></div></div><div><br></div><div><br></div><div>Title: Structured Attention Networks<br></div><div><br></div><div>Abstract: </div><div>Recent deep learning systems for NLP and related fields have relied heavily on the use of neural attention, which allows models to learn to focus on selected regions of their input or memory. The use of neural attention has proven to be a crucial component for advances in machine translation, image captioning, question answering, summarization, end-to-end speech recognition, and more. In this talk, I will give an overview of the current uses of neural attention and memory, describe how the selection paradigm has provided NLP researchers flexibility in designing neural models, and demonstrate some fun applications of this approach from our group. <br></div><div><br></div><div>I will then argue that selection-based attention may be an unnecessarily simplistic approach for NLP, and discuss our recent work on Structured Attention Networks [Kim et al 2017]. These models integrate structured prediction as a hidden layer within deep neural networks to form a variant of attention that enables soft-selection over combinatorial structures, such as segmentations, labelings, and even parse trees. While this approach is inspired by structured prediction methods in NLP, building structured attention layers within a deep network is quite challenging, and I will describe the interesting dynamic programming approach needed for exact computation. Experiments test the approach on a range of NLP tasks including translation, question answering, and natural langauge inference, demonstrating improvements upon standard attention in performance and interpretability. Time pending, I will conclude by discussing recent related work exploring other variants of neural memory for algorithmic learning [Yang+Rush, 2017].</div><div><br></div><div><br></div><div>Bio: Alexander "Sasha" Rush is an Assistant Professor at Harvard School of Engineering and Applied Sciences where he runs the HarvardNLP group. Alexander received his PhD from MIT (2014) under the guidance of Michael Collins and worked as a postdoc at Facebook Artificial Intelligence Research (FAIR) under Yann LeCun. He is interested in machine learning and deep learning methods for large-scale natural language processing and understanding, including applications in neural machine translation (<a href="http://opennmt.net">http://opennmt.net</a>), abstractive summarization, image-to-text prediction, and long-form generation. His past work introduced novel combinatorial methods for structured prediction with applications to syntactic parsing and machine translation. His work has received three best paper/honorable mention awards at major NLP conferences. His group web page is <a href="http://nlp.seas.harvard.edu/">http://nlp.seas.harvard.edu/</a>, and he tweets at <a href="http://twitter.com/harvardnlp">http://twitter.com/harvardnlp</a>.</div><div><br></div><div><br></div><div>Host: <a href="mailto:kgimpel@ttic.edu">Kevin Gimpel</a></div><div><br></div><div><br></div><div><div style="font-size:12.8px"><span style="font-size:12.8px;font-family:arial,helvetica,sans-serif">For more information on the </span><span style="font-size:12.8px;font-family:arial,helvetica,sans-serif">colloquium</span><span style="font-size:12.8px;font-family:arial,helvetica,sans-serif"> series or to subscribe to the mailing list, please see </span><a href="http://www.ttic.edu/colloquium.php" target="_blank" style="font-size:12.8px;font-family:arial,helvetica,sans-serif">http://www.ttic.edu/colloq<wbr>uium.php</a><br></div></div><div><br></div><div><br></div><div><br></div><div><div class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><font face="arial, helvetica, sans-serif">Mary C. Marre</font><div><font face="arial, helvetica, sans-serif">Administrative Assistant</font></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6"><b>Toyota Technological Institute</b></font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">6045 S. Kenwood Avenue</font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Room 504</font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Chicago, IL 60637</font></i></div><div><i><font face="arial, helvetica, sans-serif">p:(773) 834-1757</font></i></div><div><i><font face="arial, helvetica, sans-serif">f: (773) 357-6970</font></i></div><div><b><i><a href="mailto:mmarre@ttic.edu" target="_blank"><font face="arial, helvetica, sans-serif">mmarre@ttic.edu</font></a></i></b></div></div></div></div></div></div></div></div></div></div>
</div>