<div dir="ltr">
<div class="gmail-m_-7020715501473028896gmail-m_5632031467166595000gmail-gE gmail-m_-7020715501473028896gmail-m_5632031467166595000gmail-iv gmail-m_-7020715501473028896gmail-m_5632031467166595000gmail-gt" style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;padding:10px 0px 3px"><div class="gmail_default" style="font-size:12.8px;color:rgb(34,34,34);font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;background-color:rgb(255,255,255)"><font color="#000000" face="arial, helvetica, sans-serif">When: Thursday, February 8th <span class="gmail-m_-7020715501473028896gmail-m_5632031467166595000gmail-m_-2847741945628619148gmail-m_-681226243108775954gmail-m_-4338767794047815756gmail-m_7960197898027616883gmail-m_8692226636264124041gmail-m_2794822896869921223gmail-m_7508998950622620526gmail-m_-7153355664495542534gmail-il">at</span> <b style="background-color:rgb(255,255,255)">11:00am</b></font></div><div class="gmail_default" style="font-size:12.8px;color:rgb(34,34,34);font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;background-color:rgb(255,255,255)"><font color="#000000" face="arial, helvetica, sans-serif"><br></font></div><div class="gmail_default" style="font-size:12.8px;color:rgb(34,34,34);font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;background-color:rgb(255,255,255)"><font color="#000000" face="arial, helvetica, sans-serif">Where: <span class="gmail-m_-7020715501473028896gmail-m_5632031467166595000gmail-m_-2847741945628619148gmail-m_-681226243108775954gmail-m_-4338767794047815756gmail-m_7960197898027616883gmail-m_8692226636264124041gmail-m_2794822896869921223gmail-m_7508998950622620526gmail-m_-7153355664495542534gmail-m_8421504075585210435gmail-m_3262824545120381495gmail-m_-1141671822915777344gmail-m_-7219251726624328345gmail-m_-8588148075564318222gmail-m_-8767966813928691312gmail-m_-1542318334608687154gmail-m_5717104778280916634gmail-m_4845490158781220632gmail-m_5124567205141626540gmail-m_3209361100497750746gmail-m_2953668934074478317gmail-m_-3155518689668024534m_9067904842688472155gmail-m_3071693547520408192gmail-il"><span class="gmail-m_-7020715501473028896gmail-m_5632031467166595000gmail-m_-2847741945628619148gmail-m_-681226243108775954gmail-m_-4338767794047815756gmail-m_7960197898027616883gmail-m_8692226636264124041gmail-m_2794822896869921223gmail-m_7508998950622620526gmail-m_-7153355664495542534gmail-il">TTIC</span></span>, 6045 S Kenwood Avenue, 5th Floor, Room 526</font></div><div class="gmail_default" style="font-size:12.8px;color:rgb(34,34,34);font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;background-color:rgb(255,255,255)"><font color="#000000" face="arial, helvetica, sans-serif"><br></font></div><div style="text-align:start;text-indent:0px;text-decoration-style:initial;text-decoration-color:initial;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif"><font color="#000000" style="color:rgb(34,34,34);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-transform:none;white-space:normal;word-spacing:0px">Who: </font><font color="#000000"><span style="font-size:12.8px">Yonatan Belinkov, MIT</span></font><font color="#000000"><span style="font-size:12.8px"> </span></font></font></div><font face="arial, helvetica, sans-serif"><br><br></font></div><div id="gmail-m_-7020715501473028896gmail-m_5632031467166595000gmail-:6pa" class="gmail-m_-7020715501473028896gmail-m_5632031467166595000gmail-ii gmail-m_-7020715501473028896gmail-m_5632031467166595000gmail-gt" style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;direction:ltr;margin:5px 15px 0px 0px;padding-bottom:5px"><div id="gmail-m_-7020715501473028896gmail-m_5632031467166595000gmail-:7up" class="gmail-m_-7020715501473028896gmail-m_5632031467166595000gmail-a3s gmail-m_-7020715501473028896gmail-m_5632031467166595000gmail-aXjCH gmail-m_-7020715501473028896gmail-m_5632031467166595000gmail-m16153195b580df91" style="overflow:hidden"><div dir="ltr"><span style="font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;color:rgb(0,0,0);white-space:pre-wrap"><div style="color:rgb(34,34,34);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><font face="arial, helvetica, sans-serif"><span style="color:rgb(34,34,34);font-family:arial,helvetica,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">Title: Internal Representations in Deep Learning for Language and Speech Processing </span><div style="color:rgb(34,34,34);font-family:arial,helvetica,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><br><div>Abstract: Language technology has become pervasive in everyday life, powering applications like Apple’s Siri or Google’s Assistant. Neural networks are a key component in these systems thanks to their ability to model large amounts of data. Contrary to traditional systems, models based on deep neural networks (a.k.a. deep learning) can be trained in an end-to-end fashion on input-output pairs, such as a sentence in one language and its translation in another language, or a speech utterance and its transcription. The end-to-end training paradigm simplifies the engineering process while giving the model flexibility to optimize for the desired task. This, however, often comes at the expense of model interpretability: understanding the role of different parts of the deep neural network is difficult, and such models are often perceived as “black-box”. In this work, we study deep learning models for two core language technology tasks: machine translation and speech recognition. We advocate an approach that attempts to decode the information encoded in such models while they are being trained. We perform a range of experiments comparing different modules, layers, and representations in the end-to-end models. Our analyses illuminate the inner workings of end-to-end machine translation and speech recognition systems, explain how they capture different language properties, and suggest potential directions for improving them. The methodology is also applicable to other tasks in the language domain and beyond. </div></div><div style="color:rgb(34,34,34);font-family:arial,helvetica,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="color:rgb(34,34,34);font-family:arial,helvetica,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">Bio:</div><div style="color:rgb(34,34,34);font-family:arial,helvetica,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><span style="background-color:rgb(255,255,255)"><span class="gmail-m_-7020715501473028896gmail-m_5632031467166595000gmail-il">Yonatan</span><span> </span><span class="gmail-m_-7020715501473028896gmail-m_5632031467166595000gmail-il">Belinkov</span><span> </span>is a PhD candidate at the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL), working on language and speech processing. His recent research interests focus on representations of language in neural network models. His research has been published at ACL, EMNLP, TACL, ICLR, and NIPS. He received an SM degree from MIT in 2014 and prior to that a BSc in Mathematics and an MA in Arabic Studies, both from Tel Aviv University.<br><br><span style="color:rgb(34,34,34);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">Host:<span> </span></span><a href="mailto:klivescu@ttic.edu" target="_blank" style="color:rgb(17,85,204);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">Karen Livescu</a><span> </span></span></div></font></div></span></div></div></div>
<br><div><br></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><font face="arial, helvetica, sans-serif">Mary C. Marre</font><div><font face="arial, helvetica, sans-serif">Administrative Assistant</font></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6"><b>Toyota Technological Institute</b></font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">6045 S. Kenwood Avenue</font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Room 504</font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Chicago, IL 60637</font></i></div><div><i><font face="arial, helvetica, sans-serif">p:(773) 834-1757</font></i></div><div><i><font face="arial, helvetica, sans-serif">f: (773) 357-6970</font></i></div><div><b><i><a href="mailto:mmarre@ttic.edu" target="_blank"><font face="arial, helvetica, sans-serif">mmarre@ttic.edu</font></a></i></b></div></div></div></div></div></div></div></div></div></div>
<br><div class="gmail_quote">On Wed, Feb 7, 2018 at 2:08 PM, Mary Marre <span dir="ltr"><<a href="mailto:mmarre@ttic.edu" target="_blank">mmarre@ttic.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">
<div class="m_-7020715501473028896gmail-m_5632031467166595000gmail-gE m_-7020715501473028896gmail-m_5632031467166595000gmail-iv m_-7020715501473028896gmail-m_5632031467166595000gmail-gt" style="padding:10px 0px 3px"><div style="font-size:12.8px;color:rgb(34,34,34);font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;background-color:rgb(255,255,255)"><font color="#000000" face="arial, helvetica, sans-serif">When: Thursday, February 8th <span class="m_-7020715501473028896gmail-m_5632031467166595000gmail-m_-2847741945628619148gmail-m_-681226243108775954gmail-m_-4338767794047815756gmail-m_7960197898027616883gmail-m_8692226636264124041gmail-m_2794822896869921223gmail-m_7508998950622620526gmail-m_-7153355664495542534gmail-il">at</span> <b style="background-color:rgb(255,255,255)">11:00am</b></font></div><div style="font-size:12.8px;color:rgb(34,34,34);font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;background-color:rgb(255,255,255)"><font color="#000000" face="arial, helvetica, sans-serif"><br></font></div><div style="font-size:12.8px;color:rgb(34,34,34);font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;background-color:rgb(255,255,255)"><font color="#000000" face="arial, helvetica, sans-serif">Where: <span class="m_-7020715501473028896gmail-m_5632031467166595000gmail-m_-2847741945628619148gmail-m_-681226243108775954gmail-m_-4338767794047815756gmail-m_7960197898027616883gmail-m_8692226636264124041gmail-m_2794822896869921223gmail-m_7508998950622620526gmail-m_-7153355664495542534gmail-m_8421504075585210435gmail-m_3262824545120381495gmail-m_-1141671822915777344gmail-m_-7219251726624328345gmail-m_-8588148075564318222gmail-m_-8767966813928691312gmail-m_-1542318334608687154gmail-m_5717104778280916634gmail-m_4845490158781220632gmail-m_5124567205141626540gmail-m_3209361100497750746gmail-m_2953668934074478317gmail-m_-3155518689668024534m_9067904842688472155gmail-m_3071693547520408192gmail-il"><span class="m_-7020715501473028896gmail-m_5632031467166595000gmail-m_-2847741945628619148gmail-m_-681226243108775954gmail-m_-4338767794047815756gmail-m_7960197898027616883gmail-m_8692226636264124041gmail-m_2794822896869921223gmail-m_7508998950622620526gmail-m_-7153355664495542534gmail-il">TTIC</span></span>, 6045 S Kenwood Avenue, 5th Floor, Room 526</font></div><div style="font-size:12.8px;color:rgb(34,34,34);font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;background-color:rgb(255,255,255)"><font color="#000000" face="arial, helvetica, sans-serif"><br></font></div><div style="text-align:start;text-indent:0px;text-decoration-style:initial;text-decoration-color:initial;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif"><font color="#000000" style="color:rgb(34,34,34);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-transform:none;white-space:normal;word-spacing:0px">Who: </font><font color="#000000"><span style="font-size:12.8px">Yonatan Belinkov, MIT</span></font><font color="#000000"><span style="font-size:12.8px"> </span></font></font></div><font face="arial, helvetica, sans-serif"><br><br></font></div><div id="m_-7020715501473028896gmail-m_5632031467166595000gmail-:6pa" class="m_-7020715501473028896gmail-m_5632031467166595000gmail-ii m_-7020715501473028896gmail-m_5632031467166595000gmail-gt" style="font-size:12.8px;direction:ltr;margin:5px 15px 0px 0px;padding-bottom:5px"><div id="m_-7020715501473028896gmail-m_5632031467166595000gmail-:7up" class="m_-7020715501473028896gmail-m_5632031467166595000gmail-a3s m_-7020715501473028896gmail-m_5632031467166595000gmail-aXjCH m_-7020715501473028896gmail-m_5632031467166595000gmail-m16153195b580df91" style="overflow:hidden"><div dir="ltr"><span style="font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;color:rgb(0,0,0);white-space:pre-wrap"><div style="color:rgb(34,34,34);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><font face="arial, helvetica, sans-serif"><span style="color:rgb(34,34,34);font-family:arial,helvetica,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">Title: Internal Representations in Deep Learning for Language and Speech Processing </span><div style="color:rgb(34,34,34);font-family:arial,helvetica,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><br><div>Abstract: Language technology has become pervasive in everyday life, powering applications like Apple’s Siri or Google’s Assistant. Neural networks are a key component in these systems thanks to their ability to model large amounts of data. Contrary to traditional systems, models based on deep neural networks (a.k.a. deep learning) can be trained in an end-to-end fashion on input-output pairs, such as a sentence in one language and its translation in another language, or a speech utterance and its transcription. The end-to-end training paradigm simplifies the engineering process while giving the model flexibility to optimize for the desired task. This, however, often comes at the expense of model interpretability: understanding the role of different parts of the deep neural network is difficult, and such models are often perceived as “black-box”. In this work, we study deep learning models for two core language technology tasks: machine translation and speech recognition. We advocate an approach that attempts to decode the information encoded in such models while they are being trained. We perform a range of experiments comparing different modules, layers, and representations in the end-to-end models. Our analyses illuminate the inner workings of end-to-end machine translation and speech recognition systems, explain how they capture different language properties, and suggest potential directions for improving them. The methodology is also applicable to other tasks in the language domain and beyond. </div></div><div style="color:rgb(34,34,34);font-family:arial,helvetica,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="color:rgb(34,34,34);font-family:arial,helvetica,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">Bio:</div><div style="color:rgb(34,34,34);font-family:arial,helvetica,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><span style="background-color:rgb(255,255,255)"><span class="m_-7020715501473028896gmail-m_5632031467166595000gmail-il">Yonatan</span><span> </span><span class="m_-7020715501473028896gmail-m_5632031467166595000gmail-il">Belinkov</span><span> </span>is a PhD candidate at the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL), working on language and speech processing. His recent research interests focus on representations of language in neural network models. His research has been published at ACL, EMNLP, TACL, ICLR, and NIPS. He received an SM degree from MIT in 2014 and prior to that a BSc in Mathematics and an MA in Arabic Studies, both from Tel Aviv University.<br><br><span style="color:rgb(34,34,34);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">Host:<span> </span></span><a href="mailto:klivescu@ttic.edu" style="color:rgb(17,85,204);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" target="_blank">Karen Livescu</a><span> </span><br></span></div><br class="m_-7020715501473028896gmail-Apple-interchange-newline">
</font></div></span></div></div></div><br><div class="gmail_extra"><br clear="all"><div><div class="m_-7020715501473028896gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><font face="arial, helvetica, sans-serif">Mary C. Marre</font><div><font face="arial, helvetica, sans-serif">Administrative Assistant</font></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6"><b>Toyota Technological Institute</b></font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">6045 S. Kenwood Avenue</font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Room 504</font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Chicago, IL 60637</font></i></div><div><i><font face="arial, helvetica, sans-serif">p:<a href="tel:(773)%20834-1757" value="+17738341757" target="_blank">(773) 834-1757</a></font></i></div><div><i><font face="arial, helvetica, sans-serif">f: <a href="tel:(773)%20357-6970" value="+17733576970" target="_blank">(773) 357-6970</a></font></i></div><div><b><i><a href="mailto:mmarre@ttic.edu" target="_blank"><font face="arial, helvetica, sans-serif">mmarre@ttic.edu</font></a></i></b></div></div></div></div></div></div></div></div></div></div>
<br><div class="gmail_quote">On Thu, Feb 1, 2018 at 5:17 PM, Mary Marre <span dir="ltr"><<a href="mailto:mmarre@ttic.edu" target="_blank">mmarre@ttic.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>
<div class="m_-7020715501473028896m_5632031467166595000gmail-gE m_-7020715501473028896m_5632031467166595000gmail-iv m_-7020715501473028896m_5632031467166595000gmail-gt" style="padding:10px 0px 3px">
<div style="font-size:12.8px;color:rgb(34,34,34);font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;background-color:rgb(255,255,255)"><font color="#000000" face="arial, helvetica, sans-serif">When: Thursday, February 8th <span class="m_-7020715501473028896m_5632031467166595000gmail-m_-2847741945628619148gmail-m_-681226243108775954gmail-m_-4338767794047815756gmail-m_7960197898027616883gmail-m_8692226636264124041gmail-m_2794822896869921223gmail-m_7508998950622620526gmail-m_-7153355664495542534gmail-il">at</span> <b style="background-color:rgb(255,255,255)">11:00am</b></font></div><div style="font-size:12.8px;color:rgb(34,34,34);font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;background-color:rgb(255,255,255)"><font color="#000000" face="arial, helvetica, sans-serif"><br></font></div><div style="font-size:12.8px;color:rgb(34,34,34);font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;background-color:rgb(255,255,255)"><font color="#000000" face="arial, helvetica, sans-serif">Where: <span class="m_-7020715501473028896m_5632031467166595000gmail-m_-2847741945628619148gmail-m_-681226243108775954gmail-m_-4338767794047815756gmail-m_7960197898027616883gmail-m_8692226636264124041gmail-m_2794822896869921223gmail-m_7508998950622620526gmail-m_-7153355664495542534gmail-m_8421504075585210435gmail-m_3262824545120381495gmail-m_-1141671822915777344gmail-m_-7219251726624328345gmail-m_-8588148075564318222gmail-m_-8767966813928691312gmail-m_-1542318334608687154gmail-m_5717104778280916634gmail-m_4845490158781220632gmail-m_5124567205141626540gmail-m_3209361100497750746gmail-m_2953668934074478317gmail-m_-3155518689668024534m_9067904842688472155gmail-m_3071693547520408192gmail-il"><span class="m_-7020715501473028896m_5632031467166595000gmail-m_-2847741945628619148gmail-m_-681226243108775954gmail-m_-4338767794047815756gmail-m_7960197898027616883gmail-m_8692226636264124041gmail-m_2794822896869921223gmail-m_7508998950622620526gmail-m_-7153355664495542534gmail-il">TTIC</span></span>, 6045 S Kenwood Avenue, 5th Floor, Room 526</font></div><div style="font-size:12.8px;color:rgb(34,34,34);font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;background-color:rgb(255,255,255)"><font color="#000000" face="arial, helvetica, sans-serif"><br></font></div><div style="text-align:start;text-indent:0px;text-decoration-style:initial;text-decoration-color:initial;background-color:rgb(255,255,255)"><font face="arial, helvetica, sans-serif"><font color="#000000" style="color:rgb(34,34,34);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-transform:none;white-space:normal;word-spacing:0px">Who: </font><font color="#000000"><span style="font-size:12.8px">Yonatan Belinkov, MIT</span></font><font color="#000000"><span style="font-size:12.8px"> </span></font></font></div><font face="arial, helvetica, sans-serif"><br><br></font></div><div id="m_-7020715501473028896m_5632031467166595000gmail-:6pa" class="m_-7020715501473028896m_5632031467166595000gmail-ii m_-7020715501473028896m_5632031467166595000gmail-gt" style="font-size:12.8px;direction:ltr;margin:5px 15px 0px 0px;padding-bottom:5px"><div id="m_-7020715501473028896m_5632031467166595000gmail-:7up" class="m_-7020715501473028896m_5632031467166595000gmail-a3s m_-7020715501473028896m_5632031467166595000gmail-aXjCH m_-7020715501473028896m_5632031467166595000gmail-m16153195b580df91" style="overflow:hidden"><div dir="ltr"><span style="font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;color:rgb(0,0,0);white-space:pre-wrap"><div style="color:rgb(34,34,34);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><font face="arial, helvetica, sans-serif">Title: Internal Representations in Deep Learning for Language and Speech Processing <div><br><div>Abstract: Language technology has become pervasive in everyday life, powering applications like Apple’s Siri or Google’s Assistant. Neural networks are a key component in these systems thanks to their ability to model large amounts of data. Contrary to traditional systems, models based on deep neural networks (a.k.a. deep learning) can be trained in an end-to-end fashion on input-output pairs, such as a sentence in one language and its translation in another language, or a speech utterance and its transcription. The end-to-end training paradigm simplifies the engineering process while giving the model flexibility to optimize for the desired task. This, however, often comes at the expense of model interpretability: understanding the role of different parts of the deep neural network is difficult, and such models are often perceived as “black-box”. In this work, we study deep learning models for two core language technology tasks: machine translation and speech recognition. We advocate an approach that attempts to decode the information encoded in such models while they are being trained. We perform a range of experiments comparing different modules, layers, and representations in the end-to-end models. Our analyses illuminate the inner workings of end-to-end machine translation and speech recognition systems, explain how they capture different language properties, and suggest potential directions for improving them. The methodology is also applicable to other tasks in the language domain and beyond. </div></div><div><br></div><div>Bio:</div><div><span style="background-color:rgb(255,255,255)"><span class="m_-7020715501473028896m_5632031467166595000gmail-il">Yonatan</span><span> </span><span class="m_-7020715501473028896m_5632031467166595000gmail-il">Belinkov</span><span> </span>is a PhD candidate at the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL), working on language and speech processing. His recent research interests focus on representations of language in neural network models. His research has been published at ACL, EMNLP, TACL, ICLR, and NIPS. He received an SM degree from MIT in 2014 and prior to that a BSc in Mathematics and an MA in Arabic Studies, both from Tel Aviv University.<br><br>
<span style="color:rgb(34,34,34);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">Host:<span> </span></span><a href="mailto:klivescu@ttic.edu" style="color:rgb(17,85,204);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" target="_blank">Karen Livescu</a>
<br></span></div><div><span style="background-color:rgb(255,255,255)"><br></span></div></font></div></span></div></div></div><div id="m_-7020715501473028896m_5632031467166595000gmail-:6pa" class="m_-7020715501473028896m_5632031467166595000gmail-ii m_-7020715501473028896m_5632031467166595000gmail-gt" style="font-size:12.8px;direction:ltr;margin:5px 0px 0px 15px;padding-bottom:5px"><div id="m_-7020715501473028896m_5632031467166595000gmail-:7up" class="m_-7020715501473028896m_5632031467166595000gmail-a3s m_-7020715501473028896m_5632031467166595000gmail-aXjCH m_-7020715501473028896m_5632031467166595000gmail-m16153195b580df91" style="overflow:hidden"><div dir="ltr"><span style="font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;color:rgb(0,0,0);white-space:pre-wrap"><div style="color:rgb(34,34,34);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><div><br></div></div></span></div></div></div></div><div><div class="m_-7020715501473028896m_5632031467166595000gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><font face="arial, helvetica, sans-serif">Mary C. Marre</font><div><font face="arial, helvetica, sans-serif">Administrative Assistant</font></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6"><b>Toyota Technological Institute</b></font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">6045 S. Kenwood Avenue</font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Room 504</font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Chicago, IL 60637</font></i></div><div><i><font face="arial, helvetica, sans-serif">p:<a href="tel:(773)%20834-1757" value="+17738341757" target="_blank">(773) 834-1757</a></font></i></div><div><i><font face="arial, helvetica, sans-serif">f: <a href="tel:(773)%20357-6970" value="+17733576970" target="_blank">(773) 357-6970</a></font></i></div><div><b><i><a href="mailto:mmarre@ttic.edu" target="_blank"><font face="arial, helvetica, sans-serif">mmarre@ttic.edu</font></a></i></b></div></div></div></div></div></div></div></div></div></div>
</div>
</blockquote></div><br></div></div>
</blockquote></div><br></div></div>