<div dir="ltr">
<div class="gmail-adn gmail-ads"><div class="gmail-gs"><div class="gmail-"><div id="gmail-:188" class="gmail-ii gmail-gt gmail-adO"><div id="gmail-:189" class="gmail-a3s gmail-aXjCH"><div dir="ltr"><div><font style="color:rgb(0,0,0);font-family:arial,sans-serif"><b>When:</b> Friday, January 24th. <b>Refreshments at 12:00pm. </b></font><b>Talk at 12:20pm. </b></div><div class="gmail_quote"><div dir="ltr"><div style="font-family:arial,sans-serif;color:rgb(0,0,0)"><span style="font-family:arial,helvetica,sans-serif"><br></span></div><div style="font-family:arial,sans-serif;color:rgb(0,0,0)"><span style="font-family:arial,helvetica,sans-serif"><b>Where:</b> </span><span style="font-family:arial,helvetica,sans-serif">TTIC</span><span style="font-family:arial,helvetica,sans-serif">, 6045 S Kenwood Avenue, 5th Floor, Room 526</span><font face="arial, helvetica, sans-serif"><br></font></div><div style="font-family:arial,sans-serif;color:rgb(0,0,0)"><div><p style="margin-bottom:0.0001pt;line-height:normal"><font face="arial, helvetica, sans-serif"><b>Who: </b> Kevin Gimpel</font>, TTIC</p><table style="border-collapse:collapse;margin-top:0px;width:auto;font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif;font-size:0.875rem;letter-spacing:0.2px;display:block" cellpadding="0"></table></div><div><br></div><div><b>Title: </b>
Learning to do Structured Inference in Natural Language Processing
</div><div><br></div><div><b>
</b><div><b>Abstract: </b>
<div>Many tasks in natural language processing, computer vision, and
computational biology involve predicting structured outputs. Researchers
are increasingly applying deep representation learning to these
problems, but the <i>structured</i> component of these approaches is
usually quite simplistic. For example, neural machine translation
systems use unstructured training of local factors followed by beam
search for test-time inference. There have been several proposals for
deep energy-based structured modeling, but they pose difficulties for
learning and inference, preventing their widespread adoption. We focus
in this talk on structured prediction energy networks (SPENs; Belanger
& McCallum 2016), which use neural network architectures to define
energy functions that can capture arbitrary dependencies among parts of
structured outputs. </div><div><br></div><div>Prior work with SPENs used
gradient descent for inference, relaxing the structured output to a set
of continuous variables and then optimizing the energy with respect to
them. We replace this use of gradient descent with a neural network
trained to approximate structured argmax inference. This "inference
network" outputs continuous values that we treat as the output
structure. We develop large-margin training objectives to jointly train
deep energy functions and inference networks. The objectives resemble
the alternating optimization framework of generative adversarial
networks (GANs; Goodfellow et al. 2014): the inference network is
analogous to the generator and the energy function is analogous to the
discriminator. We present experimental results on several NLP tasks,
including multi-label classification, part-of-speech tagging, named
entity recognition, and machine translation. Inference networks achieve a
better speed/accuracy/search error trade-off than gradient descent,
while also being faster than exact inference at similar accuracy levels.
This increased efficiency allows us to experiment with deep, global
energy terms, which further improve results. </div>
<div><div id="gmail-m_3989296263997119630gmail-m_2753869368894010505m_-6281119864947526141m_-4821438133843384681gmail-m_6291452947409543257m_5058370782281332144:7sa"><img src="https://ssl.gstatic.com/ui/v1/icons/mail/images/cleardot.gif" class="gmail-CToWUd"></div></div></div></div></div><div style="font-family:arial,sans-serif;color:rgb(0,0,0)"><div style="font-size:13px;margin:0px;line-height:normal"><b style="font-size:12.8px"><font face="arial, helvetica, sans-serif">******************************************************************************************************</font></b><br></div><div style="font-family:arial;font-size:13px;margin:0px;line-height:normal"><div style="font-family:arial,sans-serif;font-size:12.8px"><b><font face="arial, helvetica, sans-serif"><br></font></b></div><div style="font-family:arial,sans-serif;font-size:12.8px"><b><font face="arial, helvetica, sans-serif">Research at TTIC Seminar Series</font></b></div><div style="font-family:arial,sans-serif;font-size:12.8px"><font face="arial, helvetica, sans-serif"> </font></div><div style="font-family:arial,sans-serif;font-size:12.8px"><font face="arial, helvetica, sans-serif">TTIC is
hosting a weekly seminar series presenting the research currently
underway at the Institute. Every week a different TTIC faculty member
will present their research. The lectures are intended both for
students seeking research topics and adviser, and for the
general TTIC and University of Chicago communities interested in hearing
what their colleagues are up to. </font></div><div style="font-family:arial,sans-serif;font-size:12.8px"><font face="arial, helvetica, sans-serif"> </font></div><div style="font-family:arial,sans-serif;font-size:12.8px"><font face="arial, helvetica, sans-serif">To receive announcements about the seminar series, please subscribe to the mailing list: <a href="https://groups.google.com/a/ttic.edu/group/talks/subscribe" style="font-family:arial,sans-serif;text-decoration-line:none" target="_blank">https://groups.google.com/a/ttic.edu/group/talks/subscribe</a></font></div><div style="font-family:arial,sans-serif;font-size:12.8px"><font face="arial, helvetica, sans-serif"><br></font></div><div style="font-family:arial,sans-serif;font-size:12.8px"><font face="arial, helvetica, sans-serif">Speaker details can be found at: <a href="http://www.ttic.edu/tticseminar.php" style="font-family:arial,sans-serif;text-decoration-line:none" target="_blank">http://www.ttic.edu/tticseminar.php</a>.</font></div><div style="font-family:arial,sans-serif;font-size:12.8px"><font face="arial, helvetica, sans-serif"> </font></div><div style="font-family:arial,sans-serif;font-size:12.8px"><font face="arial, helvetica, sans-serif">For additional questions, please contact Nathan Srebro at <a href="mailto:mcallester@ttic.edu" style="font-family:arial,sans-serif;text-decoration-line:none" target="_blank">nati@ttic.edu</a>.</font></div></div></div></div></div><div class="gmail-yj6qo gmail-ajU"><div id="gmail-:1c4" class="gmail-ajR" tabindex="0"><img class="gmail-ajT" src="https://ssl.gstatic.com/ui/v1/icons/mail/images/cleardot.gif"></div></div></div></div></div></div></div></div><div class="gmail-gA gmail-gt gmail-acV"><div class="gmail-gB gmail-xu"><div class="gmail-ip gmail-iq"><div id="gmail-:187"><table class="gmail-cf gmail-wS"><tbody><tr><td class="gmail-amq"><img id="gmail-:9e_27" name=":9e" src="https://www.google.com/s2/u/0/photos/private/AIbEiAIAAABDCK7S-NKKzdbbTCILdmNhcmRfcGhvdG8qKGE5MTA2ZWEyMmE0MDM4YmMwMWEzYzBkMmUwNjk1YTk5NGVkZTk0N2QwASzLdVTfH1uwR6ZV7WcEHXW0xofW?sz=40" class="gmail-ajn gmail-bofPge"></td><td class="gmail-amr"><br></td></tr></tbody></table></div></div></div></div><div class="gmail-nH gmail-aHU"><div class="gmail-nH gmail-hx"><div class="gmail-nH"><div class="gmail-h7 gmail-ie gmail-nH gmail-oy8Mbf" tabindex="-1"><div class="gmail-Bk"><div class="gmail-G3 gmail-G2 gmail-afm"><div><div id="gmail-:1ck"><div class="gmail-gA gmail-gt gmail-acV"><div class="gmail-gB gmail-xu"><div class="gmail-ip gmail-iq"><div id="gmail-:187"><table class="gmail-cf gmail-wS"><tbody><tr><td class="gmail-amr"><br></td></tr></tbody></table></div></div></div></div></div></div></div></div></div></div></div></div><div class="gmail-nH"><div class="gmail-l2 gmail-pfiaof"><div id="gmail-:1dr" class="gmail-aeV"><div><div class="gmail-md gmail-mj"><div class="gmail-aeW"><span dir="ltr"></span></div></div></div></div><div class="gmail-aeU"><div id="gmail-:1do"><div><div class="gmail-ma"> </div></div></div></div><div id="gmail-:1cw" class="gmail-ae3"><div><div class="gmail-l6"><span id="gmail-:s0" class="gmail-l8 gmail-LJOhwe" tabindex="0"></span></div></div></div></div></div>
<div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><font face="arial, helvetica, sans-serif"><b>Jerome Allen</b><br></font><div><font face="arial, helvetica, sans-serif">Executive Assistant<br></font></div><div><font face="arial, helvetica, sans-serif" color="#3d85c6"><b>Toyota Technological Institute</b></font></div><div><font face="arial, helvetica, sans-serif" color="#3d85c6">6045 S. Kenwood Avenue</font></div><div><font face="arial, helvetica, sans-serif" color="#3d85c6">Room 518</font></div><div><font face="arial, helvetica, sans-serif" color="#3d85c6">Chicago, IL 60637</font></div><div><font face="arial, helvetica, sans-serif">p:(773) 702-2311<br></font></div><div><i><b><a href="mailto:jallen@ttic.edu" target="_blank"><font face="arial, helvetica, sans-serif">jallen@ttic.edu</font></a></b></i></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div>