[Theory] 3/3 Talks at TTIC: Kartik Goyal, Carnegie Mellon

[Mary Marre]

*When:*      Wednesday, March 3rd at* 11:10 am CT*



*Where:*     Zoom Virtual Talk (*register in advance here
<https://uchicagogroup.zoom.us/webinar/register/WN_Sbr9rWroR2i3L_0U8Mt6-w>*)


*Who: *       Kartik Goyal, Carnegie Mellon


*Title**:*  Revisiting Training and Decoding in Neural Sequence Models


*Abstract**:* Commonly prevalent locally-normalized autoregressive neural
sequence models, while being highly effective for various NLP tasks, suffer
from optimization issues including, but not limited to exposure bias, label
bias, and inadequate representation of context. Furthermore during
decoding, they yield degenerate sequences and exhibit poor calibration. To
address these issues, first I describe our work on designing differentiable
training procedures for neural sequence models that take into account the
methods used for decoding with these models like local argmax, sampling,
and beam search. Then, I discuss the potential of globally-normalized
models to ameliorate these issues and describe their relationship to the
highly effective masked token reconstruction objective for training neural
sequence models. Specifically, I describe our scheme inspired by Metropolis
Hastings Monte Carlo that enables drawing of representative samples from
these masked language models.


*Bio**:* Kartik Goyal is a PhD candidate at Language Technologies
Institute, Carnegie Mellon University, where he is coadvised by Chris Dyer
and Taylor Berg-Kirkpatrick. He is interested in designing statistical
training and inference procedures for modelling artifacts with rich latent
structure to address research questions in Natural Language Processing and
Digital Humanities.

*Host: * Kevin Gimpel <kgimpel at ttic.edu>



Mary C. Marre
Faculty Administrative Support
*Toyota Technological Institute*
*6045 S. Kenwood Avenue*
*Room 517*
*Chicago, IL  60637*
*p:(773) 834-1757*
*f: (773) 357-6970*
*mmarre at ttic.edu <mmarre at ttic.edu>*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/theory/attachments/20210224/8ffa8b04/attachment-0001.html>