[Theory] REMINDER: 3/3 Talks at TTIC: Kartik Goyal, Carnegie Mellon

[Mary Marre]

*When:*      Wednesday, March 3rd at* 11:10 am CT*



*Where:*     Zoom Virtual Talk (*register in advance here
<https://uchicagogroup.zoom.us/webinar/register/WN_Sbr9rWroR2i3L_0U8Mt6-w>*)


*Who: *       Kartik Goyal, Carnegie Mellon


*Title**:*  Revisiting Training and Decoding in Neural Sequence Models


*Abstract**:* Commonly prevalent locally-normalized autoregressive neural
sequence models, while being highly effective for various NLP tasks, suffer
from optimization issues including, but not limited to exposure bias, label
bias, and inadequate representation of context. Furthermore during
decoding, they yield degenerate sequences and exhibit poor calibration. To
address these issues, first I describe our work on designing differentiable
training procedures for neural sequence models that take into account the
methods used for decoding with these models like local argmax, sampling,
and beam search. Then, I discuss the potential of globally-normalized
models to ameliorate these issues and describe their relationship to the
highly effective masked token reconstruction objective for training neural
sequence models. Specifically, I describe our scheme inspired by Metropolis
Hastings Monte Carlo that enables drawing of representative samples from
these masked language models.


*Bio**:* Kartik Goyal is a PhD candidate at Language Technologies
Institute, Carnegie Mellon University, where he is coadvised by Chris Dyer
and Taylor Berg-Kirkpatrick. He is interested in designing statistical
training and inference procedures for modelling artifacts with rich latent
structure to address research questions in Natural Language Processing and
Digital Humanities.

*Host: * Kevin Gimpel <kgimpel at ttic.edu>


Mary C. Marre
Faculty Administrative Support
*Toyota Technological Institute*
*6045 S. Kenwood Avenue*
*Room 517*
*Chicago, IL  60637*
*p:(773) 834-1757*
*f: (773) 357-6970*
*mmarre at ttic.edu <mmarre at ttic.edu>*


On Tue, Mar 2, 2021 at 3:30 PM Mary Marre <mmarre at ttic.edu> wrote:

> *When:*      Wednesday, March 3rd at* 11:10 am CT*
>
>
>
> *Where:*     Zoom Virtual Talk (*register in advance here
> <https://uchicagogroup.zoom.us/webinar/register/WN_Sbr9rWroR2i3L_0U8Mt6-w>*
> )
>
>
> *Who: *       Kartik Goyal, Carnegie Mellon
>
>
> *Title**:*  Revisiting Training and Decoding in Neural Sequence Models
>
>
> *Abstract**:* Commonly prevalent locally-normalized autoregressive neural
> sequence models, while being highly effective for various NLP tasks, suffer
> from optimization issues including, but not limited to exposure bias, label
> bias, and inadequate representation of context. Furthermore during
> decoding, they yield degenerate sequences and exhibit poor calibration. To
> address these issues, first I describe our work on designing differentiable
> training procedures for neural sequence models that take into account the
> methods used for decoding with these models like local argmax, sampling,
> and beam search. Then, I discuss the potential of globally-normalized
> models to ameliorate these issues and describe their relationship to the
> highly effective masked token reconstruction objective for training neural
> sequence models. Specifically, I describe our scheme inspired by Metropolis
> Hastings Monte Carlo that enables drawing of representative samples from
> these masked language models.
>
>
> *Bio**:* Kartik Goyal is a PhD candidate at Language Technologies
> Institute, Carnegie Mellon University, where he is coadvised by Chris
> Dyer and Taylor Berg-Kirkpatrick. He is interested in designing statistical
> training and inference procedures for modelling artifacts with rich latent
> structure to address research questions in Natural Language Processing and
> Digital Humanities.
>
> *Host: * Kevin Gimpel <kgimpel at ttic.edu>
>
>
>
> Mary C. Marre
> Faculty Administrative Support
> *Toyota Technological Institute*
> *6045 S. Kenwood Avenue*
> *Room 517*
> *Chicago, IL  60637*
> *p:(773) 834-1757*
> *f: (773) 357-6970*
> *mmarre at ttic.edu <mmarre at ttic.edu>*
>
>
> On Wed, Feb 24, 2021 at 6:06 PM Mary Marre <mmarre at ttic.edu> wrote:
>
>> *When:*      Wednesday, March 3rd at* 11:10 am CT*
>>
>>
>>
>> *Where:*     Zoom Virtual Talk (*register in advance here
>> <https://uchicagogroup.zoom.us/webinar/register/WN_Sbr9rWroR2i3L_0U8Mt6-w>*
>> )
>>
>>
>> *Who: *       Kartik Goyal, Carnegie Mellon
>>
>>
>> *Title**:*  Revisiting Training and Decoding in Neural Sequence Models
>>
>>
>> *Abstract**:* Commonly prevalent locally-normalized autoregressive
>> neural sequence models, while being highly effective for various NLP tasks,
>> suffer from optimization issues including, but not limited to exposure
>> bias, label bias, and inadequate representation of context. Furthermore
>> during decoding, they yield degenerate sequences and exhibit poor
>> calibration. To address these issues, first I describe our work on
>> designing differentiable training procedures for neural sequence models
>> that take into account the methods used for decoding with these models like
>> local argmax, sampling, and beam search. Then, I discuss the potential of
>> globally-normalized models to ameliorate these issues and describe their
>> relationship to the highly effective masked token reconstruction objective
>> for training neural sequence models. Specifically, I describe our scheme
>> inspired by Metropolis Hastings Monte Carlo that enables drawing of
>> representative samples from these masked language models.
>>
>>
>> *Bio**:* Kartik Goyal is a PhD candidate at Language Technologies
>> Institute, Carnegie Mellon University, where he is coadvised by Chris
>> Dyer and Taylor Berg-Kirkpatrick. He is interested in designing statistical
>> training and inference procedures for modelling artifacts with rich latent
>> structure to address research questions in Natural Language Processing and
>> Digital Humanities.
>>
>> *Host: * Kevin Gimpel <kgimpel at ttic.edu>
>>
>>
>>
>> Mary C. Marre
>> Faculty Administrative Support
>> *Toyota Technological Institute*
>> *6045 S. Kenwood Avenue*
>> *Room 517*
>> *Chicago, IL  60637*
>> *p:(773) 834-1757*
>> *f: (773) 357-6970*
>> *mmarre at ttic.edu <mmarre at ttic.edu>*
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/theory/attachments/20210303/a1f7845d/attachment-0001.html>