[Theory] REMINDER: 11/16 Young Researcher Seminar Series: Lisa Li, Stanford

[Mary Marre]

*When:*        Wednesday, November 16th at* 10:30** a**m CT   *


*Where:       *Talk will be given *live, in-person* at

                   TTIC, 6045 S. Kenwood Avenue

                   5th Floor, Room 530


*Virtually:*  *via* Panopto (*livestream
<https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=72cf36c1-6849-4d1f-acca-af480189a155>*
)


*Who: *         Lisa Li, Stanford University


------------------------------

*Title:*   Diffusion-LM Improves Controllable Text Generation.
*Abstract: *Controlling the behavior of language models (LMs) without
re-training is a major open problem in natural language generation. While
recent works have demonstrated successes on controlling simple sentence
attributes (e.g., sentiment), there has been little progress on complex,
fine-grained controls (e.g., syntactic structure). To address this
challenge, we develop a new non-autoregressive language model based on
continuous diffusions that we call Diffusion-LM. Building upon the recent
successes of diffusion models in continuous domains, Diffusion-LM
iteratively denoises a sequence of Gaussian vectors into word vectors,
yielding a sequence of intermediate latent variables. The continuous,
hierarchical nature of these intermediate variables enables a simple
gradient-based algorithm to perform complex, controllable generation tasks.
We demonstrate successful control of Diffusion-LM for six challenging
fine-grained control tasks, significantly outperforming prior work.

*Bio:* *Xiang Lisa Li* is a third-year PhD student in computer science at
Stanford University, advised by Percy Liang and Tatsunori Hashimoto. She
works on controllable text generation and efficient adaptation of
pre-trained language models. Lisa is supported by a Stanford Graduate
Fellowship and is the recipient of an EMNLP Best Paper award.

*Host: David McAllester <mcallester at ttic.edu>*

**************************************************************************************************



The *TTIC Young Researcher Seminar Series* (http://www.ttic.edu/young-
researcher.php) features talks by Ph.D. students and postdocs whose research is
of broad interest to the computer science community. The series provides an
opportunity for early-career researchers to present recent work to and meet
with students and faculty at TTIC and nearby universities.


The seminars are typically held on Wednesdays at 10:30am in TTIC Room 530.

For additional information, please contact *David McAllester *(
mcallester at ttic.edu).




Mary C. Marre
Faculty Administrative Support
*Toyota Technological Institute*
*6045 S. Kenwood Avenue, Rm 517*
*Chicago, IL  60637*
*773-834-1757*
*mmarre at ttic.edu <mmarre at ttic.edu>*


On Wed, Nov 9, 2022 at 6:17 PM Mary Marre <mmarre at ttic.edu> wrote:

> *When:*        Wednesday, November 16th at* 10:30** a**m CT   *
>
>
> *Where:       *Talk will be given *live, in-person* at
>
>                    TTIC, 6045 S. Kenwood Avenue
>
>                    5th Floor, Room 530
>
>
> *Virtually:*  *via* Panopto (*livestream
> <https://uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=72cf36c1-6849-4d1f-acca-af480189a155>*
> )
>
>
> *Who: *         Lisa Li, Stanford University
>
>
> ------------------------------
>
> *Title:*   Diffusion-LM Improves Controllable Text Generation.
> *Abstract: *Controlling the behavior of language models (LMs) without
> re-training is a major open problem in natural language generation. While
> recent works have demonstrated successes on controlling simple sentence
> attributes (e.g., sentiment), there has been little progress on complex,
> fine-grained controls (e.g., syntactic structure). To address this
> challenge, we develop a new non-autoregressive language model based on
> continuous diffusions that we call Diffusion-LM. Building upon the recent
> successes of diffusion models in continuous domains, Diffusion-LM
> iteratively denoises a sequence of Gaussian vectors into word vectors,
> yielding a sequence of intermediate latent variables. The continuous,
> hierarchical nature of these intermediate variables enables a simple
> gradient-based algorithm to perform complex, controllable generation tasks.
> We demonstrate successful control of Diffusion-LM for six challenging
> fine-grained control tasks, significantly outperforming prior work.
>
> *Bio:* *Xiang Lisa Li* is a third-year PhD student in computer science at
> Stanford University, advised by Percy Liang and Tatsunori Hashimoto. She
> works on controllable text generation and efficient adaptation of
> pre-trained language models. Lisa is supported by a Stanford Graduate
> Fellowship and is the recipient of an EMNLP Best Paper award.
>
> *Host: David McAllester <mcallester at ttic.edu>*
>
>
> **************************************************************************************************
>
>
>
> The *TTIC Young Researcher Seminar Series* (http://www.ttic.edu/young-
> researcher.php) features talks by Ph.D. students and postdocs whose
> research is of broad interest to the computer science community.
> The series provides an opportunity for early-career researchers to
> present recent work to and meet with students and faculty at TTIC and
> nearby universities.
>
>
> The seminars are typically held on Wednesdays at 10:30am in TTIC Room 530.
>
> For additional information, please contact *David McAllester *(
> mcallester at ttic.edu).
>
>
>
> Mary C. Marre
> Faculty Administrative Support
> *Toyota Technological Institute*
> *6045 S. Kenwood Avenue, Rm 517*
> *Chicago, IL  60637*
> *773-834-1757*
> *mmarre at ttic.edu <mmarre at ttic.edu>*
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/theory/attachments/20221115/f7e15fe0/attachment-0001.html>