[Theory] NOW: 6/10 Thesis Defense: Lingyu Gao, TTIC

[Mary Marre via Theory]

*When*:     Monday, June 10th from *12 **- 2 pm CT*

*Virtually*: via *Zoom*
<https://uchicago.zoom.us/j/98062714782?pwd=UFU2b1I2SkY1ZUZYWlFCaEFjN3Zvdz09>


*Who*:       Lingyu Gao, TTIC

------------------------------
*Title:* Harnessing the Intrinsic Knowledge of Pretrained Language Models
for Challenging Text Classification Settings

*Abstract: *Text classification is essential for applications such as
sentiment analysis and toxic text filtering, yet it faces challenges due to
the complexity and ambiguity of natural language. Recent advancements in
deep learning, particularly transformer architectures and large-scale
pretraining, have significantly improved performance in NLP tasks,
including zero-shot scenarios without available training data. In this
thesis, we explore three challenging settings in text classification,
leveraging the intrinsic knowledge of pretrained language models (PLMs).
Firstly, to address the challenge of selecting misleading yet incorrect
distractors for cloze questions, we develop models that utilize features
designed with contextualized word representations derived from PLMs,
achieving performance that rivals or surpasses human accuracy. Secondly, to
improve generalization to unseen labels, we create small finetuning
datasets with domain-independent task label descriptions, enhancing model
performance and robustness. Lastly, we tackle the sensitivity of PLMs to
in-context learning prompts by selecting effective demonstrations, focusing
on misclassified examples and resolving model ambiguity about test example
labels.

*Thesis Committee: **Kevin Gimpel <kgimpel at ttic.edu> *(Thesis Advisor), Karen
Livescu, Debanjan Ghosh.



Mary C. Marre
Faculty Administrative Support
*Toyota Technological Institute*
*6045 S. Kenwood Avenue, Rm 517*
*Chicago, IL  60637*
*773-834-1757*
*mmarre at ttic.edu <mmarre at ttic.edu>*


On Sun, Jun 9, 2024 at 3:43 PM Mary Marre <mmarre at ttic.edu> wrote:

> *When*:     Monday, June 10th from *12 **- 2 pm CT*
>
> *Virtually*: via *Zoom*
> <https://uchicago.zoom.us/j/98062714782?pwd=UFU2b1I2SkY1ZUZYWlFCaEFjN3Zvdz09>
>
>
> *Who*:       Lingyu Gao, TTIC
>
> ------------------------------
> *Title:* Harnessing the Intrinsic Knowledge of Pretrained Language Models
> for Challenging Text Classification Settings
>
> *Abstract: *Text classification is essential for applications such as
> sentiment analysis and toxic text filtering, yet it faces challenges due to
> the complexity and ambiguity of natural language. Recent advancements in
> deep learning, particularly transformer architectures and large-scale
> pretraining, have significantly improved performance in NLP tasks,
> including zero-shot scenarios without available training data. In this
> thesis, we explore three challenging settings in text classification,
> leveraging the intrinsic knowledge of pretrained language models (PLMs).
> Firstly, to address the challenge of selecting misleading yet incorrect
> distractors for cloze questions, we develop models that utilize features
> designed with contextualized word representations derived from PLMs,
> achieving performance that rivals or surpasses human accuracy. Secondly, to
> improve generalization to unseen labels, we create small finetuning
> datasets with domain-independent task label descriptions, enhancing model
> performance and robustness. Lastly, we tackle the sensitivity of PLMs to
> in-context learning prompts by selecting effective demonstrations, focusing
> on misclassified examples and resolving model ambiguity about test example
> labels.
>
> *Thesis Committee: **Kevin Gimpel <kgimpel at ttic.edu> *(Thesis Advisor), Karen
> Livescu, Debanjan Ghosh.
>
>
>
>
> Mary C. Marre
> Faculty Administrative Support
> *Toyota Technological Institute*
> *6045 S. Kenwood Avenue, Rm 517*
> *Chicago, IL  60637*
> *773-834-1757*
> *mmarre at ttic.edu <mmarre at ttic.edu>*
>
>
> On Thu, Jun 6, 2024 at 7:20 PM Mary Marre <mmarre at ttic.edu> wrote:
>
>> *When*:     Monday, June 10th from *12 **- 2 pm CT*
>>
>> *Virtually*: via *Zoom*
>> <https://uchicago.zoom.us/j/98062714782?pwd=UFU2b1I2SkY1ZUZYWlFCaEFjN3Zvdz09>
>>
>>
>> *Who*:       Lingyu Gao, TTIC
>>
>> ------------------------------
>> *Title:* Harnessing the Intrinsic Knowledge of Pretrained Language
>> Models for Challenging Text Classification Settings
>>
>> *Abstract: *Text classification is essential for applications such as
>> sentiment analysis and toxic text filtering, yet it faces challenges due to
>> the complexity and ambiguity of natural language. Recent advancements in
>> deep learning, particularly transformer architectures and large-scale
>> pretraining, have significantly improved performance in NLP tasks,
>> including zero-shot scenarios without available training data. In this
>> thesis, we explore three challenging settings in text classification,
>> leveraging the intrinsic knowledge of pretrained language models (PLMs).
>> Firstly, to address the challenge of selecting misleading yet incorrect
>> distractors for cloze questions, we develop models that utilize features
>> designed with contextualized word representations derived from PLMs,
>> achieving performance that rivals or surpasses human accuracy. Secondly, to
>> improve generalization to unseen labels, we create small finetuning
>> datasets with domain-independent task label descriptions, enhancing model
>> performance and robustness. Lastly, we tackle the sensitivity of PLMs to
>> in-context learning prompts by selecting effective demonstrations, focusing
>> on misclassified examples and resolving model ambiguity about test example
>> labels.
>>
>> *Thesis Committee: **Kevin Gimpel <kgimpel at ttic.edu> *(Thesis Advisor), Karen
>> Livescu, Debanjan Ghosh.
>>
>>
>>
>>
>>
>> Mary C. Marre
>> Faculty Administrative Support
>> *Toyota Technological Institute*
>> *6045 S. Kenwood Avenue, Rm 517*
>> *Chicago, IL  60637*
>> *773-834-1757*
>> *mmarre at ttic.edu <mmarre at ttic.edu>*
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/theory/attachments/20240610/c4162de9/attachment.html>