[Colloquium] Aswathy Ajith MS Presentation/May 28, 2024

[via Colloquium]

This is an announcement of Aswathy Ajith's MS Presentation
===============================================
Candidate: Aswathy Ajith

Date: Tuesday, May 28, 2024

Time:  1 pm CT

Location: JCL 390

Title: Domain Adaptability of Language Models and Scientific Information Extraction

Abstract: "The recent surge in interest in large language models and their general language capabilities calls into question the effectiveness of adapting them to tasks that require scientific know-how. Transformer- based masked language models such as BERT, trained on general corpora, have shown impressive performance on downstream tasks. It has also been demonstrated that the downstream task performance of such models can be improved by pretraining larger models for longer on more data. In this work, we empirically evaluate the extent to which these results extend to tasks in science. We use 14 domain-specific transformer-based models (including ScholarBERT, a new 770M-parameter science-focused masked language model pretrained on up to 225B tokens) to evaluate the impact of training data, model size, pretraining and finetuning time on 12 downstream scientific tasks. Interestingly, we find that increasing model size, training data, or compute time does not always lead to significant improvements (i.e., > 1% F1), if any, in scientific information extraction tasks. We offer plausible explanations for this surprising result. 

We also explore the extractive capabilities of generative models and introduce MatPropXtractor, an extraction system that uses GPT-3 to extract materials and their properties from materials science literature without fine-tuning. MatPropXtractor consists of a three-step pipeline that includes 1) a document selection tool to identify related articles; 2) a paragraph classifier to identify passages containing important materials properties; and 3) a property extractor exploiting in-context learning in GPT-3. We evaluate MatPropXtractor by applying it to five materials science papers. It can extract 154 material-property pairs from these papers and upon further evaluation by a domain expert, it was found to have an average precision of 72.73% to identify paragraphs that contain information of interest and an average precision of 56.7% on material-property identification."

Advisors: Ian Foster and Kyle CHard

Committee Members: Ian Foster, Kyle Chard, and Eamon Duede