[Colloquium] TODAY-Do language models practice what they preach?

[Catherine Clemons via Colloquium]

UChicago/TTIC NLP Seminar Presents


Julia Watson

PhD Candidate

University of Toronto


Thursday, November 21st

2:00pm - 3:00pm

In Person: John Crerar Library 298

Zoom link: https://uchicago.zoom.us/j/97708787366?pwd=qD9KdtzwGCTPuciAfr6iaap3NubZgy.1


Title: Do language models practice what they preach? Examining language ideologies about gendered language reform encoded in LLMs

Abstract: We study language ideologies in text produced by LLMs through a case study on English gendered language reform (related to role nouns like congressperson/-woman/-man, and singular they). First, we find political bias: when asked to use language that is “correct” or “natural”, LLMs use language most similarly to when asked to align with conservative (vs. progressive) values. This shows how LLMs’ metalinguistic preferences can implicitly communicate the language ideologies of a particular political group, even in seemingly non-political contexts. Second, we find LLMs exhibit internal inconsistency: LLMs use gender-neutral variants more often when more explicit metalinguistic context is provided. This shows how the language ideologies expressed in text produced by LLMs can vary, which may be unexpected to users. We discuss the broader implications of these findings for value alignment.

Bio: I'm Julia Watson (she/her), a PhD candidate in Computer Science at the University of Toronto. My background is in linguistics and computer science, and my research integrates ideas from these fields. Most recently, my research focuses on formalizing how language communicates social meaning, paving the way for more equitable and socially intelligent technology.

[https://lh5.googleusercontent.com/nfumH4QNAc4chfOGMou3CiZq0vH7DTsj2ZYNvL6W12TR6yzT7_dgexwjJTVB5u2wWJo0KmFE5yNtyaEyaSv6PhUvrXMFflc1NyM0TSVupGym7X80JdLVolx2zk2bOCOMMA=w1280]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/colloquium/attachments/20241121/628ee7bc/attachment-0001.html>