<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div class="elementToProof" style="text-align: left; text-indent: 0px; line-height: 1.2; margin: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(87, 6, 6);">
<i>UNIVERSITY OF CHICAGO</i></div>
<div style="direction: ltr; text-align: left; text-indent: 0px; margin: 0in 0px; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(87, 6, 6);">
<i>COMPUTER SCIENCE DEPARTMENT</i></div>
<div style="direction: ltr; text-align: left; text-indent: 0px; margin: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(87, 6, 6);">
<i>PRESENTS</i></div>
<div style="direction: ltr; text-align: left; text-indent: 0px; margin: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<b><br>
</b></div>
<div class="elementToProof" style="direction: ltr; text-align: left; text-indent: 0px; margin: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<b>Xiangyu Zhang</b></div>
<div class="elementToProof" style="direction: ltr; text-align: left; text-indent: 0px; margin: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<b>Purdue University</b></div>
<div class="elementToProof" style="direction: ltr; text-align: left; text-indent: 0px; margin: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<b>Samuel Conte Professor</b></div>
<div style="direction: ltr; text-align: left; text-indent: 0px; margin: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<b><br>
</b></div>
<div class="elementToProof" style="direction: ltr; text-align: left; text-indent: 0px; margin: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<b>Tuesday, November 5th</b></div>
<div style="direction: ltr; text-align: left; text-indent: 0px; margin: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<b>12:30 pm - 1:30 pm </b></div>
<div class="elementToProof" style="direction: ltr; text-align: left; text-indent: 0px; margin: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<b>In Person: John Crerar Library Rm 298</b></div>
<div class="elementToProof" style="direction: ltr; text-align: left; text-indent: 0px; margin: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<b><br>
</b></div>
<div class="elementToProof" style="direction: ltr; text-align: left; text-indent: 0px; margin: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<b>Zoom link: <a href="https://uchicago.zoom.us/j/97013580765?pwd=7ATw1tnJYageS2SkhZfYV8KmMD2vVn.1#success" id="LPlnk462146" class="OWAAutoLink" title="https://uchicago.zoom.us/j/97013580765?pwd=7ATw1tnJYageS2SkhZfYV8KmMD2vVn.1#success" data-auth="NotApplicable">
https://uchicago.zoom.us/j/97013580765?pwd=7ATw1tnJYageS2SkhZfYV8KmMD2vVn.1#success</a></b></div>
<div class="elementToProof" style="direction: ltr; text-align: left; text-indent: 0px; margin: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<b>Meeting ID: 970 1358 0765<br>
Passcode: 783088</b></div>
<div class="elementToProof" style="direction: ltr; text-align: left; text-indent: 0px; margin: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<b><br>
</b></div>
<p class="elementToProof" style="text-align: left; text-indent: 0px; line-height: 1.2; margin-top: 0px; margin-bottom: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<b>Title: Reducing LLM Hallucination in Program Analysis Tasks</b></p>
<p class="elementToProof" style="text-align: left; text-indent: 0px; line-height: 1.2; margin-top: 0px; margin-bottom: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<b><br>
</b></p>
<p style="text-align: left; text-indent: 0px; line-height: 1.38; margin-top: 0px; margin-bottom: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<b>Abstract:</b> In this talk, I will present our recent efforts in reducing LLM hallucination in program analysis tasks such as</p>
<p style="text-align: left; text-indent: 0px; line-height: 1.38; margin-top: 0px; margin-bottom: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
decompilation, data-flow analysis, and bug finding. Although many have started to use LLMs and Code-Language models </p>
<p style="text-align: left; text-indent: 0px; line-height: 1.38; margin-top: 0px; margin-bottom: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
in program analysis and program transformation tasks, the results haven't met our expectations. The reason is that these</p>
<p style="text-align: left; text-indent: 0px; line-height: 1.38; margin-top: 0px; margin-bottom: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
large models hallucinate a lot in complex tasks. There are various reasons behind this. For example, these models treat</p>
<p style="text-align: left; text-indent: 0px; line-height: 1.38; margin-top: 0px; margin-bottom: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
programs no different from natural language texts during pretraining, although the former have a fundamentally different nature (e.g., </p>
<p style="text-align: left; text-indent: 0px; line-height: 1.38; margin-top: 0px; margin-bottom: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
due to loops, recursions, and modular design). In addition, the models usually have limited input sizes, which are insufficient for </p>
<p style="text-align: left; text-indent: 0px; line-height: 1.38; margin-top: 0px; margin-bottom: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
complex tasks. I will present a few methods we have developed to reduce hallucination in program analysis, including a novel </p>
<p style="text-align: left; text-indent: 0px; line-height: 1.38; margin-top: 0px; margin-bottom: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
pre-taining method that challenges the model to understand program semantics by understanding data-flow, a novel context propagation </p>
<p style="text-align: left; text-indent: 0px; line-height: 1.38; margin-top: 0px; margin-bottom: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
method that addresses model input limits, and a new end-to-end LLM based bug detection pipeline that does not directly prompt the </p>
<p class="elementToProof" style="text-align: left; text-indent: 0px; line-height: 1.38; margin-top: 0px; margin-bottom: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
LLM to find bugs, but rather requests the LLM to synthesize code to perform deterministic detection and result sanitization.</p>
<p class="elementToProof" style="text-align: left; text-indent: 0px; line-height: 1.38; margin-top: 0px; margin-bottom: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</p>
<div class="elementToProof" style="text-align: left; text-indent: 0px; line-height: 1.38; margin-top: 1em; margin-bottom: 1em; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<b>Bio:</b> Xiangyu Zhang is a Samuel Conte Professor at Purdue specializing in AI security, software analysis and cyber forensics. His work involves developing techniques to detect bugs, including security vulnerabilities, in traditional software systems as
well as AI models and systems, and to leverage AI techniques to perform software engineering and cybersecurity tasks. He has served as the Principal Investigator (PI) for numerous projects funded by organizations such as DARPA, IARPA, ONR, NSF, AirForce, and
industry.</div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; line-height: 1.38; margin-top: 1em; margin-bottom: 1em; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; line-height: 1.38; margin-top: 1em; margin-bottom: 1em; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<img id="image_0" width="211" height="215" style="width: 211px; height: 215px;" size="87648" contenttype="image/png" data-outlook-trace="F:1|T:1" src="cid:621feec0-742a-416a-a329-528aa938be8a"></div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; line-height: 1.38; margin-top: 1em; margin-bottom: 1em; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; line-height: 1.38; margin-top: 1em; margin-bottom: 1em; font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<b>Host: Kexin Pei</b></div>
<div class="elementToProof" style="font-family: Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div id="Signature" class="elementToProof"></div>
</body>
</html>