[Colloquium] 2/27 Empowering Medical LLMs: Holistic Evaluation and Clinician-Guided Instruction Tuning

[Markayla Epps via Colloquium]

UNIVERSITY OF CHICAGO
COMPUTER SCIENCE DEPARTMENT
PRESENTS

Hejie Cu
Postdoctoral Scholar
Stanford University

Thursday, February 6th, 2025
2:00PM – 3:00PM CT .
Location : TTIC 530
Zoom Link : Click Here <https://uchicago.zoom.us/j/97708787366?pwd=qD9KdtzwGCTPuciAfr6iaap3NubZgy.1>

Title: Empowering Medical LLMs: Holistic Evaluation and Clinician-Guided Instruction Tuning

Abstract: Recent advancements in large language models (LLMs) and multimodal foundation models have shown remarkable potential for transforming healthcare applications. While these models demonstrate impressive performance on standardized medical exams, their real-world clinical utility remains underexplored. This talk presents two complementary frameworks addressing critical gaps in medical AI development and evaluation. First, we introduce a comprehensive benchmarking framework for evaluating LLMs across 121 healthcare tasks derived from clinician-validated taxonomies. Unlike existing evaluations that rely primarily on standardized tests, this framework assesses model performance on real-world medical scenarios: clinical decision support, note generation, patient communication, research assistance, and administrative workflows. Second, we present a novel data-centric approach that incorporates clinician preferences into biomedical visual instruction tuning. By integrating expert knowledge in both data generation and selection processes, this method produces high-quality instruction datasets that significantly enhance model performance in visual medical tasks. Together, these frameworks advance the field toward more clinically relevant and trustworthy AI systems for healthcare applications.

Bio: Hejie is a Postdoctoral Scholar working on AI for Healthcare and Trustworthy Machine Learning at Stanford University. Her research interests focuses on Large Language Model (LLM) Evaluation and Post-training for Healthcare.

 [Hejie Cui]
[image001.png]





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/colloquium/attachments/20250225/cec716d5/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 24077 bytes
Desc: image001.jpg
URL: <http://mailman.cs.uchicago.edu/pipermail/colloquium/attachments/20250225/cec716d5/attachment-0001.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 263 bytes
Desc: image003.png
URL: <http://mailman.cs.uchicago.edu/pipermail/colloquium/attachments/20250225/cec716d5/attachment-0001.png>