[CS] Kuntai Du Dissertation Defense/Jun 6, 2025

[via cs]

This is an announcement of Kuntai Du's Dissertation Defense.
===============================================
Candidate: Kuntai Du

Date: Friday, June 06, 2025

Time: 12 PM CST

Remote Location: https://uchicago.zoom.us/j/4015085138?pwd=eU5ZNzc3Mmg3bTFiQ2E3ejZqSXlWQT09 


Title: Optimizing tensor transmission for distributed machine learning inference pipeline via real-time feedback

Abstract: In order to fully take advantage of hardware capability, machine learning pipelines, including video analytics pipeline and LLM inference pipeline, are becoming more and more distributed. We focus on optimizing the tensor transmission in these pipelines. Our key observation is that: the end-to-end system can achieve much higher performance if the tensor transmission is driven by the feedback from the tensor receiver side, but this feedback is delay-sensitive and must be obtained in real-time. This thesis identifies such feedback, namely region-of-interest in video analytics pipeline, and KV cache hit information in LLM inference pipeline, and proposes real-time feedback-driven tensor transmission that maximally realizes the potential of this feedback.  



Advisors: Junchen Jiang

Committee Members: Junchen Jiang, Ariel Holtzman, Shan Lu, Ganesh Ananthanarayanan, Ion Stoica.