[CS] REMINDER: Kuntai Du Candidacy Exam/Nov 11th

[via cs]

This is an announcement of Kuntai Du's Candidacy Exam.
===============================================
Candidate: Kuntai Du

Date: Monday, November 11

Time:   2:00PM -3:00PM CST

Remote Location:  https://uchicago.zoom.us/j/4015085138?pwd=eU5ZNzc3Mmg3bTFiQ2E3ejZqSXlWQT09

Title: Optimizing Tensor Communication for Distributed Inference

Abstract: Distributed DNN inference pipelines like video analytics and LLM inference often require transferring large tensor data, such as video pixels or KV caches. However, the network bandwidth is limited, making it infeasible to transmit all data in real-time, necessitating an efficient communication support for such distributed inference pipeline. Prior work uses strategies like sender-side heuristics (e.g., content-based video compression or selective KV cache eviction) and periodic profiling to optimize for tensor communication. However, we argue that the system needs to be built based on real-time feedback from the receiver so that the system prioritizes the transfer of those data that are needed by the receiver. This allows us to transmit large tensors in real-time under limited network bandwidth without significantly degrading the performance of distributed DNN inference pipelines.

Advisors: Junchen Jiang

Committee Members: Junchen Jiang, Ariel Holtzman, Shan Lu, Ganesh Ananthanarayanan, Ion Stoica.