[CS] Yihua Cheng Candidacy Exam/Dec 2, 2024

[via cs]

This is an announcement of Yihua Cheng's Candidacy Exam.
===============================================
Candidate: Yihua Cheng

Date: Monday, December 02, 2024

Time: 11 am CST

Remote Location: https://uchicago.zoom.us/j/3150893650?pwd=RFgxNEE2MUQ0QzlsVXF3Ym94bDQ2Zz09

Location: JCL 298

Title: Revolutionizing Large Language Model Serving with Knowledge Delivery Network

Abstract: As the use of large language models (LLMs) expands rapidly, so does the range of knowledge needed to supplement various LLM queries. Thus, enabling modular and efficient injection of new knowledge in LLM inference is critical. We argue that compared to the more popular fine-tuning and in-context learning, using KV caches as the medium of knowledge could simultaneously improve the modularity of knowledge injection and the efficiency of LLM serving with low cost and fast response. To make it practical, we envision a Knowledge Delivery Network (KDN), a new component in LLM services that dynamically optimizes the storage, transfer, and composition of KV cache across LLM engines and other compute and storage resources. Just like content delivery networks (CDNs), such as Akamai, enabled the success of the Internet ecosystem through their efficient data delivery, KDNs will be critical to the success of LLM applications through their efficient knowledge delivery.

Advisors: Junchen Jiang

Committee Members: Junchen Jiang, Kexin Pei, Hui Zhang