[Colloquium] Zhuokai Zhao Candidacy Exam/Mar 28, 2024

[Megan Woodward]

This is an announcement of Zhuokai Zhao's Candidacy Exam.
===============================================
Candidate: Zhuokai Zhao

Date: Thursday, March 28, 2024

Time: 10 am CT

Location: JCL 356

Title: Enhanced Data Utilization for Efficient and Robust Deep Learning Systems

Abstract: Deep learning (DL) has made significant impacts in many domains, including computer vision (CV), natural language processing (NLP), recommender systems, and many others. Besides the breakthroughs made to the model architectures, data has been another fundamental factor that significantly impacts the model performance. Such emphasize on data has been widely recognized as data-centric artificial intelligence (AI).

Despite being an emerging concept, studies focusing on developing novel data utilization strategies that boost model performance without changes to its architecture are still lacking. Therefore, in this thesis, we propose novel data utilization algorithms in both learning and inference schemes within the modern deep learning systems to improve model performance, robustness as well as develop more well-rounded model evaluations.

In the learning scheme, we propose two novel strategies that target both data-scarcity and data-abundance, which are two opposite dilemma commonly found in vision models when applied to real-world application domains where labeled data is expensive to obtain; and recommender systems where model performance saturates with abundant amount of user-item interaction data. Specifically, the former is often referred to as active learning (AL), where we propose \textit{Direct Acquisition Optimization} (\dao), a novel AL algorithm that optimizes sample selections based on expected true loss reduction. And for the latter, we propose \textit{User-Centric~Ranking} (\ucr), an alternative
data formulation strategy that is based on the transposed view of the dyadic user-item interactions.

In the inference scheme, we explore how novel data utilization leads to improved model performance and robustness, as well as more well-rounded model evaluation pipeline, without the need of acquiring new data or conducting additional fine-tuning. Specifically, we introduce \textit{Hallucination Reduction through Adaptive Focal-Contrast decoding} (\halc), a novel decoding strategy that utilizes fine-grained visual context to help pretrained large vision-language models (LVLMs) mitigate object hallucinations (OH) and produce more trust-worthy outputs. And we develop \textit{Non-Equivariance Revealed on Orbits} (\nero), a novel model evaluation tool that employs a combination of task-agnostic interactive interface and task-dependent visualizations to intricately evaluate and interpret model behaviors through analyzing its equivariance on purposefully designed data permutations.

In summary, this thesis introduces novel data utilization algorithms aimed at enhancing model performance, robustness, and evaluation in deep learning systems, without altering their architecture. It addresses the challenges of data scarcity and abundance in CV and recommender systems by proposing two learning strategies: \dao for active learning, targeting efficient sample selection, and \ucr for rethinking data formulation in recommender systems. Additionally, it explores novel inference strategies such as \halc, which improves output trustworthiness of vision-language models by mitigating object hallucinations, and \nero, an evaluation tool that enhances model interpretability and robustness by analyzing behavior on specially designed data permutations. These contributions highlight the significance of data-centric approaches in advancing deep learning models without necessitating architectural modifications.

Advisors: Yuxin Chen

Committee Members: Yuxin Chen, Bo Li, and Michael Maire



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/colloquium/attachments/20240327/a20d0027/attachment.html>