[CS] Tony Zhang MS Presentation/Apr 21, 2025

[via cs]

This is an announcement of Tony Zhang's MS Presentation
===============================================
Candidate: Tony Zhang

Date: Monday, April 21, 2025

Time: 11 am CST

Location: JCL 354

Title: Generating Unlimited Memory Level Parallelism with UpDown

Abstract: With the rapid growth of machine learning hardware demand, High Bandwidth Memory (HBM) has emerged as a prominent choice for architectural and system design considerations. HBM3e achieves a remarkable memory bandwidth of 1.2 TB/s per stack and is projected to attain 2+ TB/s on the roadmap. This substantial bandwidth enhancement presents a compelling opportunity to accelerate memory bandwidth-critical applications. Nevertheless, current CPUs with a multilevel cache hierarchy lack the requisite memory level parallelism to fully saturate such high bandwidths. We employ the Memory Parallelism Abstract Machine (MPAM) to highlight the inherent namespace that limits the memory level parallelism in architectures. Through MPAM, we investigate the hardware constraints of four commercial platforms. We also evaluate the UpDown architecture using MPAM. The UpDown architecture mitigates these namespace limitations and achieves unlimited memory level parallelism through the split transaction of DRAM requests, explicit compute-name synchronization of memory responses, and efficient memory parallelism scaling. As a result, a single UpDown lane can attain 3.5 times memory level parallelism over a single Out-of-Order core with a significantly smaller footprint. A 256-lane UpDown system achieves approximately 34 to 172 times speedup on STREAM, SPMV, and CNN compared to a single Out-of-Order core with prefetching, all while utilizing a comparable area. Furthermore, we demonstrate that UpDown can effectively scale out and saturate HBM3e and future HBM memory bandwidths at a low cost.

Advisor: Andrew Chien

Committee: Andrew Chien, Fred Chong, Rick Stevens