[CS] Jerry Xu MS Presentation/Feb 24, 2025

[via cs]

This is an announcement of Jerry Xu's MS Presentation
===============================================
Candidate: Jerry Xu

Date: Monday, February 24, 2025

Time: 9:30am CST

Location: 298

Remote: https://uchicago.zoom.us/j/95157375230?pwd=aWbHIbGayyoV6gN67CwaJwO6gDBvwN.1
Title: Accelerating GNN Aggregation Using a Vertex-Centric Approach

Abstract: While Graph Neural Networks (GNNs) serve as an important tool for learning graph-structured data, they present new challenges to existing architectures. As opposed to being dominated by regular tensor operations with high arithmetic intensity, GNN introduces message passing, which allows vertices to exchange information with their neighbors, producing irregular and data-dependent behaviors. These characteristics make it difficult to express the algorithms and optimize their performance.

We demonstrate that UpDown, with its fine-grained MIMD architecture, enables a vertex-centric programming approach that suits the pairwise messaging model and maximizes data reuse. GPUs, by comparison, are limited by their primitives which fail to effectively express message passing, leading to unnecessary materializations of intermediate results and high performance overhead. We evaluate the UpDown architecture on GCN, GIN, GAT, and AIMNet, a state-of-the-art chemistry model. Compared to the H100 GPU with the best implementations we experimented with from the Deep Graph Library, an UpDown system with an equal instruction issue rate can achieve 6.96x, 5.56x, and 19.32x improvements on GCN, GIN, and GAT aggregation, leading to forward pass speedups of 1.84x, 2.70x, and 4.03x assuming constant combination phase costs. For AIMNet, we observe a 34.25x aggregation speedup and a 5.18x application speedup. Additionally, we reduce traffic by exploiting intra-vertex reuse, dataset characteristics, and graph partitioning, increasing aggregation speedups by 27% on GAT, 50% on AIMNet, and 41% on GCN.

Advisor: Andrew Chien

Committee: Andrew Chien, Rick Stevens, Michael Maire, Yanjing Li