[Theory] REMINDER: 3/28 Talks at TTIC: Beidi Chen, Stanford University

[Mary Marre]

*When:*        Monday, March 28th at* 11:30 am CT*


*Where:       *Talk will be given *live, in-person* at

                   TTIC, 6045 S. Kenwood Avenue

                   5th Floor, Room 530


*Where:*       Zoom Virtual Talk (*register in advance here
<https://uchicagogroup.zoom.us/webinar/register/WN_Ad_x3I08RUuS99gZKU8h6A>*)


*Who: *         Beidi Chen, Stanford University



*Title:          *Randomized Algorithms for Efficient Machine Learning
Systems

*Abstract:*
Machine learning (ML) has demonstrated great promise in scientific
discovery, healthcare, and education, especially with the rise of large
neural networks. However, large models trained on complex and rapidly
growing data consume enormous computational resources. In this talk, I will
describe my work on exploiting model sparsity with randomized algorithms to
accelerate large ML systems on current hardware with no drop in accuracy.

I will start by describing SLIDE, an open-source system for efficient
sparse neural network training on CPUs that has been deployed by major
technology companies and academic labs. SLIDE blends Locality Sensitive
Hashing with multi-core parallelism and workload optimization to
drastically reduce computations. SLIDE trains industry-scale recommendation
models on a 44 core CPU 3.5x faster than TensorFlow on V100 GPU with no
drop in accuracy.

Next, I will present Pixelated Butterfly, a simple yet efficient sparse
training framework on GPUs. It uses a simple static block-sparse pattern
based on butterfly and low-rank matrices, taking into account GPU
block-oriented efficiency. Pixelated Butterfly trains up to 2.5x faster
(wall-clock) than the dense Vision Transformer and GPT-2 counterparts with
no drop in accuracy.

I will conclude by outlining future research directions for further
accelerating ML pipelines and making ML more accessible to the general
community, such as software-hardware co-design, data-centric AI, and ML for
scientific computing and medical imaging.

*Bio: *
Beidi Chen is a postdoctoral scholar in the CS department at Stanford
University, working with Prof. Christopher Ré. Her research focuses on
large-scale machine learning and deep learning. Specifically, she designs
and optimizes randomized algorithms (algorithm-hardware co-design) to
accelerate large machine learning systems for real-world problems. Prior to
joining Stanford, she received her Ph.D. from the CS department at Rice
University, advised by Prof. Anshumali Shrivastava. She received a BS in
EECS from UC Berkeley. She has held internships in Microsoft Research,
NVIDIA Research, and Amazon AI. Her work has won Best Paper awards at LISA
and IISA. She was selected as a Rising Star in EECS by MIT and UIUC.

*Host: David McAllester <mcallester at ttic.edu>*




Mary C. Marre
Faculty Administrative Support
*Toyota Technological Institute*
*6045 S. Kenwood Avenue*
*Chicago, IL  60637*
*mmarre at ttic.edu <mmarre at ttic.edu>*


On Tue, Mar 22, 2022 at 5:38 PM Mary Marre <mmarre at ttic.edu> wrote:

> *When:*        Monday, March 28th at* 11:30 am CT*
>
>
> *Where:       *Talk will be given *live, in-person* at
>
>                    TTIC, 6045 S. Kenwood Avenue
>
>                    5th Floor, Room 530
>
>
> *Where:*       Zoom Virtual Talk (*register in advance here
> <https://uchicagogroup.zoom.us/webinar/register/WN_Ad_x3I08RUuS99gZKU8h6A>*
> )
>
>
> *Who: *         Beidi Chen, Stanford University
>
>
>
> *Title:          *Randomized Algorithms for Efficient Machine Learning
> Systems
>
> *Abstract:*
> Machine learning (ML) has demonstrated great promise in scientific
> discovery, healthcare, and education, especially with the rise of large
> neural networks. However, large models trained on complex and rapidly
> growing data consume enormous computational resources. In this talk, I will
> describe my work on exploiting model sparsity with randomized algorithms to
> accelerate large ML systems on current hardware with no drop in accuracy.
>
> I will start by describing SLIDE, an open-source system for efficient
> sparse neural network training on CPUs that has been deployed by major
> technology companies and academic labs. SLIDE blends Locality Sensitive
> Hashing with multi-core parallelism and workload optimization to
> drastically reduce computations. SLIDE trains industry-scale recommendation
> models on a 44 core CPU 3.5x faster than TensorFlow on V100 GPU with no
> drop in accuracy.
>
> Next, I will present Pixelated Butterfly, a simple yet efficient sparse
> training framework on GPUs. It uses a simple static block-sparse pattern
> based on butterfly and low-rank matrices, taking into account GPU
> block-oriented efficiency. Pixelated Butterfly trains up to 2.5x faster
> (wall-clock) than the dense Vision Transformer and GPT-2 counterparts with
> no drop in accuracy.
>
> I will conclude by outlining future research directions for further
> accelerating ML pipelines and making ML more accessible to the general
> community, such as software-hardware co-design, data-centric AI, and ML for
> scientific computing and medical imaging.
>
> *Bio: *
> Beidi Chen is a postdoctoral scholar in the CS department at Stanford
> University, working with Prof. Christopher Ré. Her research focuses on
> large-scale machine learning and deep learning. Specifically, she designs
> and optimizes randomized algorithms (algorithm-hardware co-design) to
> accelerate large machine learning systems for real-world problems. Prior to
> joining Stanford, she received her Ph.D. from the CS department at Rice
> University, advised by Prof. Anshumali Shrivastava. She received a BS in
> EECS from UC Berkeley. She has held internships in Microsoft Research,
> NVIDIA Research, and Amazon AI. Her work has won Best Paper awards at LISA
> and IISA. She was selected as a Rising Star in EECS by MIT and UIUC.
>
> *Host: David McAllester <mcallester at ttic.edu>*
>
>
>
> Mary C. Marre
> Faculty Administrative Support
> *Toyota Technological Institute*
> *6045 S. Kenwood Avenue*
> *Chicago, IL  60637*
> *mmarre at ttic.edu <mmarre at ttic.edu>*
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/theory/attachments/20220327/59c5c7b6/attachment.html>