[Colloquium] 4/4: Data Science/CS Candidate Talk - Hao Zhang (UC Berkeley)

[Rob Mitchum]

*Data Science Institute/Computer Science Candidate Seminar*

*Hao Zhang*
*Postdoctoral Researcher*
*University of California, Berkeley*

*Monday, April 4th*
*3:00 p.m. - 4:00 p.m.*
*In Person: John Crerar Library, Room 390*
*Remote: Live Stream <http://live.cs.uchicago.edu/haozhang/> or Zoom
<https://uchicago.zoom.us/j/94061294260?pwd=YkFYWDE2MXJvaFMxS2thZFB5cC9tdz09>
(details
below)*


*Machine Learning Parallelization Could Be Automated, Performant, and
Easy-to-use*

As models and data grow bigger, ML parallelization is more essential than
ever. However, the amount of engineering effort and domain knowledge
required for scaling up ML is often underestimated. The marginal cost for
developing specialized systems with hand-tuned parallel strategies is
extremely high in the face of emerging models and heterogeneous cluster
setups.

In this talk, I will present a better way to build better ML systems. I
view ML system building as an optimation over a parallel strategy space,
with the objective of maximizing the system “goodput”, conditioned on model
and cluster configurations. I show that by formulating each piece in the
optimization as math representations, we can make it solvable using
existing tools. Unlike specialized systems, this formulation enables
building generic ML compilers that automate ML parallelization, generalize
to many models, and achieve strong performance, simultaneously. In
particular, I’ll describe two compiler systems: Alpa and Cavs, which
automate model parallelism on large-scale distributed clusters, and the
batching of dynamic neural network computation on accelerators,
respectively. My open-source artifacts have been used by organizations such
as AI2, Meta, and Google, and parts of my research have been commercialized
at multiple start-ups including Petuum and AnyScale.

*Bio*: Hao Zhang <https://people.eecs.berkeley.edu/~hao/> is a postdoc
researcher at UC Berkeley working with Ion Stoica. He completed his Ph.D.
at CMU where he worked with Eric Xing. His research interests are in the
intersection of machine learning and systems, with the focus on improving
the performance and ease-of-use of today’s distributed ML systems. Hao’s
research has been recognized with an NVIDIA pioneer research award at
NeurIPS’17, and the Jay Lepreau best paper award at OSDI’21.

*Host*: Sanjay Krishnan

*Zoom Info:*
https://uchicago.zoom.us/j/94061294260?pwd=YkFYWDE2MXJvaFMxS2thZFB5cC9tdz09
Meeting ID: 940 6129 4260
Password: ds2022


-- 
*Rob Mitchum*

*Associate Director of Communications for Data Science and Computing*
*University of Chicago*
*rmitchum at uchicago.edu <rmitchum at ci.uchicago.edu>*
*773-484-9890*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.uchicago.edu/pipermail/colloquium/attachments/20220331/34e55e13/attachment.html>