<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<i class="">Data Science Institute/Computer Science Candidate Seminar</i>
<div class=""><br class="">
</div>
<div class=""><b class="">Hao Zhang</b></div>
<div class="">
<div class=""><b class="">Postdoctoral Researcher</b><br class="">
</div>
<div class=""><b class="">University of California, Berkeley</b><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><b class="">TODAY Monday, April 4th</b></div>
<div class=""><b class="">3:00 p.m. - 4:00 p.m.</b></div>
<div class=""><b class="">In Person: John Crerar Library, Room 390</b></div>
<div class=""><b class="">Remote: <a href="http://live.cs.uchicago.edu/haozhang/" target="_blank" class="">Live Stream</a> or <a href="https://urldefense.com/v3/__https://uchicago.zoom.us/j/94061294260?pwd=YkFYWDE2MXJvaFMxS2thZFB5cC9tdz09__;!!BpyFHLRN4TMTrA!vLO9PBjeXHJ0aZNOdLwKjgyoHRGtEVj4fAngbpuw6OT54AuNvGfivTaOqx-ySjw7sw$" target="_blank" class="">Zoom</a> (details
below)</b></div>
<div class=""><b class=""><br class="">
</b></div>
<div class=""><b class="">Machine Learning Parallelization Could Be Automated, Performant, and Easy-to-use<br class="">
</b></div>
<div class=""><br class="">
</div>
<div class="">As models and data grow bigger, ML parallelization is more essential than ever. However, the amount of engineering effort and domain knowledge required for scaling up ML is often underestimated. The marginal cost for developing specialized systems
with hand-tuned parallel strategies is extremely high in the face of emerging models and heterogeneous cluster setups.<br class="">
<br class="">
In this talk, I will present a better way to build better ML systems. I view ML system building as an optimation over a parallel strategy space, with the objective of maximizing the system “goodput”, conditioned on model and cluster configurations. I show that
by formulating each piece in the optimization as math representations, we can make it solvable using existing tools. Unlike specialized systems, this formulation enables building generic ML compilers that automate ML parallelization, generalize to many models,
and achieve strong performance, simultaneously. In particular, I’ll describe two compiler systems: Alpa and Cavs, which automate model parallelism on large-scale distributed clusters, and the batching of dynamic neural network computation on accelerators,
respectively. My open-source artifacts have been used by organizations such as AI2, Meta, and Google, and parts of my research have been commercialized at multiple start-ups including Petuum and AnyScale.<br class="">
<br class="">
<b class="">Bio</b>: <a href="https://urldefense.com/v3/__https://people.eecs.berkeley.edu/*hao/__;fg!!BpyFHLRN4TMTrA!vLO9PBjeXHJ0aZNOdLwKjgyoHRGtEVj4fAngbpuw6OT54AuNvGfivTaOqx_etOO_YA$" class="">Hao Zhang</a> is a postdoc researcher at UC Berkeley working
with Ion Stoica. He completed his Ph.D. at CMU where he worked with Eric Xing. His research interests are in the intersection of machine learning and systems, with the focus on improving the performance and ease-of-use of today’s distributed ML systems. Hao’s
research has been recognized with an NVIDIA pioneer research award at NeurIPS’17, and the Jay Lepreau best paper award at OSDI’21.<br class="">
</div>
</div>
<div class=""><br class="">
</div>
<div class=""><b class="">Host</b>: Sanjay Krishnan</div>
<div class=""><br class="">
</div>
<div class=""><b class="">Zoom Info:</b></div>
<div class=""><a href="https://urldefense.com/v3/__https://uchicago.zoom.us/j/94061294260?pwd=YkFYWDE2MXJvaFMxS2thZFB5cC9tdz09__;!!BpyFHLRN4TMTrA!vLO9PBjeXHJ0aZNOdLwKjgyoHRGtEVj4fAngbpuw6OT54AuNvGfivTaOqx-ySjw7sw$" class="">https://uchicago.zoom.us/j/94061294260?pwd=YkFYWDE2MXJvaFMxS2thZFB5cC9tdz09</a><br class="">
Meeting ID: 940 6129 4260<br class="">
Password: ds2022</div>
<br clear="all" class="">
<div class=""><br class="">
</div>
--- <br class="">
<div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature">
<div dir="ltr" class="">
<div dir="ltr" class="">
<div dir="ltr" class="">
<div dir="ltr" class="">
<div dir="ltr" class="">Holly Santos</div>
</div>
</div>
</div>
</div>
</div>
<div class="">
<div>Executive Assistant to Michael J. Franklin, Chairman<br class="">
Department of Computer Science<br class="">
The University of Chicago<br class="">
5730 S Ellis Ave-217 Chicago, IL 60637<br class="">
P: 773-834-8977<br class="">
<a href="mailto:hsantos@uchicago.edu" class="">hsantos@uchicago.edu</a></div>
<div class=""><br class="">
</div>
<br class="Apple-interchange-newline">
</div>
<br class="">
</body>
</html>