<html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div dir="auto" style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div dir="auto" class="" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); word-wrap: break-word; line-break: after-white-space;"><font face="Arial" class=""><b class="" style="font-size: 14px;">University of Chicago </b></font><b class="" style="font-family: Arial; font-size: 14px;">and Toyota Technological Institute at Chicago</b></div><b class="" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Arial; font-size: 14px;">Machine Learning Seminar Series</b><br class="" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Arial;"><br class=""><div dir="auto" class="" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); word-wrap: break-word; line-break: after-white-space;"><font face="Arial" class=""><br class=""><b class="" style="font-size: 14px;">Zhaoran Wang</b></font></div><div dir="auto" class="" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); word-wrap: break-word; line-break: after-white-space;"><font face="Arial" class=""><span class="" style="font-size: 14px;"><b class="">Northwestern</b></span></font></div><div dir="auto" class="" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); word-wrap: break-word; line-break: after-white-space;"><font class=""><br class=""><font color="#000000" class="" style="font-family: Arial;"><span class="" style="font-size: 14px;"><b class="">Friday, December 6 10:30 - 11:30 am<br class=""></b></span></font><font face="Arial" class="">TTIC, Rm 526</font></font></div><div dir="auto" class="" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); word-wrap: break-word; line-break: after-white-space;"><font class=""><br class=""><br class=""><b class="" style="font-family: Arial; font-size: 14px;">Title:</b></font></div><div dir="auto" class="" style="word-wrap: break-word; line-break: after-white-space;"><font color="#000000" face="Arial" class=""><span style="caret-color: rgb(0, 0, 0); font-size: 15.333333015441895px;" class=""><b class="">On the Computational and Statistical Efficiency of Policy Optimization in (Deep) Reinforcement Learning</b></span></font></div><div dir="auto" class="" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); word-wrap: break-word; line-break: after-white-space;"><font class=""><b class="" style="font-family: Arial;"><font face="Arial" class="" style="font-weight: normal;"><b class="" style="font-size: 15.333333015441895px;"><br class=""></b></font></b></font></div><div dir="auto" class="" style="word-wrap: break-word; line-break: after-white-space;"><font face="Arial" class=""><span class=""><font face="Arial" class="" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-size: 14px;"><b class="">Abstract:</b><br class=""></font><div class=""><font color="#000000" class=""><span style="caret-color: rgb(0, 0, 0); font-size: 14px;" class="">Coupled with powerful function approximators such as neural networks, policy optimization plays a key role in the tremendous empirical successes of deep reinforcement learning. In sharp contrast, the theoretical understandings of policy optimization remain rather limited from both the computational and statistical perspectives. From the perspective of computational efficiency, it remains unclear whether policy optimization converges to the globally optimal policy in a finite number of iterations, even given infinite data. From the perspective of statistical efficiency, it remains unclear how to attain the globally optimal policy with a finite regret or sample complexity.<br class=""> <br class="">To address the computational question, I will show that, under suitable conditions, natural policy gradient/proximal policy optimization/trust-region policy optimization (NPG/PPO/TRPO) converges to the globally optimal policy at a sublinear rate, even when it is coupled with neural networks. To address the statistical question, I will present an optimistic variant of NPG/PPO/TRPO, namely OPPO, which incorporates exploration in a principled manner and attains a \sqrt{T}-regret.<br class=""> <br class="">(Joint work with Qi Cai, Chi Jin, Jason Lee, Boyi Liu, Zhuoran Yang)</span></font></div><div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class=""><font color="#000000" class=""><span class="" style="font-size: 14px;"><b class=""><br class=""></b></span></font></div><div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class=""><font color="#000000" class=""><span class="" style="font-size: 14px;"><b class="">Bio:</b></span></font></div><div class=""><font class="" color="#000000"><span class=""><span class="" style="caret-color: rgb(0, 0, 0); font-size: 14px;">Zhaoran Wang is an assistant professor at Northwestern University, working at the interface of machine learning, statistics, and optimization. He is the recipient of the AISTATS (Artificial Intelligence and Statistics Conference) notable paper award, ASA (American Statistical Association) best student paper in statistical learning and data mining, INFORMS (Institute for Operations Research and the Management Sciences) best student paper finalist in data mining, and the Microsoft fellowship.<br class=""><b class="">Host: Mladen Kolar </b> </span></span></font></div><div class=""><font class="" color="#000000"><span class=""><span class="" style="caret-color: rgb(0, 0, 0); font-size: 14px;"><br class=""></span></span></font></div><div class=""><font class="" color="#000000"><span class=""><span class="" style="caret-color: rgb(0, 0, 0); font-size: 14px;"><br class=""></span></span></font></div><div class=""><font class="" color="#000000"><span class=""><span class="" style="caret-color: rgb(0, 0, 0); font-size: 14px;"><br class=""></span></span></font></div><div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class=""><font color="#000000" class=""><span class="" style="font-size: 14px;"><b class=""></b></span></font></div></span></font></div></div></body></html>