<div dir="ltr"><div class="gmail_default"><div><font face="arial, sans-serif" color="#000000"><b>When:</b>           Friday, April 22nd at <span style="background-color:rgb(255,255,0)"><b>12:30pm CT</b></span></font></div><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> <br><b>Where:</b><b>  </b>       Talk will be given <font style="font-weight:bold"><u style="background-color:rgb(255,255,0)">live, in-person</u></font><font style="font-weight:bold"> </font>at</font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000">                     TTIC, 6045 S. Kenwood Avenue</font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000">                      5th Floor, Room 530<b>              </b>   </font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> </font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"><b>Virtually:</b>      via zoom: <a href="https://uchicagogroup.zoom.us/meeting/register/tJIvduChqjMiHNTEa-QzLrhMT1KJWHxWHqUh" target="_blank"><b>register in advance here</b></a> </font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> </font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"><b>Who:</b><b> </b>            Bradly Stadie, TTIC</font></p><table border="0" cellspacing="0" cellpadding="0" width="0" style="width:0in;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial;border-collapse:collapse"><tbody><tr><td style="padding:0in"></td></tr></tbody></table><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> </font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font color="#000000"><font face="arial, sans-serif"><b>Title:</b><b> </b>            </font></font><font face="arial, sans-serif">Frontiers of robotic intelligence</font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><br></font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><b>Abstract:      </b>When I joined TTIC two years ago, I set out to examine two key problems in robotics: goal-based learning and sim2real transfer. As my time at TTIC wraps up, I want to look back at the progress we’ve made in these areas during the past two years.</font></p><font face="arial, sans-serif"><br>In goal-based learning, we have introduced L3P, a new method that pairs non-parametric graph search with low-level reinforcement learning. Ultimately, this algorithm was bottlenecked by the difficulty in learning to accomplish individual goals corresponding to different nodes in the graph. While investigating this problem, I noticed that Hindsight Experience Replay performs much worse with (0, 1) rewards than with (-1, 0) rewards, even though this is a simple linear shift in reward scale! Trying to understand why this happens took the better part of a year, and resulted in some new theory that explains the optimal reward scale for goal-based RL algorithms. We choose to name the resulting algorithm Hindsight Divergence Minimization (HDM).<br><br>Meanwhile, in Sim2Real, we introduced Invariance Through Inference (ITI). This algorithm has a deep connection to generative models, and allows us to treat generated samples in a domain-independent fashion. As a result, our RL algorithms can now leverage features from offline ancillary datasets, in a fashion similar to recent successes in contrastive learning. Visualizing PCA plots of feature dimensions before and after training, we see that ITI is much better than prior Sim2Real methods at projecting samples from different environments onto a shared latent structure. This enables our algorithm to learn rich transferable features, and consequently we are able to train a policy on a simulated Fetch robot and transfer it onto a real world UR5. </font><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"> </font></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><br></font></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><span style="color:rgb(34,34,34)"><font face="arial, sans-serif"><br></font></span></p><p class="MsoNormal" style="margin:0in;color:rgb(80,0,80);line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><b><span style="color:black"><font face="arial, sans-serif">*********************************************************************************************</font></span></b></p><p class="MsoNormal" style="margin:0in 0in 8pt;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial;line-height:15.6933px"><b><font face="arial, sans-serif" color="#000000">Presence at TTIC requires being fully vaccinated for COVID-19 or having a TTIC or UChicago-approved exemption. Masks are optional in all common areas. Full visitor guidance available at <a href="http://ttic.edu/visitors" target="_blank">ttic.edu/visitors</a>.</font></b></p><p class="MsoNormal" style="margin:0in 0in 12pt;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial;line-height:15.6933px"><b><font face="arial, sans-serif" color="#000000">*********************************************************************************************</font></b></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><b><i><font face="arial, sans-serif" color="#000000">Research at TTIC Seminar Series</font></i></b></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> </font></p><font face="arial, sans-serif">TTIC is hosting a weekly seminar series presenting the research currently underway at the Institute. Every week a different TTIC faculty member will present their research.  The lectures are intended both for students seeking research topics and advisors, and for the general TTIC and University of Chicago communities interested in hearing what their colleagues are up to. <br><br><br>To receive announcements about the seminar series, please subscribe to the mailing list: <a href="https://groups.google.com/a/ttic.edu/group/talks/subscribe" target="_blank">https://groups.google.com/a/ttic.edu/group/talks/subscribe</a><br><br>Speaker details can be found at: <a href="https://ttic.edu/research-at-ttic/" target="_blank">https://ttic.edu/research-at-ttic/</a><br><br></font><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif">For additional questions, please contact Nathan Srebro at <a href="mailto:nati@ttic.edu" target="_blank">nati@ttic.edu</a>.<font color="#500050"> </font></font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font color="#500050"><br></font></font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font color="#500050"><br></font></font></p><p class="MsoNormal" style="margin:0in;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font color="#500050"><br></font></font></p></div><div><div dir="ltr" data-smartmail="gmail_signature"><div dir="ltr"><b style="background-color:rgb(255,255,255)"><font color="#3d85c6" face="arial, sans-serif">Brandie Jones </font></b><div><div><i><b style="background-color:rgb(255,255,255)"><font color="#3d85c6" face="arial, sans-serif">Administrative Assistant</font></b></i></div><div><span style="background-color:rgb(255,255,255)"><font color="#3d85c6" face="arial, sans-serif">Toyota Technological Institute</font></span></div><div><span style="background-color:rgb(255,255,255)"><font color="#3d85c6" face="arial, sans-serif">6045 S. Kenwood Avenue</font></span></div><div><span style="background-color:rgb(255,255,255)"><font color="#3d85c6" face="arial, sans-serif">Chicago, IL  60637</font></span></div></div><div><span style="background-color:rgb(255,255,255)"><font color="#3d85c6" face="arial, sans-serif"><a href="http://www.ttic.edu" target="_blank">www.ttic.edu</a> </font></span></div></div></div></div></div>