<div dir="ltr"><div dir="ltr"><div class="gmail_default" style="font-size:small"><div class="gmail_default"><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><font face="arial, sans-serif" color="#000000"><font style="vertical-align:inherit"><font style="vertical-align:inherit"><b>When:</b> </font></font><font style="vertical-align:inherit"><font style="vertical-align:inherit"> Thursday, April 2nd at 11:00 am</font></font><br></font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> </font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font color="#000000"><font style="vertical-align:inherit"><font style="vertical-align:inherit"><b>Where:</b> </font></font></font>TTIC <Virtually> </font><a href="https://zoom.us/j/644963077" target="_blank" style="background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial;box-sizing:border-box;text-decoration-line:none;font-family:Lato,Helvetica,Arial;font-size:14px"><font color="#0000ff"><b>https://zoom.us/j/644963077</b></font></a></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> </font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;text-align:justify;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font color="#000000"><font style="vertical-align:inherit"><font style="vertical-align:inherit"><b>Who: </b> </font></font></font>Bradly Stadie, Vector Institute</font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;text-align:justify;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> </font></p></div><div class="gmail_default"><font face="arial, sans-serif" color="#000000"><br></font></div><div class="gmail_default"><div><div><font face="arial, sans-serif"><b>Title: </b>Weakly Supervised Reinforcement Learning </font></div><div><font face="arial, sans-serif"><br></font></div><div><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><font face="arial, sans-serif"><b>Abstract: </b>Consider a scenario wherein we are given the entire ImageNet training set without access to labels. What unsupervised features should we learn from this unlabeled data? In particular, which features should we learn that might be useful for downstream transfer to classification at test time? This problem, known as weakly supervised learning, has seen tremendous advancement in the past 18 months thanks to methods like Contrastive Predictive Coding. </font></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px;min-height:14px"><font face="arial, sans-serif"><br></font></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><font face="arial, sans-serif">In this talk, we will consider the analogous problem in a reinforcement learning (RL) setting. In the absence of any guidance, which tasks should an agent pursue during an unsupervised play phase? Can an agent learn unsupervised behaviors that are useful for downstream transfer to real tasks at test time? While the de facto method for solving this class of problems has long been curiosity methods, we argue that naively applying this approach leads to agents that don’t learn semantically meaningful behaviors. In other words, curiosity doesn’t provide the right inductive bias for downstream transfer. </font></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><font face="arial, sans-serif"><br></font></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><font face="arial, sans-serif">We propose two methods to fix curiosity's shortcomings. First, we leverage the notion of self-imitation learning to derive an analogue to curiosity in an abstract imitation space. This form of curiosity encourages agents to learn diverse behaviors with semantic differences. Second, we introduce the notion of goals to our agents. Even when we never tell agents which goals are interesting, we see that simply introducing the existence of goals provides enough inductive bias for agents to learn a meaningful distribution of interesting unsupervised behaviors. These behaviors can be readily leveraged to achieve a variety of difficult tasks at test time, including extremely challenging robotic navigation, manipulation, and locomotion tasks. </font></p></div></div><div><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div></div><div style="color:rgb(80,0,80)"><br></div><div style="color:rgb(80,0,80)"><b style="color:rgb(0,0,0);font-family:arial,sans-serif">Host:</b><span style="color:rgb(0,0,0);font-family:arial,sans-serif"> </span><a href="mailto:mwalter@ttic.com" target="_blank" style="font-family:arial,sans-serif">Matthew Walter</a> <br></div><div style="color:rgb(80,0,80)"> <br></div><div style="color:rgb(80,0,80)"><br></div><div style="color:rgb(80,0,80)">**********************************************************************************************************</div><div><font color="#500050"><b><i>Matthew Walter</i></b> is inviting you to a scheduled Zoom meeting.</font><br><br><font color="#500050">Topic: Talks at TTIC: Bradly Stadie, Vector Institute</font><br><font color="#500050">Time: Apr 2, 2020 11:00 AM Central Time (US and Canada)</font><br><br><b><font size="4" style="background-color:rgb(255,255,0)"><font color="#500050">Join Zoom Meeting</font><br><font color="#0000ff"><a href="https://zoom.us/j/644963077" target="_blank">https://zoom.us/j/644963077</a></font></font></b><br><br><font color="#500050">Meeting ID: 644 963 077</font><br><br><font color="#500050">One tap mobile</font><br><font color="#500050">+14086380968,,644963077# US (San Jose)</font><br><font color="#500050">+16468769923,,644963077# US (New York)</font><br><br><font color="#500050">Dial by your location</font><br><font color="#500050"> +1 408 638 0968 US (San Jose)</font><br><font color="#500050"> +1 646 876 9923 US (New York)</font><br><font color="#500050"> +1 669 900 6833 US (San Jose)</font><br><font color="#500050"> +1 253 215 8782 US</font><br><font color="#500050"> +1 301 715 8592 US</font><br><font color="#500050"> +1 312 626 6799 US (Chicago)</font><br><font color="#500050"> +1 346 248 7799 US (Houston)</font><br><font color="#500050">Meeting ID: 644 963 077</font></div><div style="color:rgb(80,0,80)"><br>Find your local number: <a href="https://zoom.us/u/akrIgOS2N" target="_blank">https://zoom.us/u/akrIgOS2N</a><br></div><div style="color:rgb(80,0,80)"><br></div><div style="color:rgb(80,0,80)"><br></div><div style="color:rgb(80,0,80)"><br></div></div></div></div></div></div></div></div><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><font face="arial, helvetica, sans-serif">Mary C. Marre</font><div><font face="arial, helvetica, sans-serif">Faculty Administrative Support</font></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6"><b>Toyota Technological Institute</b></font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">6045 S. Kenwood Avenue</font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Room 517</font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Chicago, IL 60637</font></i></div><div><i><font face="arial, helvetica, sans-serif">p:(773) 834-1757</font></i></div><div><i><font face="arial, helvetica, sans-serif">f: (773) 357-6970</font></i></div><div><b><i><a href="mailto:mmarre@ttic.edu" target="_blank"><font face="arial, helvetica, sans-serif">mmarre@ttic.edu</font></a></i></b></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Mar 27, 2020 at 4:36 PM Mary Marre <<a href="mailto:mmarre@ttic.edu">mmarre@ttic.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div><div><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><font face="arial, sans-serif" color="#000000"><font style="vertical-align:inherit"><font style="vertical-align:inherit"><b>When:</b> </font></font><font style="vertical-align:inherit"><font style="vertical-align:inherit"> Thursday, April 2nd at 11:00 am</font></font><br></font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> </font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font color="#000000"><font style="vertical-align:inherit"><font style="vertical-align:inherit"><b>Where:</b> </font></font></font>TTIC <Virtually> </font><a href="https://zoom.us/j/644963077" style="background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial;box-sizing:border-box;text-decoration-line:none;font-family:Lato,Helvetica,Arial;font-size:14px" target="_blank"><font color="#0000ff"><b>https://zoom.us/j/644963077</b></font></a></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> </font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;text-align:justify;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif"><font color="#000000"><font style="vertical-align:inherit"><font style="vertical-align:inherit"><b>Who: </b> </font></font></font>Bradly Stadie, Vector Institute</font></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;text-align:justify;line-height:normal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><font face="arial, sans-serif" color="#000000"> </font></p></div><div><font face="arial, sans-serif" color="#000000"><br></font></div><div><div><div><font face="arial, sans-serif"><b>Title: </b>Weakly Supervised Reinforcement Learning </font></div><div><font face="arial, sans-serif"><br></font></div><div><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><font face="arial, sans-serif"><b>Abstract: </b>Consider a scenario wherein we are given the entire ImageNet training set without access to labels. What unsupervised features should we learn from this unlabeled data? In particular, which features should we learn that might be useful for downstream transfer to classification at test time? This problem, known as weakly supervised learning, has seen tremendous advancement in the past 18 months thanks to methods like Contrastive Predictive Coding. </font></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px;min-height:14px"><font face="arial, sans-serif"><br></font></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><font face="arial, sans-serif">In this talk, we will consider the analogous problem in a reinforcement learning (RL) setting. In the absence of any guidance, which tasks should an agent pursue during an unsupervised play phase? Can an agent learn unsupervised behaviors that are useful for downstream transfer to real tasks at test time? While the de facto method for solving this class of problems has long been curiosity methods, we argue that naively applying this approach leads to agents that don’t learn semantically meaningful behaviors. In other words, curiosity doesn’t provide the right inductive bias for downstream transfer. </font></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><font face="arial, sans-serif"><br></font></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:normal;margin:0px"><font face="arial, sans-serif">We propose two methods to fix curiosity's shortcomings. First, we leverage the notion of self-imitation learning to derive an analogue to curiosity in an abstract imitation space. This form of curiosity encourages agents to learn diverse behaviors with semantic differences. Second, we introduce the notion of goals to our agents. Even when we never tell agents which goals are interesting, we see that simply introducing the existence of goals provides enough inductive bias for agents to learn a meaningful distribution of interesting unsupervised behaviors. These behaviors can be readily leveraged to achieve a variety of difficult tasks at test time, including extremely challenging robotic navigation, manipulation, and locomotion tasks. </font></p></div></div><div><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div></div><div style="color:rgb(80,0,80)"><br></div><div style="font-size:small;color:rgb(80,0,80)"><b style="color:rgb(0,0,0);font-family:arial,sans-serif">Host:</b><span style="color:rgb(0,0,0);font-family:arial,sans-serif"> </span><a href="mailto:mwalter@ttic.com" style="font-family:arial,sans-serif" target="_blank">Matthew Walter</a> <br></div><div style="font-size:small;color:rgb(80,0,80)"> <br></div><div style="font-size:small;color:rgb(80,0,80)"><br></div><div style="font-size:small;color:rgb(80,0,80)">**********************************************************************************************************</div><div><font color="#500050" style="font-size:small"><b><i>Matthew Walter</i></b> is inviting you to a scheduled Zoom meeting.</font><br><br><font color="#500050" style="font-size:small">Topic: Talks at TTIC: Bradly Stadie, Vector Institute</font><br><font color="#500050" style="font-size:small">Time: Apr 2, 2020 11:00 AM Central Time (US and Canada)</font><br><br><b><font size="4" style="background-color:rgb(255,255,0)"><font color="#500050">Join Zoom Meeting</font><br><font color="#0000ff"><a href="https://zoom.us/j/644963077" target="_blank">https://zoom.us/j/644963077</a></font></font></b><br><br><font color="#500050" style="font-size:small">Meeting ID: 644 963 077</font><br><br><font color="#500050" style="font-size:small">One tap mobile</font><br><font color="#500050" style="font-size:small">+14086380968,,644963077# US (San Jose)</font><br><font color="#500050" style="font-size:small">+16468769923,,644963077# US (New York)</font><br><br><font color="#500050" style="font-size:small">Dial by your location</font><br><font color="#500050" style="font-size:small"> +1 408 638 0968 US (San Jose)</font><br><font color="#500050" style="font-size:small"> +1 646 876 9923 US (New York)</font><br><font color="#500050" style="font-size:small"> +1 669 900 6833 US (San Jose)</font><br><font color="#500050" style="font-size:small"> +1 253 215 8782 US</font><br><font color="#500050" style="font-size:small"> +1 301 715 8592 US</font><br><font color="#500050" style="font-size:small"> +1 312 626 6799 US (Chicago)</font><br><font color="#500050" style="font-size:small"> +1 346 248 7799 US (Houston)</font><br><font color="#500050" style="font-size:small">Meeting ID: 644 963 077</font></div><div style="font-size:small;color:rgb(80,0,80)"><br>Find your local number: <a href="https://zoom.us/u/akrIgOS2N" target="_blank">https://zoom.us/u/akrIgOS2N</a><br></div><div style="font-size:small;color:rgb(80,0,80)"><br></div><div style="font-size:small;color:rgb(80,0,80)"><br></div></div></div></div></div></div></div></div><div style="font-size:small"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><br></div></div></div></div></div></div></div></div></div></div><div><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><font face="arial, helvetica, sans-serif">Mary C. Marre</font><div><font face="arial, helvetica, sans-serif">Faculty Administrative Support</font></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6"><b>Toyota Technological Institute</b></font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">6045 S. Kenwood Avenue</font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Room 517</font></i></div><div><i><font face="arial, helvetica, sans-serif" color="#3d85c6">Chicago, IL 60637</font></i></div><div><i><font face="arial, helvetica, sans-serif">p:(773) 834-1757</font></i></div><div><i><font face="arial, helvetica, sans-serif">f: (773) 357-6970</font></i></div><div><b><i><a href="mailto:mmarre@ttic.edu" target="_blank"><font face="arial, helvetica, sans-serif">mmarre@ttic.edu</font></a></i></b></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div>
</blockquote></div></div>