<div dir="ltr">
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;font-size:11pt;font-family:Calibri,sans-serif"><b><span style="font-size:12pt;color:rgb(34,34,34);background:white">Distinguished Lecture Series: Naftali Tishby, Hebrew
University of Jerusalem</span></b><span style="font-size:12pt"><span></span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background:white;font-size:11pt;font-family:Calibri,sans-serif"><b><span style="font-size:10pt;color:rgb(48,48,48)"><span> </span></span></b></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background:white;font-size:11pt;font-family:Calibri,sans-serif"><b><span style="font-size:10pt;color:rgb(48,48,48)">Wednesday, April 18, 2018 at 11:00 am<span></span></span></b></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background:white;font-size:11pt;font-family:Calibri,sans-serif"><b><span style="font-size:10pt;color:rgb(48,48,48)">TTIC</span></b><span style="font-size:10pt;color:rgb(34,34,34)"><span></span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background:white;font-size:11pt;font-family:Calibri,sans-serif"><b><span style="font-size:10pt;color:rgb(48,48,48)">6045 S. Kenwood Avenue</span></b><span style="font-size:10pt;color:rgb(34,34,34)"><span></span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background:white;font-size:11pt;font-family:Calibri,sans-serif"><b><span style="font-size:10pt;color:rgb(48,48,48)">Room #526</span></b><span style="font-size:10pt;color:rgb(34,34,34)"><span></span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background:white;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-size:10pt;color:rgb(48,48,48)">For more details please visit our
website at </span><a href="http://www.ttic.edu/dls" style="color:rgb(5,99,193);text-decoration:underline" target="_blank"><span style="font-size:10pt">www.ttic.edu/dls</span></a><span style="font-size:10pt;color:rgb(48,48,48)"> <span></span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background:white;font-size:11pt;font-family:Calibri,sans-serif"><b><span style="font-size:10pt;color:rgb(48,48,48)"><span> </span></span></b></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background:white;font-size:11pt;font-family:Calibri,sans-serif"><b><span style="font-size:10pt;color:rgb(48,48,48)">Naftali Tishby</span></b><span style="font-size:10pt;color:rgb(34,34,34)"><span></span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background:white;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-size:10pt;color:black">Professor of Computer Science, and the incumbent of the Ruth and
Stan Flinkman Chair for Brain Research at the Edmond and Lily Safra Center for
Brain Science (ELSC)<span></span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:14.6pt;background:white;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-size:10pt;color:black">Hebrew University of Jerusalem<span></span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:14.6pt;background:white;font-size:11pt;font-family:Calibri,sans-serif"><a href="http://naftali-tishby.strikingly.com/" style="color:rgb(5,99,193);text-decoration:underline" target="_blank">http://naftali-tishby.<wbr>strikingly.com/</a><span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:14.6pt;background:white;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-size:10pt;color:rgb(34,34,34)"><span> </span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:14.6pt;background:white;font-size:11pt;font-family:Calibri,sans-serif"><b><u><span style="font-size:10pt;color:rgb(34,34,34)">Title</span></u></b><span style="font-size:10pt;color:rgb(34,34,34)">: Information Theory of Deep Learning<span></span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background:white;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-size:10pt;color:rgb(34,34,34)"> <span></span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background:white;font-size:11pt;font-family:Calibri,sans-serif"><b><u><span style="font-size:10pt;color:rgb(34,34,34)">Abstract</span></u></b><span style="font-size:10pt;color:rgb(34,34,34)">: Abstract: I will present a novel
comprehensive theory of large scale learning with Deep Neural Networks, based
on the correspondence between Deep Learning and the Information Bottleneck
framework.<span> </span>The theory is based on the
following components: (1) rethinking Learning theory. I will prove a new generalization
bound, the input-compression bound, which shows that compression of the input
variable is far more important for generalization than the dimension of the
hypothesis class, an ill defined notion for deep learning. (2) I will than
prove that for large scale Deep Neural Networks the mutual information on the
input and the output variables, for the last hidden layer, provide a complete
characterization of the sample complexity and accuracy of the network. This put
the information Bottlneck bound as the optimal trade-off between sample
complexity and accuracy with ANY learning algorithm. (3) I will then show how
stochastic gradient descent, as used in Deep Learning, actually achieves this
optimal bound. In that sense, Deep Learning is a method for solving the
Information Bottleneck problem for large scale supervised learning
problems.<span> </span>The theory gives concrete
predictions for the structure of the layers of Deep Neural Networks, and design
principles for such Networks, which turns out to depend solely on the joint
distribution of the input and output and the sample size.<span></span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background:white;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-size:10pt;color:rgb(34,34,34)"><span> </span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;background:white;font-size:11pt;font-family:Calibri,sans-serif"><b><u><span style="font-size:10pt;color:rgb(34,34,34)">Bio</span></u></b><b><span style="font-size:10pt;color:rgb(34,34,34)">: </span></b><span style="font-size:10pt;color:rgb(34,34,34)">Dr. Naftali Tishby is a professor of Computer Science, and the
incumbent of the Ruth and Stan Flinkman Chair for Brain Research at the Edmond
and Lily Safra Center for Brain Science (ELSC) at the Hebrew University of
Jerusalem. He is one of the leaders of machine learning research and
computational neuroscience in Israel and his numerous ex-students serve at key
academic and industrial research positions all over the world. Prof. Tishby was
the founding chair of the new computer-engineering program, and a director of
the Leibnitz research center in computer science, at the Hebrew university.
Tishby received his PhD in theoretical physics from the Hebrew university in
1985 and was a research staff member at MIT and Bell Labs from 1985 and 1991.
Prof. Tishby was also a visiting professor at Princeton NECI, University of
Pennsylvania, UCSB, and IBM research. His current research is at the interface
between computer science, statistical physics, and computational neuroscience.
He pioneered various applications of statistical physics and information theory
in computational learning theory. More recently, he has been working on the
foundations of biological information processing and the connections between
dynamics and information. He has introduced with his colleagues new theoretical
frameworks for optimal adaptation and efficient information representation in
biology, such as the Information Bottleneck method and the Minimum Information
principle for neural coding.<span></span></span></p>
</div>