[Colloquium] [TTIC Talks] Distinguished Lecture Series: Geoffrey Hinton, Univ of Toronto

[George Papandreou]

Goeff Hinton's talk starts in 5 mins.

On Fri, Sep 26, 2014 at 9:16 AM, Dawn Ellis <dellis at ttic.edu> wrote:

> [image: Geoffrey Hinton]
> Thursday, October 2, 2014 at 3:00 pm
> TTIC
> 6045 S. Kenwood Ave.
> Room #526​
>
> *Geoffrey Hinton*, Distinguished Professor, CS Department, University of
> Toronto, and Distinguished Researcher, Google.
>
> Homepage <http://www.cs.toronto.edu/~hinton>
>
> *Title:* Dark Knowledge
>
> *Abstract:* A simple way to improve classification performance is to
> average the predictions of a large ensemble of different classifiers. This
> is great for winning competitions but requires too much computation at test
> time for practical applications such as speech recognition. In a widely
> ignored paper in 2006, Caruana and his collaborators showed that the
> knowledge in the ensemble could be transferred to a single, efficient model
> by training the single model to mimic the log probabilities of the ensemble
> average. This technique works because most of the knowledge in the learned
> ensemble is in the relative probabilities of extremely improbable wrong
> answers. For example, the ensemble may give a BMW a probability of one in a
> billion of being a garbage truck but this is still far greater (in the log
> domain) than its probability of being a carrot. This "dark knowledge",
> which is practically invisible in the class probabilities, defines a
> similarity metric over the classes that makes it much easier to learn a
> good classifier. I will describe a new variation of this technique called
> "distillation" and will show some surprising examples in which good
> classifiers over all of the classes can be learned from data in which some
> of the classes are entirely absent, provided the targets come from an
> ensemble that has been trained on all of the classes. I will also show how
> this technique can be used to improve a state-of-the-art acoustic model and
> will discuss its application to learning large sets of specialist models
> without overfitting. This is joint work with Oriol Vinyals and Jeff Dean.
>
> *Bio:* Geoffrey Hinton received his BA in experimental psychology from
> Cambridge in 1970 and his PhD in Artificial Intelligence from Edinburgh in
> 1978. He did postdoctoral work at Sussex University and the University of
> California San Diego and spent five years as a faculty member in the
> Computer Science department at Carnegie-Mellon University. He then became a
> fellow of the Canadian Institute for Advanced Research and moved to the
> Department of Computer Science at the University of Toronto. He spent three
> years from 1998 until 2001 setting up the Gatsby Computational Neuroscience
> Unit at University College London and then returned to the University of
> Toronto where he is a University Professor. He is the director of the
> program on "Neural Computation and Adaptive Perception" which is funded by
> the Canadian Institute for Advanced Research.
>
> Geoffrey Hinton is a fellow of the Royal Society, the Royal Society of
> Canada, and the Association for the Advancement of Artificial Intelligence.
> He is an honorary foreign member of the American Academy of Arts and
> Sciences, and a former president of the Cognitive Science Society. He has
> received honorary doctorates from the University of Edinburgh and the
> University of Sussex. He was awarded the first David E. Rumelhart prize
> (2001), the IJCAI award for research excellence (2005), the IEEE Neural
> Network Pioneer award (1998), the ITAC/NSERC award for contributions to
> information technology (1992) the Killam prize for Engineering (2012) and
> the NSERC Herzberg Gold Medal (2010) which is Canada's top award in Science
> and Engineering.
>
> Geoffrey Hinton designs machine learning algorithms. His aim is to
> discover a learning procedure that is efficient at finding complex
> structure in large, high-dimensional datasets and to show that this is how
> the brain learns to see. He was one of the researchers who introduced the
> back-propagation algorithm that has been widely used for practical
> applications. His other contributions to neural network research include
> Boltzmann machines, distributed representations, time-delay neural nets,
> mixtures of experts, variational learning, products of experts and deep
> belief nets. His current main interest is in unsupervised learning
> procedures for multi-layer neural networks with rich sensory input.
>
> ​Host:  George ​
> ​
> ​Papandreou, ​gpapan at ttic.edu
>
>
> ------------------------------
>
> *Dawn Ellis*
> Administrative Coordinator,
> Bookkeeper
> 773-834-1757
> dellis at ttic.edu
>
> TTIC
> 6045 S. Kenwood Ave.
> Chicago, IL. 60637
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.cs.uchicago.edu/pipermail/colloquium/attachments/20141002/bfc5699d/attachment.htm