[Colloquium] Distinguished Lecture Series: Geoffrey Hinton, Univ of Toronto

[Dawn Ellis]

[image: Geoffrey Hinton]
Thursday, October 2, 2014 at 3:00 pm
TTIC
6045 S. Kenwood Ave.
Room #526​

*Geoffrey Hinton*, Distinguished Professor, CS Department, University of
Toronto, and Distinguished Researcher, Google.

Homepage <http://www.cs.toronto.edu/~hinton>

*Title:* Dark Knowledge

*Abstract:* A simple way to improve classification performance is to
average the predictions of a large ensemble of different classifiers. This
is great for winning competitions but requires too much computation at test
time for practical applications such as speech recognition. In a widely
ignored paper in 2006, Caruana and his collaborators showed that the
knowledge in the ensemble could be transferred to a single, efficient model
by training the single model to mimic the log probabilities of the ensemble
average. This technique works because most of the knowledge in the learned
ensemble is in the relative probabilities of extremely improbable wrong
answers. For example, the ensemble may give a BMW a probability of one in a
billion of being a garbage truck but this is still far greater (in the log
domain) than its probability of being a carrot. This "dark knowledge",
which is practically invisible in the class probabilities, defines a
similarity metric over the classes that makes it much easier to learn a
good classifier. I will describe a new variation of this technique called
"distillation" and will show some surprising examples in which good
classifiers over all of the classes can be learned from data in which some
of the classes are entirely absent, provided the targets come from an
ensemble that has been trained on all of the classes. I will also show how
this technique can be used to improve a state-of-the-art acoustic model and
will discuss its application to learning large sets of specialist models
without overfitting. This is joint work with Oriol Vinyals and Jeff Dean.

*Bio:* Geoffrey Hinton received his BA in experimental psychology from
Cambridge in 1970 and his PhD in Artificial Intelligence from Edinburgh in
1978. He did postdoctoral work at Sussex University and the University of
California San Diego and spent five years as a faculty member in the
Computer Science department at Carnegie-Mellon University. He then became a
fellow of the Canadian Institute for Advanced Research and moved to the
Department of Computer Science at the University of Toronto. He spent three
years from 1998 until 2001 setting up the Gatsby Computational Neuroscience
Unit at University College London and then returned to the University of
Toronto where he is a University Professor. He is the director of the
program on "Neural Computation and Adaptive Perception" which is funded by
the Canadian Institute for Advanced Research.

Geoffrey Hinton is a fellow of the Royal Society, the Royal Society of
Canada, and the Association for the Advancement of Artificial Intelligence.
He is an honorary foreign member of the American Academy of Arts and
Sciences, and a former president of the Cognitive Science Society. He has
received honorary doctorates from the University of Edinburgh and the
University of Sussex. He was awarded the first David E. Rumelhart prize
(2001), the IJCAI award for research excellence (2005), the IEEE Neural
Network Pioneer award (1998), the ITAC/NSERC award for contributions to
information technology (1992) the Killam prize for Engineering (2012) and
the NSERC Herzberg Gold Medal (2010) which is Canada's top award in Science
and Engineering.

Geoffrey Hinton designs machine learning algorithms. His aim is to discover
a learning procedure that is efficient at finding complex structure in
large, high-dimensional datasets and to show that this is how the brain
learns to see. He was one of the researchers who introduced the
back-propagation algorithm that has been widely used for practical
applications. His other contributions to neural network research include
Boltzmann machines, distributed representations, time-delay neural nets,
mixtures of experts, variational learning, products of experts and deep
belief nets. His current main interest is in unsupervised learning
procedures for multi-layer neural networks with rich sensory input.

​Host:  George ​
​
​Papandreou, ​gpapan at ttic.edu


------------------------------

*Dawn Ellis*
Administrative Coordinator,
Bookkeeper
773-834-1757
dellis at ttic.edu

TTIC
6045 S. Kenwood Ave.
Chicago, IL. 60637
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.cs.uchicago.edu/pipermail/colloquium/attachments/20140926/3dd05909/attachment-0001.htm