ColloquiaTalk by Geoffrey Hinton, University of Toronto - Tuesday, February 19th

[Margery Ishmael]

Joint Seminar sponsored by the Committee on Computational Neuroscience
& Department of Computer Science

Date: Tuesday, February 19th, 2002
Time:	12:00 noon
Place: BSLC - Room # 205 (924 E. 57th Street)

Speaker: Geoffrey Hinton, Department of Computer Science, University of Toronto

Host: Yali Amit (773) 702-2568

Title:	Contrastive Backpropagation: A New Way to Learn Undirected Graphical 
Models

Abstract: The standard backpropagation learning procedure requires that 
each training input be accompanied by a specification of the correct 
output.  To apply backpropagation to the unsupervised learning of internal 
representations of sensory data, it is necessary to find a way of doing 
without the supervision signal.  I shall first describe a way of modelling 
high-dimensional data distributions, such as ensembles of images, by using 
a multi layer network in which the activity of each hidden unit represents 
a "goodness" or "badness" that makes an additive contribution to the log 
probability that the model assigns to the input vector.  Standard maximum 
likelihood methods cannot be used to train models of this kind because they 
require intractable computations.  There is, however, an efficient new 
learning procedure that changes each parameter in proportion to the 
difference between two goodness derivatives.  One derivative is computed by 
using backpropagation through the multi layer network when the input is 
real data.  The other derivative is computed when the input has been 
slightly corrupted to fit better with the network's current model.  The 
learning procedure eliminates the tendency of the model to prefer 
corruptions of the data to the data itself.  I shall demonstrate the 
effectiveness of the new learning procedure on a number of unsupervised 
learning tasks and show that it naturally learns population codes.

Bio: Geoffrey Hinton received his BA in experimental psychology from 
Cambridge in 1970 and his PhD in Artificial Intelligence from Edinburgh in 
1978.  He did postdoctoral work at Sussex University and the University of 
California San Diego and spent five years as a faculty member in the 
Computer Science Department at Carnegie-Mellon University.  He then became 
a fellow of the Canadian Institute for Advanced Research and moved to the 
Department of Computer Science at the University of Toronto.  He spent 
three years from 1998 to 2001 setting up the Gatsby Computational 
Neuroscience Unit at University College, London, and then returned to 
Toronto.  http://www.cs.toronto/edu/~hinton/

Computational Neuroscience Seminar Series: http://cns.bsd.uchicago.edu

*Refreshments will be served at 11:45 a.m.*
Persons with a disability who may need assistance should call Don Churilla 
in advance at 773-702-2978