[Colloquium] Beal talk today - 2:45 at TTI

[Meridel Trimble]

TOYOTA TECHNOLOGICAL INSTITUTE TALK

Matthew J. Beal
University of Toronto
Speaker 's Homepage: http://www.cs.toronto.edu/~beal

Time: Friday, April 16th 2004, 2:45pm
Place: TTI-C (1427 E. 60th St. – 2nd Floor)
Refreshments provided

Title: Variational Bayesian methods for Model Selection 

Abstract:
One of the key problems in machine learning and statistics is learning the
structure of graphical models from data. Bayesians would like to select between
available models based on their posterior probabilities, but for models
containing hidden variables it is often intractable to compute a key required
quantity, the marginal likelihood (that which results from marginalising out the
hidden variables and parameters). Estimating the marginal likelihood presents a
difficult challenge for approximate methods such as asymptotic-data criteria and
sampling techniques.

In this talk I present the Variational Bayesian (VB) algorithm, and describe its
novel application to the problem of estimating the marginal likelihoods of all
available models in a small class. We will see that the VB algorithm provides an
approximation in the form of a lower bound on the marginal likelihood, and
results in an elegant generalisation of the commonly used EM algorithm for
ML/MAP optimisation. In particular we show that the algorithm takes on
intuitively simple forms for so-called conjugate exponential models.

I will present experiments comparing the accuracy of VB to various competitor
approximations, such as the BIC, the Cheeseman-Stutz criterion (CS), and also to
a MCMC method known as Annealed Importance Sampling (AIS). We will see that the
VB estimate performs as well as, if not better than, all the other competitors
and more reliably than AIS for a given computation time. We will also discover
an intriguing connection between CS and VB --- a very recent result that
guarantees the existence of a VB bound that is superior to the CS approximation.
If time allows, we will look at other applications of VB, including one in the
bioinformatics domain: inferring genetic regulatory networks using VB
State-Space Models.

If you have questions, or would like to meet the speaker, please contact Meridel 
at 4-9873 or mtrimble at tti-c.org 
   
For information on future TTI-C talks or events, please go to the TTI-C Events 
page: http://www.tti-c.org/events.shtml