[Colloquium] TTIC Talk: Shie Mannor, Technion

[Julia MacGlashan]

When:             *Friday, Oct 1 @ 1:00pm
*

Where:           * TTIC Room #401*, 6045 S Kenwood Ave, 4th Floor


Who:              * **Shie Mannor*, Technion


Title:          *      **All Learning Is Robust*****



 Controlling overfitting (i.e., when the decision rules obtained fit the
training samples extremely well, but fail to "generalize" well and perform
poorly on the true distribution) is a long standing topic of study in
machine learning. Regularization is a widely used technique to control
overfitting where a penalty is added to the cost function (typically the
classification or regression error). The success of regularization in a host
of different algorithms is usually interpreted as coming from penalizing the
complexity of the resulting decision rules favoring "simple" rules. In this
talk we propose a different perspective to learning base on robust
optimization. That is, assuming that each sample corrupted by a certain
disturbance, we find the best decision under the most adversarial
disturbance. We show that a special choice of the disturbance exactly
recovers the solution obtained by penalizing complexity via regularization.
Both Support Vector Machines and Lasso can be re-derived from a robust
optimization perspective. The equivalence relationship between
regularization and robustness gives a physical interpretation of the
regularization process. Moreover, it helps us explain from a robustness
point of view why support vector machines are consistent, and why Lasso
produces sparse solutions. Generalizing these results we use the robustness
perspective to derive new algorithms in new domains that have both favorable
statistical and computational properties. We finally argue that robustness
is a necessary and sufficient condition for consistency of learning
algorithms and in fact every useful learning algorithm must possess some
robustness properties.

Host:              Nati Srebro, nati at ttic.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.cs.uchicago.edu/pipermail/colloquium/attachments/20100927/a4d09898/attachment.htm