Dale Schuurmans: Convex Training Algorithms for Hard Machine Learning Problems

Value of the Information & Strength of the Argument

The talk provides a novel perspective on unsupervised learning by formulating it as a convex optimization problem, which is a significant departure from traditional EM-based approaches. The argumentation is logically structured: starting with the simple case of unsupervised SVMs, then generalizing to structured prediction. The speaker clearly explains the mathematical derivations and the intuition behind the convex relaxation. He also honestly discusses limitations and open questions, such as the lack of NP-hardness proofs. The value lies in offering a new direction for research in unsupervised discriminative training, potentially leading to more reliable and scalable methods.

Scientific Rigor, Source Quality, Title Accuracy

The talk is scientifically rigorous, with clear mathematical formulations and references to prior work (e.g., transductive SVMs by Joachims, Vapnik’s work). However, it does not cite specific papers in the talk itself, but the description provides a link to the workshop page. The title accurately reflects the content, focusing on convex training algorithms for hard problems. The presentation is well-organized and the speaker is an established expert, enhancing credibility. The lack of published peer-reviewed sources for the specific results presented is a limitation, but the talk is intended as a research presentation.

The title accurately reflects the content: the speaker discusses convex training algorithms for hard machine learning problems, focusing on unsupervised discriminative training for HMMs and SVMs.

8/10

The talk is a technical lecture by a recognized researcher (Canada Research Chair) presenting original research with mathematical derivations. It is not peer-reviewed but is grounded in established theory and includes references to prior work. The presentation is clear and rigorous, though some claims are not fully proven (e.g., NP-hardness).

Markers derived by PSI from the transcript: the creator did not define chapters.

• 00:00Introduction by Jason, introducing Dale Schuurmans.
• 02:00Schuurmans outlines the talk: unsupervised discriminative training for HMMs, starting with unsupervised SVMs.
• 10:00Introduces the feature-based representation of HMMs and connection to CRFs.
• 20:00Presents the unsupervised SVM principle: find labeling that maximizes margin, with balance constraint.
• 30:00Discusses the computational challenge and the convex relaxation using semi-definite programming.
• 40:00Shows how the labeling enters the SVM dual as an equivalence matrix, enabling relaxation.
• 50:00Extends the approach to structured prediction models like HMMs, proposing a convex training objective.
• 1:00:00Discusses limitations and future work, including robust SVM training and structure learning.
• 1:10:00Concludes with a summary and acknowledges the work is preliminary.

• Workshop 2006 Plenary Lectures — The talk was part of the 2006 Workshop at Johns Hopkins University, and this link provides information about the seminar series.

• Transductive Support Vector Machines — The talk extends the idea of transductive SVMs, which aim to use unlabeled data in SVM training.