[ИАД, осень 2025] Вероятностные тематические модели. Лекция 2

Value of the Information & Strength of the Argument

The lecture provides substantial value by presenting a novel perspective on topic modeling: eliminating the bag-of-words assumption and reducing model parameters to only the topic-word matrix. The argumentation is rigorous, with step-by-step mathematical derivations. The instructor builds on previous knowledge, introduces the online algorithm as a practical solution for large collections, and then generalizes the model to incorporate local contexts. He clearly explains the motivation and the mathematical steps, making the content accessible to advanced students. The use of the ‘main lemma’ to derive the M-step is elegant and demonstrates a deep understanding of the optimization problem. The lecture also discusses practical considerations, such as the choice between offline and online algorithms, and the implementation in BigARTM. Overall, the value is high for an audience with a background in machine learning and probability.

Scientific Rigor, Source Quality, Title Accuracy

The lecture demonstrates scientific rigor through careful mathematical derivations and references to established work (e.g., Hofmann’s 1999 paper on PLSA, and the BigARTM library). However, the video description contains no links to sources, and the instructor does not explicitly cite specific papers during the lecture, aside from mentioning his own work and the BigARTM paper. The title accurately reflects the content, as it is indeed the second lecture on probabilistic topic models. The content is well-structured, progressing from review to new material, and the instructor handles potential pitfalls (like index notation) with care. The lack of external references in the description is a minor weakness, but the lecture’s internal consistency and mathematical correctness compensate.

The title accurately reflects the content: a lecture on probabilistic topic models, specifically the second in a series.

8/10

Lecture by an expert (likely Konstantin Vorontsov) with rigorous mathematical derivations, references to established algorithms (EM, PLSA, BigARTM), and clear explanations. Some technical depth, but no external sources cited in the video itself; the description lacks links.

Markers derived by PSI from the transcript: the creator did not define chapters.

• 00:00Introduction and recap of previous lecture on topic modeling and EM algorithm.
• 05:00Discussion of the offline EM algorithm and its computational complexity.
• 10:00Introduction of the online algorithm with frequent updates of phi and the gamma decay factor.
• 15:00Explanation of the batch processing in BigARTM and its parallelization.
• 20:00Motivation for moving beyond bag-of-words: need for local contexts and fragment-level topic distributions.
• 25:00Derivation of theta as a function of phi by averaging word topic vectors.
• 30:00General formulation: maximizing likelihood with theta as a function of phi, leading to modified EM.
• 35:00Detailed derivation of the modified M-step for phi, including the additional term p'.
• 40:00Discussion of the theorem and the structure of the new algorithm.
• 45:00Potential applications: visualizing topic dynamics within documents, summarization, and topic labeling.

• Hofmann, T. (1999). Probabilistic Latent Semantic Analysis. UAI. — Mentioned as the origin of the basic EM algorithm for topic models.
• Vorontsov, K. (2014). Additive Regularization for Topic Models of Text Collections. — Mentioned as the author's work on regularized topic models.
• Frei, S., & Apishev, M. (2016). Parallel Online Topic Modeling. (BigARTM) — Mentioned as the implementation of the online algorithm in BigARTM.

• Hofmann, T. (1999). Probabilistic Latent Semantic Analysis. UAI. — The basic EM algorithm for PLSA is the foundation of the lecture.
• Blei, D., Ng, A., & Jordan, M. (2003). Latent Dirichlet Allocation. JMLR. — LDA is a well-known extension of PLSA, often compared with it.