Cai Zhou: Coevolutionary Continuous Discrete DLMs, Semantic Scale Prediction via Hierarchical DLMs

Value of the Information & Strength of the Argument

The talk provides substantial value by addressing fundamental limitations of current diffusion language models and proposing novel solutions. The argumentation is solid, grounded in theoretical analysis (expressivity, computational complexity) and supported by experimental results. The speaker clearly motivates the need for more expressive hidden states and demonstrates how hierarchical and continuous approaches can bridge the gap. The theoretical contributions, such as proving the expressivity advantages of continuous diffusion over discrete diffusion and loop transformers, are significant. The practical challenges of continuous diffusion are honestly acknowledged, and the proposed CCDD model is a principled attempt to combine the strengths of both paradigms. The argumentation is coherent and well-structured, with clear logical flow from problem identification to solution proposal and validation.

Scientific Rigor, Source Quality, Title Accuracy

The talk demonstrates high scientific rigor, with detailed mathematical formulations and references to prior work (e.g., loop transformers, coconut). The speaker cites their own papers (accepted at NeurIPS 2025) and mentions collaborations with MIT, Microsoft, and other institutions. The sources are credible, though the talk itself is a presentation of original research rather than a review. The title accurately reflects the content, covering both hierarchical and coevolutionary continuous-discrete diffusion models. The presentation is technical and assumes familiarity with diffusion models and language modeling, but it is well-organized and clear. No comments were provided, so no analysis of public reception is possible.

The title accurately reflects the content, covering both hierarchical diffusion language models and coevolutionary continuous-discrete diffusion models.

8/10

The talk presents original research from two papers accepted at NeurIPS 2025, with rigorous theoretical analysis and experimental validation. The speaker is a PhD student at MIT, and the work involves collaborations with Microsoft and other institutions. The presentation is technical and detailed, providing mathematical formulations and empirical results. However, as a seminar talk, it lacks peer-reviewed publication details and full experimental reproducibility, and the claims are based on the speaker's own work, which may have inherent biases.

Markers derived by PSI from the transcript: the creator did not define chapters.

• 00:00Introduction and overview of the talk, contrasting autoregressive and diffusion language models.
• 02:00Recap of previous work on the limitations of any-order generation and the need for generalized diffusion models.
• 05:00Introduction of Hierarchical Diffusion Language Models (HDLM) with cluster tokens as intermediate states.
• 10:00Mathematical formulation of HDLM using CTMC and derivation of the training loss as a weighted sum of cross-entropy losses.
• 15:00Experimental results showing HDLM outperforms MDLM, with optimal cluster size around square root of vocabulary size.
• 20:00Transition to continuous diffusion models and comparison with loop transformers.
• 25:00Theoretical analysis of expressivity: continuous diffusion is strictly more expressive than discrete diffusion and at least as expressive as loop transformers.
• 30:00Discussion of practical challenges of continuous diffusion, including optimization difficulties and decoding complexity.
• 35:00Introduction of Coevolutionary Continuous-Discrete Diffusion (CCDD) model, combining discrete and continuous diffusion.
• 40:00Summary of contributions and potential future directions.

• Semantic Scale Prediction via Hierarchical Diffusion Language Models (NeurIPS 2025) — First paper presented, introducing HDLM.
• Coevolutionary Continuous Discrete Diffusion Models (NeurIPS 2025) — Second paper presented, introducing CCDD.

• Diffusion Language Models — Foundational work on diffusion language models.
• Masked Diffusion Language Models — Baseline method for discrete diffusion.