Zahra Monfared: Dynamical Systems Theory for Deep Learning

Value of the Information & Strength of the Argument

The talk provides a compelling and rigorous argument that dynamical systems theory offers deep insights into deep learning. The value lies in unifying several architectures under a single mathematical framework and linking training challenges to well-known dynamical phenomena (chaos, bifurcations). The argumentation is solid: the speaker presents specific theorems with references, and uses concrete examples (Lorentz system, bifurcation diagrams) to illustrate concepts. The proposed GTF method is backed by mathematical proofs and experimental demonstrations. The speaker also addresses practical concerns, such as gradient clipping, explaining its limitations. Overall, the value is high for researchers in ML theory and dynamical systems, offering a fresh perspective and actionable solutions.

Scientific Rigor, Source Quality, Title Accuracy

The talk is grounded in several peer-reviewed publications: [1] Bolager et al. (TMLR 2026), [2] Eisenmann et al. (NeurIPS 2023), [3] Hess et al. (ICML 2023), [4] Mikhaeil et al. (NeurIPS 2022). These are credible sources in top ML venues. The title accurately describes the content, though it is broad; the substance focuses heavily on RNNs, but the speaker notes that the principles extend to other architectures. No public comments or feedback were provided for analysis.

The title accurately reflects the content: the talk explicitly explains how dynamical systems theory applies to deep learning, using RNNs as a case study.

8/10

The talk is based on published peer-reviewed papers (NeurIPS, ICML, TMLR), with rigorous mathematical formulations and proofs. The speaker is a recognized researcher in the field. The content is highly specialized and technically accurate, though some claims are asserted without full derivation in this presentation format.

Markers derived by PSI from the transcript: the creator did not define chapters.

• 00:30Introduction to dynamical systems in deep learning, overview of talk.
• 02:00Definition of continuous and discrete-time dynamical systems, examples.
• 05:00Showing feedforward, RNN, residual networks, and transformers as discrete-time dynamical systems.
• 08:00Training as a dynamical system; question of using dynamical systems theory for deep learning.
• 10:00Introduction to bifurcations in smooth and piecewise-smooth systems, border collision bifurcations.
• 15:00Definition of PLRNs and their piecewise-smooth nature, analytical tractability.
• 20:00Explaining the link between Lyapunov exponents and gradient explosion/vanishing; two theorems.
• 25:00Generalized Teacher Forcing (GTF) technique and its mathematical justification.
• 30:00Application of GTF to stabilize training on chaotic data, demonstration with Lorentz system.
• 35:00Introduction of SHILRN variant and application to EEG signal reconstruction.
• 40:00Brief discussion of Koopman operator theory and future directions in interpretable AI.

• Koopman-informed recurrent neural networks — Referenced as [1] in the description, describing Koopman operator theory applied to RNNs.
• Bifurcations and loss jumps in RNN training — Referenced as [2] in description, NeurIPS 2023 paper by Eisenmann et al.
• Generalized Teacher Forcing for Learning Chaotic Dynamics — Referenced as [3] in description, ICML 2023 paper by Hess et al.
• On the difficulty of learning chaotic dynamics with RNNs — Referenced as [4] in description, NeurIPS 2022 paper by Mikhaeil et al.

• On the difficulty of learning chaotic dynamics with RNNs — Provides supporting evidence for the theoretical claims about chaotic dynamics and gradient explosion.
• Generalized Teacher Forcing for Learning Chaotic Dynamics — Directly supports the GTF method presented in the talk.