NVIDIA's 4B Model That Re-Runs Its Own Layers

NVIDIA's 4B Model That Re-Runs Its Own Layers

Title history
  • NVIDIA's 4B Model That Re-Runs Its Own Layers — observed on 20261110-1537 UTC (YouTube)
🎙 Kai 👥 72K 📅 November 10, 2026 ⏱ 31 min 👁 46 📄 expert opinion 🧭 2026-11-10
Available in: English (current) Français

Keywords

loop transformertest-time computeKV cacheSWE-benchlatent recurrence

Summary

The video explains NVIDIA’s Dory 1.0-4B model, a 4B-parameter transformer that re-runs its middle 20 layers up to four times per token. Dory is a looped (latent recurrent) transformer, splitting 36 layers into 8 input, 20 recurrent, and 8 output layers. The loop count is user-controlled, allowing scaling of reasoning depth without extra weights. On SWE-bench Verified, Dory scores 50.3% at one loop, 59.5% at two, and 61.8% at four, beating Gemma 4 26B (61.0%). The model was trained on 4.5T tokens in NVFP4, then an additional 1.5T with random loop counts and a difficulty-aware curriculum; reinforcement learning taught it to use deeper loops for harder tasks. However, the performance gains are limited to reasoning tasks; factual recall (AA-Omniscience) remains flat at ~10 regardless of loops, while Gemma scores 21.1. The ‘beats 26B’ headline relies heavily on a non-hallucination benchmark where Dory excels. Running locally, four loops read ~6.9 GB per token from a 3.67 GB file, causing 2.3-2.6x slowdowns on RTX 4090, RX 5700 XT, and MacBook Air M5. The KV cache grows per loop, reaching 26.1 GB at 256K context and four loops; a ’last_loop’ flag shares the cache, reducing memory drastically. NVIDIA’s recommended GGUF defaults are problematic: num_loops=4, context=4096, and -ngl 36, leading to poor performance and memory issues. For typical users, two loops with last_loop and all layers on GPU is optimal. Four loops pay off for agentic tasks (GDPval improves from 815 to 967 Elo) but not for quick batched jobs. The Q5 quantized file shows diminishing returns beyond two loops. Software improvements like wavefront decoding can reduce the speed penalty significantly.

267 words

Critical Evaluation

Value of the Information & Strength of the Argument

The video provides substantial value by dissecting the architecture, training procedure, benchmark nuances, and practical trade-offs of Dory. It goes beyond marketing claims, showing exactly where the model excels and fails, supported by multiple independent community measurements and NVIDIA’s own data. The argumentation is coherent and evidence-based, clearly distinguishing between observed metrics and extrapolations. The author also offers concrete guidance for running the model on consumer hardware, which is highly actionable.

Scientific Rigor, Source Quality, Title Accuracy

Scientific rigor is high: the video cites the official model card, the arXiv paper, GitHub implementations, and independent benchmark runs from multiple users on different GPUs. It transparently notes when figures are author’s own arithmetic. The title accurately reflects the content. Sources are clearly listed and appropriately contextualized. The analysis identifies the non-hallucination benchmark’s outsized influence on the ‘beats 26B’ claim, showing critical thinking. However, some speed/memory figures are derived from the author’s calculations rather than directly measured, which introduces minor uncertainty.

168 words

Title / Content Match

Title accurately describes the core mechanism: a 4B model that re-runs its own middle layers for increased reasoning depth.

Quality & Reliability

8/10

Demonstrates deep technical understanding, cross-references multiple independent measurements, and clearly separates observed data from interpretation. Minor caveats: some arithmetic is author's own and not independently verified, but overall reliable.

Chapters

Cited Sources

Concurring Sources

Contribution & Novelties

The video’s original contribution is a thorough, critical analysis of Dory’s real-world performance, moving beyond the marketing headline. It synthesizes multiple independent measurements to quantify the speed-memory-accuracy trade-offs, explains the KV cache trap and the last_loop fix, and offers practical configuration advice for consumer GPUs. It also highlights the Q5 quantization effects and the potential of wavefront decoding to mitigate the loop slowdown.

Pour aller plus loin :

126 words

Radar Profile

The radar chart would show high scores in technical depth, information quantity, and reliability, with slightly lower but still strong marks in quality of information due to some extrapolations and a few unverified arithmetic steps. The overall profile indicates a comprehensive and technically rigorous analysis.

Reliability 8/10