DeepSeek Just Shrunk the KV Cache 437×

Item: DeepSeek Just Shrunk the KV Cache 437×
Rating: 5
Author: PostSingularity Institute

Title history

DeepSeek Just Shrunk the KV Cache 437× — observed on 20261031-2326 UTC (YouTube)

by Kai · View the channel on PSI

Formal & Physical Sciences AI — Models & Research UYQArtificial intelligence (AI)UYQNNeural networks and fuzzy systems

🎙 Kai 👥 65K 📅 September 20, 2026 ⏱ 15 min 👁 35K 📄 expert opinion 🧭 2026-10-31

Available in: English (current) Français

Keywords

KV cacheDeepSeek V4.1 Flashcompressed sparse attentionH200concurrency

Summary

This video dissects DeepSeek’s claim of shrinking the KV cache 437 times with the release of V4.1-Flash. The creator first reproduces the exact ratio from public config files: V1 uses 389,120 bytes per token, while V4.1-Flash uses 890 bytes, giving a factor of 437.2. However, the comparison is misleading because V1 is a dense 2023 model with a 4,096-token context, while V4.1-Flash is a 2026 MoE with a million-token context. The real architectural innovations are four-fold: only 10% of layers retain a cache (compressed sparse attention), generation layers reuse prefill keys, token pairs are compressed, and 4-bit quantization is applied. A fairer comparison shows V4.1-Flash holds 256× more context in 41% less memory when both caches are full. On a node of eight H200 GPUs, the model serves about 915 concurrent users versus 40 for DeepSeek V3, explaining the ultra-low cached token pricing. The video also notes that this architecture is terrible for solo developers on small hardware, with 23 seconds per token on a 16 GB Mac Mini. The paper’s limitations include selection errors in sparse attention and non-deterministic bounded replay. Finally, the creator corrects misinformation about model retirement: DeepSeek retired V4-Flash and V4-Flash-Vision-XP, not V4-Pro, and reversed a planned cutover after developer backlash. The takeaway is to always examine the denominator behind impressive ratios.

216 words

Critical Evaluation

Value of the Information & Strength of the Argument

The video provides exceptional value by going beyond the headline number. It meticulously reconstructs the 437× figure from source files, demonstrating the reliability of its calculations. The argumentation is persuasive because it offers multiple, fairer comparisons (256× more context with less memory, 4× reduction vs V4-Flash) and connects technical architecture to real-world consequences like concurrency, pricing, and solo development difficulties. The creator also critically evaluates the paper’s limitations, avoiding any overstatement. The reasoning is logical and well-structured, making a complex subject accessible without oversimplifying.

Scientific Rigor, Source Quality, Title Accuracy

The scientific rigor is exemplary: every claim is backed by public resources, and the creator openly shows arithmetic that either verifies or refutes figures. The use of primary sources (Hugging Face configs, arxiv report) and independent third-party analyses (Artificial Analysis) strengthens credibility. The title, though attention-grabbing, is technically accurate but is critiqued within the video for being misleading without context. The video effectively corrects widespread misinterpretations, demonstrating a high level of care for factual accuracy.

174 words

Title / Content Match

The title reflects the paper's claim, but the video argues the 437× figure is misleading without context; the content nonetheless matches the theme of KV cache size reduction.

Quality & Reliability

9/10

The creator independently recomputes the headline ratio from public config files, compares relevant alternatives, and checks the paper's own numbers, demonstrating high accuracy and honesty about the broader context.

Chapters

Cited Sources

DeepSeek V4.1-Flash Technical Report (arxiv) — Primary source for the KV cache sizes and architectural details.
DeepSeek V4.1-Flash Model and config.json — Used to verify the 890 bytes per token and layer structure.
DeepSeek V1 config.json (deepseek-llm-67b-chat) — Reference for V1's dense architecture and cache dimensions.
DeepSeek V3 config.json — Used for comparison of cache sizes and concurrency counts.
Official DeepSeek V4.1-Flash Announcement — Context for the model's release and highlights.
DeepSeek API Pricing — Source for the 0.3 cents per million cached input tokens claim.
Latent Space Analysis of V4.1-Flash — Parameter breakdown and model architecture insights.
Hardware Reality Check for Local Deployment — Data on running the model on a Mac Mini, used to illustrate solo developer issues.
Artificial Analysis: DeepSeek V4.1-Flash — Throughput and verbosity metrics (e.g., 208 tokens/s output, high token burn).

Concurring Sources

arxiv technical report — The paper itself provides both the 437× and the 4× numbers; the video's analysis aligns with the paper's data.
DeepSeek V4.1-Flash config — Confirms the exact layer configuration and cache size used in the calculations.
DeepSeek V1 config — Supports the arithmetic for V1's cache size.

Contribution & Novelties

The video’s original contribution is its rigorous deconstruction of a widely circulated metric. It goes beyond simply repeating the paper’s claims by recomputing the ratio, providing a more relevant benchmark (256× more context with less memory), and linking architectural choices to concrete economic and practical consequences. The correction about model retirement is also valuable for the community.

Pour aller plus loin :

KV cache in transformers — Explains the role of key-value caching in modern transformers.
Mixture of experts — Relevant to the backbone and sparse activation of larger models.
Sparse attention — Covers techniques to reduce attention computation, analogous to compressed sparse attention mentioned in the video.

108 words

Radar Profile

The radar profile is exceptionally well balanced with high scores across all dimensions, indicating a video that is information-dense, technically accurate, and trustworthy. The relatively lower quantite_information score reflects the focused scope, while others are near maximum, highlighting its value as an expert analysis.

Reliability 9/10