Xiaomi & DeepSeek Just Solved the Biggest Bottleneck in Your Local AI

Value of the Information & Strength of the Argument

The video offers significant value by dissecting the technical papers and correcting common misconceptions about KV cache compression. The presenter’s argumentation is solid: he uses specific numbers from the papers, compares the measurements appropriately, and explains the architecture changes clearly. He also provides real-world context, such as API pricing and local inference benchmarks, which makes the analysis actionable. The reasoning is logical and well-supported, with a clear distinction between the advertised memory reduction and the actual compute improvements. However, some simplifications may gloss over complexities, but overall the argument is convincing.

Scientific Rigor, Source Quality, Title Accuracy

The video demonstrates scientific rigor by directly citing the papers and providing links to primary sources. It critically evaluates the claims, distinguishing between different metrics, and acknowledges limitations such as the lack of standard recall benchmarks. The title is largely accurate, though it might overemphasize ’local AI’ given the broader scope. The sources are credible, including arXiv preprints, official model pages, and user reports. The presenter’s approach aligns with good scientific practice, as he neither overstates nor understates the findings. The only minor issue is the somewhat clickbait-style title, but the content substantiates it.

The title accurately reflects the content, though the focus on 'Local AI' is slightly narrower than the video's scope, which also covers API-side implications.

8/10

The video provides a well-researched analysis of two recent papers, correctly identifying the main innovation beyond the headline memory numbers. It cites official sources and transparently discusses limitations, though it occasionally simplifies complex concepts.

• 00:00The 2 AM agent that stopped making sense
• 01:234x vs 4.5x: two different rulers
• 03:17Both papers: memory was never the problem
• 04:46Two bills: re-reading vs reading
• 06:05YOCO: cut the model in half
• 07:23DeepSeek and Xiaomi: one cut, two versions
• 09:14Same month, same people
• 10:37The API bill: a fifth from half a percent
• 11:39On a Mac, reading is the wait
• 12:32The catch: recall is barely tested
• 13:12Ask for the reading speed

• DeepSeek API pricing — Used to illustrate the cost difference between cache hits and misses.
• YOCO paper (You Only Cache Once) — Reference for the YOCO architecture that both papers build upon.
• DeepSeek V4.1 report — Primary source for DeepSeek's claims and architecture.
• HySparse2 paper — Primary source for Xiaomi's HySparse2 design.
• DeepSeek V4.1 Flash model page — Model card and download link.
• Hacker News thread about cache cost cliff — Developer report about hitting the cache cost cliff.
• Reddit thread about 11.59M-token app build — Example of a real-world token usage with high cache hit rate.
• Reddit thread about V4.1 Flash forgetting — User reports of recall issues beyond 400K tokens.
• Reddit thread about M3 Ultra prefill — Measurement of prefill speed on Apple Silicon.
• Related video: What It Actually Costs to Run DeepSeek V4.1 Flash Locally — A companion video on running DeepSeek locally.
• Related video: DeepSeek Just Shrunk the KV Cache 437x — Another related video on KV cache reduction.

• DeepSeek V4.1 report — Supports the claim that prefill compute is the primary focus.
• HySparse2 paper — Aligns with the video's explanation of architecture and prefill reduction.
• YOCO paper — Foundational for the split architecture described.
• Hacker News thread — Real-world evidence of the cache cost cliff, illustrating the same issue.

• Reddit thread about V4.1 Flash forgetting — Reports of recall issues beyond 400K tokens, contrasting with the video's optimistic tone about the architecture's benefits, and highlighting the unresolved memory retention problem.