How to Run TurboQuant - "Lossless" Quantization for Local AI TESTED ✅

How to Run TurboQuant - "Lossless" Quantization for Local AI TESTED ✅

by xCreate

Formal & Physical Sciences AI — Models & Research UYQArtificial intelligence (AI)UYQMMachine learning
🎙 xCreate 👥 26K 📅 March 29, 2026 ⏱ 16 min 👁 70K 📄 tutorial 🧭 2026-09-09
Available in: English (current) Français

Keywords

quantizationKV cacheLLMcontext precisionperplexity

Summary

The video tests TurboQuant, a new quantization method for KV cache, claiming near-lossless compression and memory savings for local AI models. The creator walks through the technique, which uses a two-pass approach involving Johnson-Lindenstrauss transform and MSE-optimal quantizers, and mentions controversy over credit with RabbitQ. They try multiple implementations in MLX-LM and VLM, including a CPU-based experimental version and a faster shader-based one. The main test involves generating a 1000-token code with Minimax and Llama 1B, comparing various bit rates (9-bit, 4-bit, 3-bit) on memory usage, visual quality of a 3D scene, token accuracy, and perplexity. Results show that 4-bit TurboQuant performs similarly to standard 4-bit affine quantization, but claimed lossless is not achieved: top-token accuracy drops to 98.8% and perplexity rises. The two-pass version is actually worse due to giving up a bit for the JL transform. Mixed-precision quantization helps a bit at 3-bit, but 2-bit fails entirely. The creator provides practical tips and references open-source implementations.

158 words

Critical Evaluation

Value of the Information & Strength of the Argument

The video provides a hands-on evaluation that is more credible than typical hype due to its systematic testing and transparent reporting. The creator acknowledges the limits of the technique and shares raw data like token accuracy and perplexity scores. However, the argumentation sometimes relies on anecdotal visual comparisons and a single model (Llama 1B) which limits generalizability. The distinction between first and second pass is clearly explained, and the measured divergence from official claims is well-documented. The value lies in practical guidance for practitioners considering TurboQuant.

Scientific Rigor, Source Quality, Title Accuracy

The creator cites the TurboQuant paper and open-source implementations, though only the paper is mentioned without specific URL. The description includes affiliate links and companion videos, which are not scientific references but help contextualize. The title accurately reflects the content, and the video delivers an honest test. No peer-reviewed sources are directly referenced; the evaluation is based on user experimentation. The creator does not misrepresent the data and clearly notes discrepancies with published results, increasing epistemic trustworthiness.

178 words

Title / Content Match

The title correctly advertises an evaluation of TurboQuant, and the video indeed tests lossless claims, debunking them partially.

Quality & Reliability

6/10

The test is practical and systematic, but limited to a few models and configurations, and some results conflict with official claims. Author openly shares methodology and limitations.

Key Moments

Markers derived by PSI from the transcript: the creator did not define chapters.

Cited Sources

Concurring Sources

Dissenting Sources

  • TurboQuant paper (claimed lossless results) — The paper reports 99.7% accuracy with 2-bit quantization, but the video's tests on Llama 1B show much lower accuracy and high perplexity, indicating a gap between official results and practical reproduction.

Contribution & Novelties

This video adds practical first-hand testing of TurboQuant, revealing that the ’lossless’ claim does not hold in real-world scenarios, at least on tested models. It provides concrete memory savings and accuracy metrics, contributing knowledge beyond the paper’s theoretical results.

Pour aller plus loin :

  • Johnson-Lindenstrauss lemma — Underlying dimensionality reduction technique used in TurboQuant’s second pass.
  • Quantization (signal processing) — General concept of quantization, relevant to understanding KV cache compression.
  • KV Cache quantization — Blog on KV cache quantization with PyTorch, providing context on existing methods.
  • MLX LM — Open-source repository for running LLMs on Apple silicon, used for implementations tested.

101 words

Radar Profile

The radar profile shows high scores in quantitative information and technical level, but lower for source reliability and overall quality, reflecting a practical yet partially speculative evaluation. The video is dense in benchmarks but lacks external validation and peer-reviewed backing.

Reliability 5/10

💬 Sur les 30 commentaires analysés, l'orientation globale est positive, avec des éloges pour la méthodologie et les tests, mais quelques interrogations sur les différences avec le papier et des demandes de clarification sur l'implémentation.