I was WRONG about M5 MacBook Pro 🤯 (vs M4 Max & M3 Ultra) for Local AI

Value of the Information & Strength of the Argument

The video provides valuable, concrete benchmark data on a newly released chip, filling a gap in independent performance assessments for local AI use. The argumentation is strengthened by the author’s willingness to revisit his own earlier predictions and by the transparency regarding potential confounds (e.g., prompt caching). The testing procedure is clearly described, with the model, quantisation, and prompt size specified, and the use of OBS screen recording is acknowledged as a possible influence. The distinction between prompt processing and generation speed is explained clearly, and the author discusses the practical implications for different types of AI workloads. The batching and distributed-computing tests add depth, showing real-world scenarios. However, the argumentation relies solely on the author’s own measurements without cross-validation from other sources, and no statistical analysis is performed, limiting the quantitative reliability.

Scientific Rigor, Source Quality, Title Accuracy

The scientific rigour is moderate: the methodology is reproducible in principle, but no official documentation from Apple or independent academic sources is cited; all data are self-generated. The sources listed in the description are primarily affiliate links or companion videos, none of which provide external validation. The title is accurate and does not exaggerate the findings, and the content matches the description. The author’s candidness about the prompt-caching ‘cheat’ enhances credibility. No attended public reactions are analysed here, but the comments (provided separately) suggest a generally favourable reception, with some requests for clearer visual representation of results.

The title accurately reflects the content, as the author openly retracts a previous assertion about the M5's prompt processing improvements.

8/10

The video is methodologically sound, with transparent benchmarks, voluntary disclosure of a confounding variable (prompt caching), and appropriate acknowledgment of uncertainty in the author's initial predictions. However, it represents a single individual's testing rather than a peer-reviewed study, so the score is slightly reduced.

Markers derived by PSI from the transcript: the creator did not define chapters.

• 00:30Introduction and unboxing of the M5 Pro, with a comparison to the M4 Max.
• 02:00Setup of the test using Qwen 3.5 27B and first generation speed measurements.
• 05:00Batching test with multiple windows to compare throughput.
• 08:00Prompt processing test showing M5 Pro is 2x faster than M4 Max and also beats M3 Ultra.
• 11:00Distributed compute demonstration between M5 Pro and M4 Max, and concluding remarks.

• Local vs Cloud AI (Companion Video) — Referenced as a companion video that discusses trade-offs between local and cloud AI.
• M1 Max Review (Companion Video) — Earlier review of the M1 Max, potentially relevant to the evolution of Apple silicon.
• M3 vs M4 Max (Companion Video) — Direct comparison of previous generation chips, providing context for the current benchmarks.

• Apple M5 chip article (unofficial) — Wikipedia entry for the Apple M5 chip, which corroborates the existence of neural accelerators and expected performance trends.