I Built An AI Slop Detector

I Built An AI Slop Detector

by Matt Wolfe · View the channel on PSI

Applied Sciences & Engineering AI — Models & Research UYQArtificial intelligence (AI)UYQMMachine learning
🎙 Matt Wolfe 👥 1.0M 📅 September 9, 2026 ⏱ 20 min 👁 570 📄 documentary 🧭 2026-09-09
Available in: English (current) Français

Keywords

AI slopvideo detectionGeminiAPIAGI

Summary

Matt Wolfe tackles the growing problem of AI-generated video content by attempting to build an AI slop detector—a website where users can paste a video URL to determine if it is AI-generated. Using ChatGPT and its Codeex environment, he starts with a straightforward plan but quickly encounters difficulties. The first prototype fails to detect an obviously AI-generated clip (Fart Zuka). After numerous iterations and eight hours of autonomous work, the AI concludes that integrating a third-party API, Site Engine, is necessary. With Site Engine’s computer vision and Gemini’s video understanding, the detector begins to work but often returns ‘inconclusive’ when the two sources disagree. Despite refinements, the creator finds that the API costs are prohibitive for public deployment, consuming over 12,000 operations in testing. He ultimately shares the code on GitHub, where users can run it locally with their own API keys. The experiment also sparks reflections on AGI claims: if advanced models like Gemini cannot reliably detect AI videos, the ‘AGI era’ narrative is questionable. The video concludes with the creator’s frustration and a broader commentary on the cat-and-mouse game between AI generation and detection.

186 words

Critical Evaluation

Value of the Information & Strength of the Argument

The value of the information lies in its real-world demonstration of the current limitations of AI video detection. The creator provides a detailed, step-by-step account of the building process, including failures and workarounds, which is instructive for anyone interested in practical AI applications. The argumentation is anecdotal but honest: the creator does not overstate the success and openly discusses the costs and reliability issues. The critique of AGI claims is thought-provoking, comparing human abilities to detect AI slop with current AI models’ inability to do so, offering a compelling counterpoint to industry hype. However, the reasoning is based on a single, narrow experiment and lacks systematic data, weakening its generalizability.

Scientific Rigor, Source Quality, Title Accuracy

The scientific rigor is limited; the video is a personal project blog, not a controlled study. The creator’s sources include his own testing and the Site Engine API, but he does not cite external research on AI detection. The GitHub repository is provided, allowing others to verify the code, which adds some transparency. The title accurately reflects the content. The use of Gemini and Site Engine is documented, but the evaluation metrics (e.g., number of operations) are not clearly defined. Overall, the methodology is informal, and conclusions should be treated as preliminary.

217 words

Title / Content Match

The title accurately reflects the content: the creator attempts to build an AI slop detector and documents the process, successes, and limitations.

Quality & Reliability

6/10

The video is a personal experiment rather than a scientific study, relying on anecdotal evidence and informal testing. It provides transparent insights into the challenges of AI detection, but the methodology is not rigorous and conclusions are based on limited samples. The reliability is moderate, as the creator does not provide quantitative data or peer-reviewed sources.

Chapters

Cited Sources

Contribution & Novelties

The video contributes a hands-on case study of building an AI detection tool, highlighting the practical difficulties and the reliance on third-party services. It challenges the AGI narrative by pointing out that even advanced models like Gemini fail at a task that humans find easy. The concept of using a hybrid approach (computer vision + LLM) is illustrated, though the integration reveals discord.

Pour aller plus loin :

  • SynthID — Google’s watermarking tool for AI-generated content, relevant to detection methods.
  • AI Content Detection — Overview of techniques used to identify synthetic media.
  • Gemini (language model) — The model used in the video for video understanding, highlighting its limitations in detection.

110 words

Radar Profile

The radar profile shows moderate scores across the board, with a slight dip in reliability due to the anecdotal nature of the project. The information quantity and technical level are higher, reflecting the detailed build process. Overall, the video provides a balanced yet subjective look at AI detection.

Reliability 5/10