Attacking AI - Jason Haddix - NDC Security 2026
Keywords
Summary
161 words
Critical Evaluation
Value of the Information & Strength of the Argument
The talk provides high value by sharing real-world case studies and a practical methodology that is often missing in academic discussions. The argumentation is solid, grounded in the speaker’s extensive experience and concrete examples. He effectively demonstrates the vulnerabilities in AI systems and the importance of holistic testing, addressing both technical and procedural aspects. The ‘first try fallacy’ concept is particularly insightful, explaining why AI testing differs from traditional security testing. The case studies are compelling and illustrate the methodology in action, making the argument persuasive.
Scientific Rigor, Source Quality, Title Accuracy
The talk demonstrates scientific rigor through its structured methodology and real-world validation. However, it lacks formal citations to academic sources, relying instead on the speaker’s expertise and anecdotal evidence. The sources cited are limited to conference links, which are not directly related to the content. The title accurately reflects the content, and the talk is well-organized. The speaker’s credibility and the practical nature of the examples enhance the overall reliability, though the lack of external references is a minor weakness.
181 words
Title / Content Match
The title accurately reflects the content, which focuses on attacking AI systems with practical methodology and case studies.
Quality & Reliability
8/10
The talk is based on extensive practical experience from real-world AI security assessments, with concrete case studies and a clear methodology. The speaker is a recognized expert in offensive security. However, claims are not backed by formal citations or peer-reviewed sources, and some details are anecdotal.
Key Moments
Markers derived by PSI from the transcript: the creator did not define chapters.
- Introduction and speaker background
- Discussion of the 'first try fallacy' and non-deterministic nature of LLMs
- Overview of AI system architectures, including RAG and agentic systems
- Introduction of the seven-point assessment methodology
- Case study: Amazon Rufus chatbot bypass using ASCII encoding
- Case study: Healthcare system compromised via malicious file uploads
- Case study: Automotive internal app with RAG and DevOps integration
- Discussion of attacking the ecosystem and supporting web apps
- Resources for learning AI security, including CTFs and taxonomy
- Conclusion and key takeaways
Cited Sources
- NDC Security Conference — Conference where the talk was presented
- NDC Conferences — Organizer of the conference
Concurring Sources
- OWASP Top 10 for LLM Applications — Aligns with the methodology's focus on prompt injection and data leakage
Contribution & Novelties
The talk provides a unique, practitioner-focused methodology for AI security assessments, filling a gap between academic research and real-world testing. It introduces the ‘first try fallacy’ and emphasizes holistic testing of AI ecosystems, including agents and supporting infrastructure. The case studies offer concrete examples of attacks and their impact.
Pour aller plus loin :
- Prompt injection — Overview of the attack technique.
- Retrieval-augmented generation — Explanation of RAG architecture.
- OWASP Top 10 for LLM Applications — Industry standard for LLM security risks.
82 words
Radar Profile
The radar profile shows high scores in quantity and quality of information, with moderate technical depth and high reliability. This indicates a talk that is rich in practical content and credible, but not highly technical in terms of formal theory.