Keynote: The dangers of probably-working software - Damian Brady - NDC London 2026

Keynote: The dangers of probably-working software - Damian Brady - NDC London 2026

by Damian Brady

Applied Sciences & Engineering Computing & Cybersecurity UMZSoftware EngineeringUMZTSoftware testing and verification
🎙 Damian Brady 👥 227K 📅 February 5, 2026 ⏱ 54 min 👁 19K 📄 expert opinion 🧭 2026-08-13
Available in: English (current) Français

Keywords

probably-working softwaregenerative AIcode understandingdependenciestrust

Summary

In this keynote at NDC London 2026, Damian Brady explores the concept of ‘probably-working software’—code that appears to function correctly but is not fully understood by its developers. He begins with a personal anecdote about implementing Huffman coding from Wikipedia without fully grasping the algorithm, leading to a production bug. This illustrates the broader issue: developers often rely on code they don’t completely understand, whether it’s their own, from dependencies, or generated by AI. Brady then demonstrates that even simple programs depend on hundreds of external libraries and underlying layers (e.g., .NET’s IL, operating systems), making it impossible to fully understand every layer. He argues that building software is an exercise in trust, and we must trust the ecosystem. However, with the rise of generative AI, the risk increases: AI-generated code is even more opaque, and blindly deploying it can lead to serious consequences. He shares an example of an AI agent that bricked a machine due to unintended actions. Brady concludes by advocating for a balanced approach: using AI for low-risk applications (like a wallpaper app) while maintaining rigorous testing and understanding for critical systems. He emphasizes the need for appropriate trust and verification, not blind acceptance.

198 words

Critical Evaluation

Value of the Information & Strength of the Argument

The talk provides valuable insights into the challenges of software reliability in the age of AI. Brady’s argument is well-structured, using personal stories and concrete examples to illustrate the dangers of ‘probably-working software.’ He effectively builds a case that understanding code is crucial, but acknowledges that complete understanding is impossible due to the complexity of modern software stacks. The argument is persuasive, though it relies heavily on anecdotal evidence rather than systematic research. He offers practical advice: trust the ecosystem but verify critical components, and use AI judiciously based on risk.

Scientific Rigor, Source Quality, Title Accuracy

Brady demonstrates scientific rigor by referencing real incidents (e.g., a .NET compiler bug from 2015, the CrowdStrike outage) and well-known concepts (Huffman coding, dependency trees). He recommends ‘Code: The Hidden Language of Computer Hardware and Software’ by Charles Petzold, a reputable source. The title accurately reflects the content. The talk is well-researched and the speaker is credible, though it is not a formal study. The audience’s engagement suggests the content resonated, but no comments were provided for analysis.

184 words

Title / Content Match

The title accurately reflects the central theme: the risks of relying on software that appears to work but is not fully understood, especially in the context of AI-generated code.

Quality & Reliability

8/10

The talk is based on the speaker's extensive experience in software development and developer relations. It references real incidents (e.g., .NET compiler bug, CrowdStrike) and concepts (Huffman coding, dependency trees) that are verifiable. The speaker is credible and the content is well-structured, though it is primarily anecdotal and opinion-based rather than presenting new empirical research.

Key Moments

Markers derived by PSI from the transcript: the creator did not define chapters.

Cited Sources

  • NDC Conferences — Mentioned as the organizer of the conference.
  • NDC London — Mentioned as the specific conference event.

Concurring Sources

Contribution & Novelties

The talk offers a fresh perspective on the risks of AI-generated code by framing it within the broader context of software trust and understanding. It highlights that ‘probably-working software’ is not a new problem but is exacerbated by AI’s opacity. The speaker provides practical guidance on when to trust AI-generated code based on risk assessment.

Pour aller plus loin :

102 words

Radar Profile

The radar profile shows high scores in quality of information and reliability, reflecting the speaker's expertise and credible examples. The quantity of information is moderate, as the talk focuses on a few key points. The technical level is moderate, accessible to a broad developer audience. Overall, the talk is well-balanced, with strengths in credibility and relevance.

Reliability 8/10