The Algorithm That Nearly Killed Me: When Testing Isn't Enough - Andrew Paul - NDC Oslo 2026
Keywords
Summary
145 words
Critical Evaluation
Value of the Information & Strength of the Argument
The talk’s value lies in its real-world, life-threatening example that vividly demonstrates the failure of conventional testing. The speaker’s personal experience adds emotional weight and credibility. His argument that tests validate assumptions rather than challenge them is well-supported, and the concept of edge cases between edge cases is insightful. However, the talk is largely anecdotal; he does not provide systematic evidence or statistics on similar incidents, limiting its scientific rigor. The argumentation is strong in raising awareness but could benefit from more structured evidence for broader claims.
Scientific Rigor, Source Quality, Title Accuracy
The talk does not cite external sources or studies, aside from mentioning the Gilded Rose exercise. The primary source is the speaker’s personal experience, which is valid but not independently verifiable. He does not refer to specific literature on ML testing or medical device regulations. The title accurately captures the core message and is compelling. The video is hosted by NDC Conferences, a reputable developer conference. The lack of concrete references reduces the level of scientific rigor, but the talk is intended as an opinion piece rather than a research presentation. No comments were provided for analysis.
199 words
Title / Content Match
The title accurately reflects the content: a personal story of a failing algorithm and the broader implications for software testing.
Quality & Reliability
7/10
The speaker is a senior software engineering trainer with over 15 years of experience, and the talk is based on a personal, well-documented incident. However, the claims about the pump's failure are anecdotal and not independently verified. The overall message about testing limitations is credible and aligns with industry concerns.
Key Moments
Markers derived by PSI from the transcript: the creator did not define chapters.
- Speaker introduces himself and the personal story.
- Discussion of HBA1C and diabetes control measures.
- Transition to insulin pump and the training process.
- The algorithm failure and dangerous blood sugar spikes.
- Analysis of edge cases and the intersection of rare events.
- Applying lessons to broader software testing and AI.
Cited Sources
- NDC Conferences — Conference website for NDC events
- NDC Oslo — Event page for NDC Oslo 2026
External References
Contribution & Novelties
The talk provides a compelling, first-hand account of an algorithmic failure in a medical device, illustrating how conventional testing can miss critical safety issues. The speaker’s concept of ’edge cases between edge cases’ offers a fresh perspective on why rare interactions of multiple conditions are overlooked. He emphasizes the need to test behavior rather than implementation and to prioritize impact over metrics in AI/ML systems. This personal narrative makes the abstract risks of insufficient testing tangible, serving as a wake-up call for the industry.
Pour aller plus loin :
- Insulin pump — Overview of the device and its risks.
- Diabetic ketoacidosis — The dangerous condition described in the talk.
- Software testing — Foundational methods and limitations.
- Edge case — Definition and relevance to the talk’s central theme.
127 words
Radar Profile
The radar shows high quality of information and moderate technical level, with lower scores on quantity and reliability, reflecting the personal nature of the talk.