Systems Thinking for Agentic AI – Building Robust, Reliable, and Explainable Systems for Healthcare

Systems Thinking for Agentic AI – Building Robust, Reliable, and Explainable Systems for Healthcare

🎙 Rajat Mani Thomas 👥 824 📅 November 5, 2025 ⏱ 40 min 👁 883 📄 expert opinion 🧭 2026-08-16
Available in: English (current) Français

Keywords

agentic AIsystems thinkinghealthcareexplainabilityrobustness

Summary

In this talk, Rajat Mani Thomas advocates for integrating systems thinking into the design of agentic AI systems, particularly for healthcare applications. He begins by contrasting a squirrel and a thermostat to illustrate the characteristics of an agent, emphasizing sensing, modeling, deciding, and acting. He then critiques large language models (LLMs) for their static knowledge, contextual fragility, and inability to handle simple tasks like counting letters, attributing these issues to architectural limitations. To address these problems, he proposes decomposing large models into smaller, specialized language models (SLMs) that can be composed with tool use, forming compound systems. However, he warns that simply combining components without considering interactions can lead to emergent misalignment, using the analogy of an ’evil corporation’ composed of well-intentioned individuals. He introduces key systems thinking principles: stocks and flows, feedback loops, delays, boundaries, and leverage points. He then presents a detailed use case: a hypothetical ‘heart failure home care app’ for a 68-year-old patient named Anna. He outlines the design process, including defining the environment, boundaries, sensors, state estimation, actuators, goals, and world model. He emphasizes the importance of modeling delays to avoid oscillations in treatment, and the need for an orchestrator agent to coordinate specialized agents. The talk concludes by highlighting the value of explicit causal loop diagrams for identifying leverage points and break points, and the importance of involving stakeholders from the outset.

228 words

Critical Evaluation

Value of the Information & Strength of the Argument

The talk provides a compelling argument for applying systems thinking to agentic AI, moving beyond a purely technical focus to consider socio-technical dynamics. The speaker effectively uses analogies (squirrel vs. thermostat, evil corporation) and a concrete use case to illustrate abstract concepts. The argumentation is logical and builds a case for a structured, multi-agent approach with explicit modeling of feedback and delays. However, the talk is largely conceptual and lacks empirical evidence or case studies demonstrating the effectiveness of the proposed framework. The speaker acknowledges this is a ‘sounding board’ for ideas, which limits the strength of the argumentation.

Scientific Rigor, Source Quality, Title Accuracy

The speaker demonstrates scientific rigor by grounding his arguments in known limitations of LLMs and systems thinking principles. However, he does not cite specific sources or studies during the talk, and the only reference provided in the description is a link to the symposium page, which does not contain detailed references. The title accurately reflects the content, and the talk is well-structured. The speaker’s credentials and the context of an academic symposium lend credibility, but the lack of explicit citations and empirical validation reduces the overall rigor.

201 words

Title / Content Match

The title accurately reflects the content, which focuses on applying systems thinking to agentic AI in healthcare.

Quality & Reliability

7/10

The speaker is a domain expert with a strong academic background, and the talk is well-structured with practical examples. However, it is a conference presentation without peer-reviewed sources or empirical validation, and the content is largely conceptual.

Key Moments

Cited Sources

Concurring Sources

Contribution & Novelties

The talk offers a novel perspective by explicitly bridging systems thinking and agentic AI design, providing a structured framework for building robust, explainable, and context-aware systems in healthcare. It emphasizes the importance of modeling delays and feedback loops to prevent unintended consequences, and advocates for a multi-agent architecture with specialized components. The use case of a heart failure home care app serves as a practical template for applying these principles.

Pour aller plus loin :

  • Systems thinking — Foundational concepts of systems thinking, including feedback loops and leverage points.
  • Agentic AI — Overview of intelligent agents and their characteristics.
  • Explainable AI — Key concepts in making AI systems transparent and interpretable.

111 words

Radar Profile

The radar profile shows a balanced distribution across all dimensions, with slightly higher scores in information quantity and quality, reflecting the talk's comprehensive coverage and expert insights. The lower scores in technical depth and reliability indicate that while the content is accessible, it lacks rigorous empirical backing and detailed technical specifics.

Reliability 6/10