Go to Prod With AI Automated Workflow at Work | Pratik Verma, Okahu.ai
Keywords
Summary
137 words
Critical Evaluation
Value of the Information & Strength of the Argument
The talk provides valuable insights into the practical challenges of deploying AI agents in enterprise environments. The speaker uses concrete examples from his own experience, which adds credibility. The argumentation is coherent, moving from the construction of a simple bot to the identification of failure modes and the solution via observability. However, the presentation is heavily focused on promoting Okahu’s product, which may bias the discussion. The value lies in the detailed walkthrough of using OpenTelemetry-based tracing to debug agentic applications, a topic that is often underexplored.
Scientific Rigor, Source Quality, Title Accuracy
The speaker references the open-source tool Monocle and provides a GitHub repository for the demo code, which is a positive aspect. However, the talk lacks citations to external research or standards beyond OpenTelemetry. The title accurately reflects the content, and the presentation is well-structured. The lack of independent sources and the promotional nature of the talk slightly reduce its scientific rigor.
163 words
Title / Content Match
The title accurately reflects the content, which focuses on taking AI agents to production using observability tools.
Quality & Reliability
7/10
The speaker is the CEO of Okahu.ai, a company specializing in AI observability, and the talk is based on practical experience with Microsoft Teams agents. The content is technically sound and provides concrete examples, but it is primarily a product demonstration with limited external validation.
Key Moments
Markers derived by PSI from the transcript: the creator did not define chapters.
- Introduction and overview of the talk
- Demonstration of a Teams bot generating marketing copy
- Explanation of what AI agents are and how they differ from traditional code
- Walkthrough of the bot code in VS Code using Teams AI SDK and Azure
- Discussion of common failure modes: incorrect dates, missing context, and silent truncation
- Introduction to Monocle and Okahu for observability
- Demo of Okahu portal showing traces and insights
- Analysis of engagement metrics and task completion rates
- Drilling down into LLM inferences and identifying bottlenecks
- Summary of common issues and recommendations for production
Cited Sources
- MLOps World — Conference website where the talk was recorded
Concurring Sources
- OpenTelemetry — The talk uses OpenTelemetry for tracing, which is a widely adopted standard.
Contribution & Novelties
The talk provides a practical, hands-on demonstration of using OpenTelemetry-based observability to debug and improve AI agents in production. It highlights specific failure modes that are often overlooked and shows how to trace issues across the entire stack. The introduction of Monocle as an open-source tool is a valuable contribution.
Pour aller plus loin :
- OpenTelemetry — The standard for observability, used in the talk for tracing.
- Microsoft Teams AI SDK — Official documentation for building Teams bots.
- Azure OpenAI Service — The LLM service used in the demo.
89 words
Radar Profile
The radar profile shows high scores in information quantity, quality, and technical level, but a slightly lower reliability score due to the promotional nature of the talk. This suggests a technically rich but potentially biased presentation.