The Agentic Flow I Designed Versus the Actual Flow: And How I Discovered It Using OpenTelemetry

Value of the Information & Strength of the Argument

The talk provides valuable insights by bridging BPM and agent development, offering a practical methodology for analyzing agent behavior. The argumentation is solid, grounded in the speaker’s experience and a concrete example. However, the evidence is limited to a single case study, and the claims about the benefits of process mining for agents are not backed by broader empirical data. The speaker effectively explains complex concepts like conformance checking and drift, making them accessible to a technical audience.

Scientific Rigor, Source Quality, Title Accuracy

The talk demonstrates scientific rigor by referencing established concepts like process mining, XES, and BPM patterns, and by providing a public Git repository for reproducibility. The sources are primarily the speaker’s own work and standard tools, with no external citations. The title accurately reflects the content, and the talk stays on topic. The speaker’s expertise in both BPM and agent development lends credibility to the presentation.

The title accurately reflects the content, which compares the intended agentic flow with the actual flow discovered via process mining.

7/10

The talk is based on the speaker's extensive experience in BPM and agent development, and presents a concrete case study with reproducible code. However, it lacks peer-reviewed sources and relies on anecdotal evidence from a single example.

Markers derived by PSI from the transcript: the creator did not define chapters.

• 00:00Introduction: agents as processes, motivation for process mining.
• 03:00Observability: OpenTelemetry, traces, and the monitoring plane.
• 06:00Agent patterns: react, graph, swarm; workflow patterns in agents.
• 09:00Process mining basics: event logs, XES, and process discovery.
• 12:00Conformance checking: fitness and precision metrics.
• 15:00Case study: trusteeship agent, intended vs. discovered flow.
• 18:00Results: fitness issues, drift, and how to address them.
• 21:00Recommendations: using bpmn.io, testing your own agents.
• 24:00Q&A: OpenTelemetry implementation, origins of drift.

• PM4Py: Process Mining for Python — Mentioned as the process mining tool used in the example.
• XES - eXtensible Event Stream — Mentioned as the preferred format for event logs in process mining.
• AWS AgentCore — Mentioned as the AWS service used to deploy the agents.
• AWS Bedrock — Mentioned as an AWS service for building agents.
• OpenTelemetry — Mentioned as the observability framework used to capture traces.
• bpmn.io — Mentioned as a tool for drawing the intended process flow.
• Git repository for the example — Mentioned as the source code for the example agents and analysis.

• Process Mining: Data Science in Action — A comprehensive book on process mining by Wil van der Aalst, supporting the methodology.
• OpenTelemetry Documentation — Official documentation for OpenTelemetry, which is used for observability in the talk.

• No sources found — No discordant sources were identified in the talk.