Your Infrastructure Just Got Smarter: AI Agents in the DevOps Loop

Kishan Rao, an Engineering Manager at Okta, presents a talk on integrating AI agents into DevOps workflows. He begins by highlighting the complexity of modern infrastructure and the problems of manual processes, tribal knowledge, and stale documentation. He proposes using LLM-based agents to overcome this ‘complexity chasm’ by providing system context, natural language queries, and automated documentation updates. He outlines three key pillars for AI integration: utility, security, and knowledge codification. He then describes an operational model for agents based on sensing, thinking, and acting, with corresponding metrics and MCP tie-ins. Rao introduces a maturity ladder for adoption: observer (read-only agents), adviser (human-in-the-loop suggestions), and actor (autonomous actions in low-risk environments). He discusses guardrails and failure modes, emphasizing hallucination risks, authorization boundaries, and the importance of human approval for critical workloads. He also covers local vs. cloud LLMs, recommending starting with local models in sandboxes and moving to cloud with proper VPC and data residency considerations. The talk concludes with the business case: faster onboarding, reduced MTTR, and freeing engineers from toil. In the Q&A, he mentions using AWS Bedrock, Kendra for RAG, and tools like K8sGPT and Rootly.