Apigee AI Observability Agent
A conversational AI agent that lets engineers investigate API gateway errors, latency, and incidents in plain English.
The Problem
Engineers troubleshooting API gateway issues had to manually query a specialized observability platform to find errors, latency bottlenecks, and performance problems — slow, expertise-gated work that required knowing a purpose-built query language.
Why Traditional Systems Failed
Standard observability tooling requires knowing a purpose-built query language and where to look, gatekeeping troubleshooting behind a handful of experts. Generic AI assistants aren't reliable enough to trust with live data retrieval and root-cause claims.
Two-Phase Agent Architecture
A two-phase agent architecture separates understanding from execution. Phase one uses an LLM purely to detect intent and extract parameters from a natural-language question (e.g., "top error proxies in the last hour," "root cause for problem P-231201"). Phase two is a deterministic, non-LLM router that dispatches to one of over 20 specialized tool functions — for errors, latency, traffic, environment health, or automated root-cause analysis — each of which builds and executes a structured query against the observability platform's log store. Results come back as HTML reports or charts in a chat interface, with fuzzy proxy-name matching and natural-language time parsing lowering the barrier to use.
Interactive Architecture
Hover a node for detail · click to isolate its connections
Interface
Intelligence
Retrieval
Platform
The LLM decides what to look for. Deterministic code decides how to look it up.
Key Capabilities
Natural Language Log Queries
Engineers ask about errors, latency, or traffic in plain English instead of writing query-language filters.
Deterministic Tool Dispatch
An LLM only detects intent; over 20 specialized Python tools handle the actual data retrieval.
Automated Root-Cause Reports
Generates root-cause analysis reports for flagged incidents directly from log data.
Fuzzy Proxy & Time Matching
Resolves approximate proxy names and natural-language time ranges (e.g., "last hour") automatically.
Visual & Narrative Output
Returns chart visualizations and narrative summaries, not just raw log rows.
What This Enables
- Self-service gateway troubleshooting for any engineer, not just observability experts
- Faster root-cause diagnosis for API gateway incidents
- Lower barrier to entry for log analysis via natural language
- Safer LLM adoption pattern: intent detection only, never direct data access
What This Is Not
- Not a generic AI chatbot
- Not an autonomous incident-remediation system
- Not a replacement for the underlying observability platform
- Not a system where the LLM directly queries or writes production data
Business Value
Let any engineer self-serve log analysis and root-cause investigation in plain English instead of depending on query-language experts, cutting the time to diagnose gateway incidents.
Key Learnings
- Splitting "LLM for understanding intent" from "deterministic code for execution" avoids the reliability risk of letting a model touch the actual data-retrieval path — the LLM decides what to look for, plain code decides how to look it up.
- Structured, hybrid retrieval (a query-builder against a purpose-built log store) beats vector search for observability data, where exact filters matter more than semantic similarity.
- Automated root-cause reports for known incidents are worth more to users than open-ended Q&A — a few well-built RCA tools got more use than the general query interface.