Planning Agent
What is a Planning Agent?
Plan work and delegate work to other agents
- Create plan: multi-step TOON DAG instead of JSON
- Use WebSockets so you get a stream of what the delegated agents are doing as well
- Separate planning from execution
- Planner can look at all skill summaries, history
- Executor only gets relevant tools and skills
- DAG property for partial recovery
TOON Plan
- 47% savings
- Header row declares field names once
- Each task is a CSV row
- Pipe separated multi-tools: q | explain
Directed Acyclic Graph (DAG)
- Use DAGs for plans because some tasks need others first whereas some are independent
- Use Kahn’s algorithm for DAG execution
Dynamic Iteration Estimation
- Allow the planner to estimate the number of iterations it will take to complete a task
- If we hardcode the iterations, we might only get halfway through planning and hallucinate the rest
Execution Agents
What is an Execution Agent?
- The agent that actually does the work
- All inherit from the base agent but have different history windows, temperatures, max iterations, and loaded tools/skills
- Can be specialized by domain
- Specialization beats one big agent because of smaller context
Optimization & Caching
Three-Layer Caching
- Prompt cache
- Response cache
- Query result cache
- Token bucket
Evaluation & Observability
What is LLM Evaluation?
- Evaluating is measuring quality at scale
- There is a range of metrics