Agent evaluation must test the complete system: model, instructions, context, tools, permissions and failure handling. This template organizes evidence before autonomy expands.

Identification

Record name, version, owner, outcome, users, environments, model, instructions, tools, data sources and date. A relevant change should trigger a new evaluation run.

Test cases

Include case ID, type, input, expected outcome, expected tool, prohibited actions, approval requirement and failure severity. Maintain normal, ambiguous, incomplete, adversarial and unavailable-tool cases.

Metrics

Measure correct completion, tool and parameter accuracy, valid evidence, permission compliance, steps, cost, latency, escalation and error recovery.

Decision

Set thresholds before running the evaluation. Classify failures by impact and record correction, owner and retest. Outcomes may be block, experiment, suggestion mode, approval-gated launch or bounded execution.

The download uses stable columns and an explicitly marked example row. Adapt thresholds to the use case and connect the decision to the AI governance guide.

[ PRACTICAL RESOURCES ]

Use these resources

CSVDownload the evaluation template
[ CONTINUE EXPLORING ]

AI Agents

Understand, build, evaluate and operate AI agents with clear goals and boundaries.

PILLAR GUIDEAI Agents: what they are, how they work and when to use themLearn how AI agents use tools, state, guardrails and evaluations, and decide when agentic systems make sense for products and operations.RELATED GUIDEHow to build AI agents: from outcome to operationsBuild AI agents with a practical method for outcomes, tools, context, safety, evaluations, staged launch and monitoring.RELATED GUIDEAI agents with n8n: architecture and production checklistBuild AI agents with n8n using triggers, tools, optional memory, human approval, evaluations and explicit error handling.RELATED GUIDEAI agent examples: use cases, metrics and risksExplore AI agent examples for product, support, operations and engineering, including tools, outcomes, metrics and risks.RELATED GUIDEAI agent evaluation: test cases, metrics and release gatesLearn how to evaluate AI agents for outcomes, trajectories, tools, safety, cost and readiness before releasing a change.RELATED GUIDEHow to operate AI agents in production: observability, SLOs and incidentsLearn how to operate AI agents with observability, proportionate SLOs, cost controls, incident response and reversible changes.RELATED GUIDEResearch: which controls do official documents recommend for operating AI agents?A reproducible audit of seven official documents covering observability, evaluation, metrics, human control and incidents in AI agents.RELATED GUIDEAI agent incident response: containment, recovery and rollbackLearn how to detect, contain, recover from and learn from AI agent incidents without confusing configuration rollback with reversal of external effects.RELATED GUIDEResearch: which official controls appear in AI agent recovery?A reproducible audit of eight official documents on detection, containment, resumption, fallback and approval in AI agent recovery.

Primary sources