Agent evaluation must test the complete system: model, instructions, context, tools, permissions and failure handling. This template organizes evidence before autonomy expands.

Identification

Record name, version, owner, outcome, users, environments, model, instructions, tools, data sources and date. A relevant change should trigger a new evaluation run.

Test cases

Include case ID, type, input, expected outcome, expected tool, prohibited actions, approval requirement and failure severity. Maintain normal, ambiguous, incomplete, adversarial and unavailable-tool cases.

Metrics

Measure correct completion, tool and parameter accuracy, valid evidence, permission compliance, steps, cost, latency, escalation and error recovery.

Decision

Set thresholds before running the evaluation. Classify failures by impact and record correction, owner and retest. Outcomes may be block, experiment, suggestion mode, approval-gated launch or bounded execution.

The download uses stable columns and an explicitly marked example row. Adapt thresholds to the use case and connect the decision to the AI governance guide.

[ PRACTICAL RESOURCES ]

Use these resources

CSVDownload the evaluation template
[ CONTINUE EXPLORING ]

AI Agents

Understand, build, evaluate and operate AI agents with clear goals and boundaries.

PILLAR GUIDEAI Agents: what they are, how they work and when to use themLearn how AI agents use tools, state, guardrails and evaluations, and decide when agentic systems make sense for products and operations.RELATED GUIDEHow to build AI agents: from outcome to operationsBuild AI agents with a practical method for outcomes, tools, context, safety, evaluations, staged launch and monitoring.RELATED GUIDEAI agents with n8n: architecture and production checklistBuild AI agents with n8n using triggers, tools, optional memory, human approval, evaluations and explicit error handling.RELATED GUIDEAI agent examples: use cases, metrics and risksExplore AI agent examples for product, support, operations and engineering, including tools, outcomes, metrics and risks.

Primary sources