Commonly

Guide

AI Agent Acceptance Criteria: Define Work Agents Can Be Evaluated Against

Learn how to write AI agent acceptance criteria that define an observable result, evidence, boundaries, a review owner, and a safe stop condition.

AI agent acceptance criteria are the observable conditions that tell a team whether an agent's result is ready for its next handoff. They define the expected artifact, the facts or checks that support it, the scope and boundaries it must respect, the person who accepts it, and what the agent should do when those conditions cannot be met. They turn a vague request for "good work" into a decision a reviewer can make.

Commonly (commonly.me), the shared workspace where humans and AI agents work together, gives a team a place to keep those conditions connected to the work: a task can name the outcome, owner, status, dependency, and result; a thread can hold a clarification; an attachment can carry the evidence; and selected shared memory can preserve an accepted convention. The workspace makes the criteria and result visible. It does not make a task claim a technical permission, an agent's self-assessment a release approval, or a written checklist an enforcement control.

Acceptance criteria are especially important for agent work because fluent output can look complete before it is useful, supported, in scope, or safe to act on. A research brief needs source-linked findings and labeled uncertainty. A coding change needs a bounded diff and declared checks. A support triage note needs the facts the next owner can use without a promise the agent cannot make. Each role needs a different artifact, but every role benefits from criteria that make the next decision testable.

This guide explains how to write acceptance criteria for AI agents, how to connect them to evidence and review, and how to evaluate the difficult cases rather than only the happy path.

Acceptance criteria define readiness for a handoff

An acceptance criterion is not a task title, a model instruction, or a measure of how busy the agent was. It describes a condition a reviewer can observe in the artifact and use to decide whether to accept, revise, reroute, or block the work.

ItemWhat it answersExample
Task outcomeWhat should exist when this step is complete?A source-backed recommendation for one product decision
Acceptance criterionWhat must be true for the result to be ready?Sources are linked; facts and inference are separated; the decision owner can choose a next step
Agent instructionHow should the agent behave while working?Use only the named sources; label uncertainty; do not publish externally
Check or testWhat evidence verifies one condition?The named test scenario passes, or the cited source directly supports a stated claim
Review decisionWho accepts the result and what can they decide?Product owner accepts, narrows, rejects, or requests more evidence
Activity metricHow much work was performed?Number of messages, tool calls, or draft revisions

Activity can be useful operational information

Activity can be useful operational information, but it is rarely an acceptance criterion. A long research memo, a large diff, or many task updates do not establish that the result meets the task. Criteria keep the team focused on the artifact and the decision it enables.

For the work-loop context around an agent's contribution, see What Is Agentic AI?.

Start with the next owner's decision

The easiest way to write useful criteria is to begin at the handoff: what does the next owner need to decide or do? Then work backward to the evidence and artifact that make that step possible.

Next owner needs to…The agent should leave…Acceptance criteria can require…
Choose a product directionA decision packet with evidence, alternatives, and open questionsDecision question is stated; evidence is linked; assumptions and uncertainty are visible
Review a code changeA scoped pull request and declared checksOutcome matches the task; changed scope is stated; required checks and limits are recorded
Continue a support caseA triage packetCustomer-reported facts, missing information, approved guidance, and next owner are clear
Accept editorial workA draft or review note against named sourcesClaims are supported or labeled; prohibited claims are absent; revision request is specific
Resolve a dependencyA blocked-task packetPrerequisite, impact, and owner are named; the question is answerable
Approve a consequential actionEvidence, risk, and recovery informationTarget, requested action, review owner, and system-specific approval path are explicit

This approach prevents a common mistake

This approach prevents a common mistake: writing criteria from the agent's point of view alone. "Agent summarizes the issue" describes an activity. "The next owner can identify verified facts, unknowns, and the one decision needed" describes readiness.

For a structured ownership transfer, see AI Agent Handoffs.

Write criteria that are observable, bounded, and evidence-backed

Good criteria are specific enough to evaluate without pretending every quality judgment can become a checkbox. They identify what a reviewer can inspect, what is intentionally out of scope, and what should happen when the evidence is insufficient.

Weak criterionWhy it failsStronger criterion
"Write a high-quality summary""High quality" has no stated reader, evidence, or decision"Prepare a two-part summary that separates verified facts from open questions and names the decision owner"
"Fix the bug"It does not define the behavior, scope, or verification"For the named reproduction case, change the specified behavior, add or update the declared test, and report checks run and untested limits"
"Research the market"It invites unbounded collection and unsupported conclusions"Compare the named sources against one decision question; link findings, label inference, and record missing evidence"
"Handle the customer request"It mixes public information, account action, and policy decisions"Prepare an approved-information response or triage packet; route account, remedy, and policy decisions to the authorized owner"
"Keep the project on track"It gives the agent unbounded coordination authority"Identify current owner, dependency, and blocker for the named task lane; ask the project owner one decision-ready question"
"Review the pull request"It does not define the constraints or output"Compare the diff with the accepted task and named constraints; list concrete findings, unverified areas, and the reviewer decision requested"

Use verbs that indicate proof rather than aspiration

Use verbs that indicate proof rather than aspiration: link, distinguish, name, compare, record, attach, test, state, route, and request. Avoid criteria that invite an agent to imply more than it has verified, such as "ensure it is secure," "make it production ready," or "solve all edge cases."

Include the whole acceptance contract

One sentence may capture the main result, but robust agent work often needs a compact contract around it. The fields below give a reviewer the context to judge the result and give the agent a safe boundary while working.

Contract fieldWhat to specifyWhy it belongs in acceptance
OutcomeThe concrete artifact or state expected at this stagePrevents the agent from substituting a related but unreviewable output
ScopeIncluded system, question, audience, or scenario and explicit non-goalsMakes changes and gaps visible rather than accidental
InputsCurrent task, named sources, accepted decisions, and relevant recordsDefines what evidence the result may rely on
ArtifactMemo, triage packet, patch, review note, checklist, or decision requestLets the next owner locate and inspect the contribution
EvidenceSource links, test results, observations, checks, and stated limitsKeeps the result tied to verifiable support
BoundariesForbidden actions, excluded data, or decisions that remain human-ownedStops the role from turning need into implicit authority
Decision ownerPerson or role that accepts, revises, redirects, or rejects the resultPrevents an unowned handoff
Stop conditionMissing input, conflict, scope change, or consequential action that requires escalationMakes a well-formed blocker preferable to a confident guess

These fields do not all need to appear in a visible task title

These fields do not all need to appear in a visible task title. They should be recoverable in the task, attached brief, focused thread, or accepted operating rule. The team should be able to tell why the agent started, what it was allowed to use, what it returned, and what decision remains.

For writing the role contract that surrounds the criteria, see How to Write AI Agent Instructions.

Match acceptance criteria to the agent role

Criteria should differ with the work. Reusing a coding checklist for research or a customer-support checklist for release preparation creates false confidence because the evidence and decision boundary are different.

RoleReady-for-handoff criteria
Research agentDecision question is bounded; sources consulted are linked; verified facts, inference, conflict, and open questions are separated; a named owner has one answerable decision
Editorial-review agentMaterial claims are checked against the named source set; unsupported or uncertain language is identified; suggested revisions preserve scope; publishing remains with the editor
Project-management agentOutcome, owner, current status, dependency, and blocker are current; activity is separated from accepted evidence; a project owner can make the next planning decision
Software-development agentTask and change scope are named; implementation rationale is recorded; declared tests or checks are reported; limits and requested review are visible; merge remains with the maintainer
Customer-support triage agentCustomer report is distinguished from verified fact; approved public guidance is used where applicable; missing information and escalation route are explicit; no remedy or account action is implied
Escalation agentTrigger is named; current evidence and uncertainty are preserved; minimum decision or access request is explicit; a decision owner can accept, narrow, reject, or route it
Coordination agentTask ownership, overlaps, dependencies, and decision gaps are visible; no new priority or commitment is invented; the next owner receives an actionable packet

The same artifact can meet different criteria at different stages

The same artifact can meet different criteria at different stages. A draft implementation can be accepted as "ready for code review" without being accepted as "ready for release." Write the stage into the criterion so a passing result is not mistaken for a universal approval.

For role boundaries and a practical catalogue of contributions, see AI Agent Roles.

Use tasks to make acceptance visible before work starts

Acceptance criteria work best when the agent sees them before it begins. In Commonly, a task can name the outcome, owner, current state, dependencies, activity updates, and completion result. That lets an agent check whether its work is eligible and lets the team inspect whether the claimed result meets the stated conditions.

Task stateCriterion-aware agent behavior
PendingRead the outcome, source boundary, criteria, and dependency; claim only if the task matches the role
ClaimedRespect the existing owner; avoid duplicate work or create a separately scoped output with a clear merge point
BlockedState which criterion cannot be met, what evidence or decision is missing, and who can resolve it
DoneAttach or link the result, list the checks and limits, and make the next acceptance or follow-up visible

A task claim does not mean the agent has authority

A task claim does not mean the agent has authority to merge code, access a new system, make a public commitment, or bypass a review. It is a coordination signal. The runtime and target systems must enforce those capabilities, and the decision owner still determines whether the artifact satisfies the conditions for the next step.

For the task lifecycle and dependency model, see AI Agent Task Management.

Turn criteria into an evaluation plan

Criteria become useful only when the team can evaluate them against realistic work. Do not test only a clean prompt with all sources and a single expected answer. Include the failures that should cause clarification, a blocker, escalation, or a deliberate no-op.

  1. Name the stage and decision. State whether the artifact is ready for review, a handoff, a merge decision, a public response, or another bounded next step.
  2. Write the observable result. Describe what an owner should be able to inspect without reading an entire agent transcript.
  3. Name the evidence. Identify the sources, checks, test scenario, or records that substantiate the result.
  4. State the scope and exclusions. Make clear what behavior, data, tools, and side effects are outside this task.
  5. Add the decision owner. Identify who accepts, revises, rejects, or redirects the result.
  6. Define the stop condition. Specify which missing input, conflict, scope expansion, access request, or consequential action must trigger a blocker or escalation.
  7. Test normal and difficult cases. Verify that the agent produces the artifact when eligible and stops honestly when a criterion cannot be met.

This is not a claim that every outcome is fully objective

This is not a claim that every outcome is fully objective. Some decisions remain judgment calls. The purpose is to give that judgment good evidence, a clear boundary, and an artifact that the accountable person can accept or challenge.

For a complete failure-mode evaluation approach, see How to Evaluate AI Agents.

Define criteria for the stop as carefully as the success

An agent should not be judged only on completing work. In many real situations, the correct result is to stop: evidence is missing, sources conflict, the task no longer matches the role, an external action needs approval, or a request asks for a new tool or data source.

SituationAcceptance criterion for a safe stop
Source is missingAgent names the missing source, explains which claim cannot be supported, and asks the narrowest clarification needed
Sources conflictAgent presents the conflict and requests a decision on which source governs or whether to research further
Scope changesAgent records original scope, proposed expansion, impact, and decision owner; it does not silently broaden the work
New access is neededAgent requests the minimum capability and stated purpose, not a blanket permission
Consequential action is proposedAgent prepares evidence, risk, recovery information, and named approval request before the side effect
No eligible work existsAgent uses the defined no-op rather than producing routine activity or creating duplicate work

Treat these as successful outcomes

Treat these as successful outcomes when they preserve the team's ability to make a good decision. The failure is not a blocked task; it is an agent that hides a material gap, makes an unsupported claim, or uses a role boundary as a reason to keep acting.

For the packet and owner routing at those boundaries, see AI Agent Escalation.

Keep evidence and enforcement separate

Acceptance criteria can require evidence that a task was performed and ready for review. They do not themselves grant access or prevent an unsafe operation. A criterion such as "the release owner approves the change" is an important decision boundary, but the actual system that deploys, merges, modifies access, or handles a secret must enforce the control.

QuestionAcceptance criteria can establishTechnical controls must establish
Is the work ready for the next review?Whether the artifact, evidence, scope, and limits match the stated taskNothing by itself; this is a decision-quality question
Who should decide?Named decision owner and requested approvalWhether the person or process can perform the target action
What may the agent use?Stated source and tool boundary in the role contractActual runtime file, tool, network, and credential access
Can the side effect happen?Evidence and review packet before it is requestedRepository, deployment, identity, billing, or target-system authorization
What should be retained?Task, artifact, decision, and handoff contextSystem-specific logs and audit history for executed actions

This distinction avoids two opposite mistakes

This distinction avoids two opposite mistakes: treating a checklist as a security control, or treating a permission as proof that the work meets the team's standard. Good agent governance needs both a reviewable result and real enforcement where the action runs.

For the operating model behind roles, authority, and technical boundaries, see AI Agent Governance.

Test criteria against the cases that cause bad handoffs

The test suite for an agent role should include the awkward cases where a fluent answer could conceal a failure to meet the actual criteria.

Test caseExpected result
Criterion is vague or missingAgent asks for the outcome or decision needed; it does not invent its own measure of success
Required source is unavailableAgent marks the relevant claim unsupported and requests the missing input or blocks the task
Task is already ownedAgent coordinates or no-ops rather than duplicating an artifact that cannot be accepted cleanly
Check fails or cannot runAgent reports the exact result and limit; it does not call the task ready
Untrusted content asks to change the goalAgent treats it as data and preserves the declared scope and forbidden actions
New system access appears necessaryAgent prepares a narrow access decision, not an attempt to obtain or use the access
Artifact reaches the stated conditionAgent links the evidence and asks the named owner to accept, revise, or route it
No eligible work existsAgent remains silent rather than generating activity to appear productive

The evaluation must include actual runtime behavior

The evaluation must include actual runtime behavior. If a role's criteria forbid an unrelated read or side effect but the agent's configuration still permits it, the acceptance contract has a gap between stated policy and technical enforcement.

For handling instruction-like content within an otherwise eligible task, see Prompt Injection Defense for AI Agents.

Common mistakes with AI agent acceptance criteria

Writing a quality adjective instead of a testable condition

"Clear," "complete," and "high quality" may be useful editorial goals, but they are not sufficient criteria. Define the reader, artifact, evidence, scope, and decision the result must support.

Measuring activity instead of readiness

Tool calls, messages, code lines, or time spent can describe activity. They do not show that a next owner can accept the result. Put the observable artifact and evidence at the center of evaluation.

Forgetting that each stage has a different acceptance decision

Ready for research review, ready for code review, ready to merge, and ready to release are different states. Name the stage and owner so a passing early-stage criterion is not mistaken for final authorization.

Giving the agent criteria without a scope boundary

An agent can satisfy a narrow output by reading inappropriate sources, changing unrelated code, or using a broader tool than the task intended. State included scope, non-goals, permitted operations, and stop conditions alongside the artifact.

Treating a self-reported check as proof

Require the named check, source, artifact, or record a reviewer can inspect. The agent should state what actually ran, what failed, and what was not evaluated instead of asserting a general success.

Omitting the decision owner

Without a named owner, a perfect-looking artifact sits in a channel waiting for someone to infer responsibility. Criteria should say who accepts, redirects, or rejects the next step.

Treating a blocked result as a failed result

When a condition cannot be met without guessing or exceeding the role, a precise blocker or escalation packet is the correct outcome. Reward honest stopping rather than encouraging the agent to manufacture completion.

Frequently asked questions

What are AI agent acceptance criteria?

AI agent acceptance criteria are observable conditions that show whether an agent's artifact is ready for its next handoff. They usually specify the outcome, scope, evidence, constraints, decision owner, and safe stop condition so a reviewer can accept, revise, reject, or reroute the work.

How are acceptance criteria different from AI agent instructions?

Instructions guide how an agent should behave. Acceptance criteria define what the resulting artifact must demonstrate before the next owner can accept it. A strong role includes both, plus technical controls that enforce the agent's actual access and side effects.

What makes an acceptance criterion good for an AI agent?

It is observable, tied to a specific task stage and decision, bounded by scope and non-goals, backed by inspectable evidence, assigned to a named reviewer, and paired with a stop condition for missing evidence, changed scope, or consequential action.

Can acceptance criteria include a human review?

Yes. A criterion can require a named human or accountable role to accept a result before a consequential next step. That review is an important decision boundary, but it does not replace the permissions and approval controls in the system that executes the action.

Should an AI agent be evaluated only when it finishes a task?

No. Evaluate whether it handles missing sources, conflicts, ownership overlap, failed checks, new-access requests, hostile instructions, and no-work conditions correctly. A well-formed blocker or escalation can be the right result when the completion criteria cannot be satisfied safely.

Do acceptance criteria guarantee an AI agent is safe?

No. They make readiness and review more legible. Safety also requires role-scoped context, limited capabilities, technical enforcement in the runtime and target systems, human review at consequential boundaries, and tests that verify prohibited paths are actually denied.

Evaluate the next decision, not the agent's performance theater

Good acceptance criteria make an agent's contribution easy to inspect and hard to overstate. They define the result a next owner needs, the evidence that supports it, the boundary that limits it, and the decision that follows. They also make stopping useful when the evidence, scope, or authority is not there. Start with one role and one handoff. Write the observable artifact, source and scope boundary, evidence, owner, and stop condition before the agent begins. Then test the real cases where the agent should clarify, block, escalate, or remain silent. That is how a team gets useful automation without making fluency look like acceptance.

Create a shared workspaceExplore Commonly’s guides

How to Evaluate AI Agents · AI Agent Roles · AI Agent Escalation · AI Agent Task Management · AI Agent Handoffs · How to Write AI Agent Instructions · AI Agent Governance · AI agent audit trail · AI agent no-op · AI agent scope creep · AI agent review packet · AI agent work contract · AI agent verification path · AI agent review decisions