Acceptance tests from tickets (Autonomous Requirements)

At L3, AI agents read requirements tickets and generate failing acceptance tests before implementation begins - making requirements machine-executable and eliminating circular testing at scale.

·TORS (Test Oracle Reliability Score) is measured and exceeds 90%
·Acceptance tests are auto-generated from ticket requirements (Autonomous Requirements pipeline)
·Incremental test selection runs only tests affected by changed code paths

·TORS is tracked per service or module, not just as an aggregate
·Test generation from tickets includes edge cases, not just happy paths

Evidence

·TORS dashboard showing 90%+ score with per-service breakdown
·Ticket-to-test pipeline configuration with sample outputs
·CI configuration showing incremental test selection (e.g., Bazel test targeting, Jest --changedSince)

What It Is

Acceptance tests from tickets is an L3 workflow where AI agents read requirements tickets (Jira, Linear, GitHub Issues) and automatically generate acceptance tests that encode the intended behavior. These tests fail until the feature is correctly implemented - by design. They are derived from requirements, not from observation of the implementation, which structurally eliminates the circular testing problem that plagues L1-L2 AI test generation.

The workflow looks like this: a product manager or engineer writes a ticket describing the desired behavior ("When a platinum customer applies a discount code, the 20% loyalty tier discount should stack with promotional discounts up to a maximum of 35%"). The AI agent reads the ticket, identifies the testable assertions (loyalty discount is 20%, stacking is permitted, maximum is 35%), and generates acceptance tests that will fail until the implementation satisfies them. The developer's job is to implement the feature against the tests - not to write the tests themselves.

This is the "Autonomous Requirements" capability in the maturity model: requirements are no longer just documentation. They're machine-executable specifications. The acceptance tests generated from the ticket become a formal contract between the product description and the code. When the tests pass, the feature matches the specification. When the feature changes, the tests must be updated - which requires updating the specification.

At Level 3 (Systematic), this workflow is integrated into the ticket lifecycle. Every ticket that describes user-observable behavior triggers automatic acceptance test generation as part of the specification process, before any implementation work begins. The tests are attached to the ticket and included in the PR that closes it.

Why It Matters

Acceptance tests from tickets is one of the highest-leverage practices in the maturity model:

Eliminates circular testing at scale - Tests written from requirements cannot be circular because they don't observe the implementation. The expected values come from the ticket, not the code.
Makes requirements testable - Vague requirements cannot produce specific acceptance tests. The process of generating tests from tickets surfaces ambiguity in requirements before any code is written.
Shifts the quality gate earlier - A failing acceptance test at the start of development is infinitely cheaper than a bug report after deployment. Requirements-derived tests move quality left.
Enables confident AI-generated implementation - When acceptance tests are written from requirements before an AI agent starts implementing, the agent has a clear, objective success criterion. It can iterate until tests pass without requiring human interpretation of "is this correct?"
Creates a living requirements document - The acceptance test suite is always synchronized with the requirements, because tests are generated from tickets and must be updated when requirements change.

Tip

Not all tickets generate meaningful acceptance tests. Refactoring tickets, infrastructure work, and performance improvements don't translate directly to behavior assertions. Focus acceptance test generation on tickets labeled as feature work or bug fixes. Add a ticket template requirement: every feature or bug ticket must include an "acceptance criteria" section in structured format for the AI to read.

Getting Started

Standardize ticket acceptance criteria format - The AI needs structured input to generate structured tests. Define a standard acceptance criteria format for your tickets: "Given [precondition], When [action], Then [expected outcome]." This is the classic Gherkin format, and most AI agents handle it well.
Configure the acceptance test generation agent - Set up an AI agent (Claude, GPT-4, or a custom pipeline) that reads ticket acceptance criteria and generates test scaffolding in your test framework. Start with a prompt that includes your test conventions and a few examples.
Integrate into the ticket workflow - Configure your project management tool to trigger the acceptance test generation when a ticket moves to "In Development" status. The generated tests should be committed to a branch or attached to the ticket for the implementing developer to find.
Validate generated tests before implementation - The implementing developer should review the generated acceptance tests before writing code. If the tests don't capture the intent, fix them before starting - not after. This is the final human checkpoint on requirements quality.
Require tests to fail before implementation starts - Confirm that generated acceptance tests fail on the current codebase before the developer begins implementing. A test that passes before implementation is either testing existing behavior (incorrect) or testing something trivial (insufficient).
Track acceptance test coverage by feature - Measure the percentage of feature tickets that have corresponding acceptance tests. At L3, this should be 90%+ for feature and bug tickets.

6 steps to get from here to the next level

Common Pitfalls

AI generating tests from vague tickets. "Improve the checkout experience" generates no useful acceptance tests. The process of trying to generate tests from vague requirements makes the vagueness visible - which is valuable - but teams need to respond by improving the requirement, not by asking the AI to interpret what "improve" means.

Accepting the first generation without review. AI-generated acceptance tests require human validation before implementation. The agent may misinterpret the acceptance criteria, generate tests at the wrong abstraction level, or miss edge cases the ticket author intended but didn't write explicitly. The review is essential and should be brief - 10 minutes - not skipped.

Letting the implementation inform the acceptance tests. If a developer reads the ticket, starts implementing, and then reviews the AI-generated acceptance tests, they'll naturally interpret ambiguous test cases in ways that match what they've already built. The acceptance tests should be finalized before implementation begins, not during or after.

Over-generating tests from every sentence. Acceptance criteria often includes context, examples, and background information that shouldn't generate tests. The agent needs guidance on which parts of a ticket are testable specifications vs. context. Tuning the prompt - and the ticket format - prevents an overwhelming volume of trivially-true tests.

Mistakes teams actually make at this stage - and how to avoid them

How Different Roles See It

BobHead of Engineering

Bob's team has been producing requirements tickets for years, but the quality is inconsistent. Some are detailed with explicit acceptance criteria; others are vague descriptions of desired outcomes. He wants to implement acceptance test generation but is worried that the ticket quality problem will undermine it.

What Bob should do: The inconsistent ticket quality is actually the first problem to solve, not a reason to delay. Acceptance test generation provides a forcing function for ticket quality: when a ticket can't generate meaningful acceptance tests, that's feedback that the requirement needs clarification. Bob should introduce the ticket template with structured acceptance criteria as a prerequisite to acceptance test generation, and then use the quality of the generated tests as feedback on the requirement-writing process. Within two months, ticket quality will measurably improve because the consequences of vague requirements are visible in the generated tests.

What Bob should do - role-specific action plan

SarahProductivity Lead

Sarah is looking for metrics that demonstrate quality improvement and faster time-to-value for features. She's been told that acceptance tests from tickets will help, but she's not sure how to measure the impact.

What Sarah should do: The key metric is bug escape rate - bugs that pass testing and reach production. Requirements-derived acceptance tests should catch more pre-production bugs than circular L1 tests. Sarah should track bug escape rate before and after implementing acceptance test generation, segmented by ticket type: tickets with structured acceptance criteria vs. those without. If the structured tickets produce fewer production bugs, that's the business case. The secondary metric: time from ticket to merge. Acceptance tests generated before implementation should reduce back-and-forth between implementation and requirements clarification, shortening cycle time.

What Sarah should do - role-specific action plan

VictorStaff Engineer - AI Champion

Victor is excited about acceptance test generation from tickets and has already prototyped it for his team. But he's hit a challenge: the AI agent generates tests at inconsistent abstraction levels - sometimes unit tests, sometimes end-to-end tests - depending on how the acceptance criteria is phrased.

What Victor should do: The inconsistency is a prompt engineering problem, not a fundamental limitation. Victor should refine the generation prompt to specify the target abstraction level: "Generate component-level acceptance tests using [test framework], not unit tests or full end-to-end tests." He should also create a few reference examples in the prompt - one good acceptance test for a representative feature - to anchor the AI's output style. The prototype Victor built is worth productionalizing because the first-mover advantage in his organization is significant: once teams experience having acceptance tests before implementation, they won't want to go back.

What Victor should do - role-specific action plan