Capability 01 of 04 · Delivery Management
CI/CD Pipeline
Speed and reliability of your build-test-feedback loop for AI-generated code.
5
maturity levels
15
practices
15
guides
L1 · Stage 01Assisted
Practices - what it looks like3 guides
- CI runs on every changeCI pipelines that take longer than 15 minutes are a defining characteristic of the Assisted maturity level.guide→
- Agent waits for CI feedbackThe L1 state where an agent ships its changes and then waits, blind: it cannot read CI, tests or lint output, so a human has to relay every result back.guide→
- Shared runner, queueThe L1 CI default: one fixed pool of runners for everyone, first in first out, so a quick lint check queues behind a full integration suite.guide→
Criteria - what to measure
- 01A CI pipeline runs on pull requests
- 02CI results are reported after the pipeline completes
- 03CI runs on every PR (not just on manual trigger)
- 04Shared runner queue exists even if slow
L2 · Stage 02Delegated
Practices - what it looks like3 guides
- Pipeline definitions live in the repo and are reviewed like application codePipeline definitions live in the repository next to the application they build, and changes to them go through the same review, the same checks and the same history as any other change.guide→
- Dedicated runners per teamDedicated runners per team means each engineering team has its own isolated pool of CI runners, not shared with other teams.guide→
- CI < 10 minutesCI under 10 minutes is the first meaningful milestone on the path to AI-native delivery infrastructure.guide→
Criteria - what to measure
- 01Pipeline definitions are versioned in the repository and changed through code review
- 02Dedicated CI runners are allocated per team (no shared queue across all teams)
- 03CI completes in under 10 minutes (median)
- 04CI duration is tracked as a metric and reviewed monthly
- 05Cache hit rate exceeds 70%
L3 · Stage 03Systematic
Practices - what it looks like3 guides
- Agent CI treated as internet-facing: no `${{ github.event.* }}` interpolated into `run:`, agent passes split into separate jobs with per-job token scopeIncremental builds are a build strategy where only the changed files, modules, or packages are recompiled and only the tests covering changed code re-run, with each agent given its own worktree pipeline and the agent CI itself hardened as an internet-facing surface.guide→
- Per-worktree pipelines, so parallel agents do not serialise on one runnerEvery worktree - every agent, every branch, every parallel line of work - gets its own pipeline instance, so parallel agents do not serialise behind each other on a single shared runner.guide→
- CI < 5 minutesCI under 5 minutes is the Systematic (L3) milestone where CI speed becomes a first-class engineering concern, not a background project.guide→
Criteria - what to measure
- 01Agent CI runs are isolated from untrusted input: no event data interpolated into shell steps, and agent passes run as separate jobs with separately scoped tokens
- 02Parallel agents get per-worktree pipelines rather than serialising on one runner
- 03CI completes in under 5 minutes (median)
- 04P95 CI duration is under 8 minutes
- 05Build system supports hermetic builds (reproducible outputs regardless of machine)
L4 · Stage 04GovernedMost teams aim here
Practices - what it looks like3 guides
- CI as Sandbox: 50 attempts in 5 min without blocking team; merge queues for parallel agent fleets (auto-merge only what builds and passes tests); scheduled / async agents land PRs overnight"CI as Sandbox" is a configuration pattern where the CI system is intentionally designed to support rapid, high-frequency iteration by AI agents, isolated from the normal developer CI workflow.guide→
- Every CI run for an agent gets its own disposable environment, so a poisoned run cannot reach the next oneEvery CI run triggered by an agent executes in its own short-lived, fully isolated environment that is destroyed afterwards, so a poisoned run cannot reach the next one.guide→
- CI < 2 minutesCI under 2 minutes is the Governed (L4) milestone and represents a qualitative shift in how CI is used.guide→
Criteria - what to measure
- 01Agent sandbox CI absorbs repeated agent iteration without blocking the team CI queue
- 02Ephemeral sandbox environments spin up in under 10 seconds for agent CI loops
- 03CI completes in under 2 minutes (median)
- 04P95 CI duration is under 3 minutes
- 05CI feedback latency (from push to result) is tracked and reported
L5 · Stage 05Self-improving
Practices - what it looks like3 guides
- Sub-minute feedbackSub-minute CI feedback is the Self-improving (L5) frontier - a pipeline that returns meaningful quality signal to an agent in under 60 seconds.guide→
- Runner capacity auto-scales with agent load, with no manual capacity planningRunner capacity auto-scales with agent load: the CI system observes its own queue, provisions and releases runners as demand moves, and nobody does manual capacity planning.guide→
- Production feedback → CI auto-adjusts test suite; every production failure becomes a permanent regression test; verification moves before the PR opens and CI checks the evidence instead of re-running it (main-branch success fell to a five-year low of 70.8% across 28M workflows)The test suite stops being a static artifact: an incident in production generates its own regression test and adds it to CI automatically.guide→
Criteria - what to measure
- 01Production feedback loop auto-adjusts the CI test suite (adds tests for observed failures, removes redundant tests)
- 02CI auto-scales runner capacity based on agent load (no manual capacity planning)
- 03CI provides sub-minute feedback for standard changes
- 04CI runner utilization stays between 50-80% (auto-scaling prevents both waste and queuing)
- 05Test suite evolution is auditable (each auto-added/removed test has a provenance record)
From the Field
Recent releases, projects and discussions the AI Radar classified into this capability.
release
openai/codex
release
kortix-ai/suna
article
[Kelsey Hightower] This generation of AI tooling will not change the CI/CD process. People will continue setting up dominos of shell scripts and using triggers to knock them over.
article
¡Hola, soy DORA. Why hasn't AI improved my metrics?
release
n8n-io/n8n
article
CircleCI Introduces Chunk Sidecars to Bring CI Validation Directly Into AI Coding Workflows
release
google-gemini/gemini-cli
release
trycua/cua
Which level is your team at in CI/CD Pipeline?
The criteria above are what we check in a facilitated assessment. Run it yourself to place this capability, see which gates you have passed, and compare it against the other 3 in Delivery Management.
Start the assessment →