Capability 01 of 04 · Delivery Management
CI/CD Pipeline
Speed and reliability of your build-test-feedback loop for AI-generated code.
5
maturity levels
15
practices
15
guides
L1 · Stage 01Ad-hoc
Practices - what it looks like3 guides
- CI runs on every changeCI pipelines that take longer than 15 minutes are a defining characteristic of the Ad-hoc maturity level.guide→
- Agent waits for CI feedback"Agent is blind, waits for feedback" describes the L1 state where an AI coding agent completes its code changes, submits them, and then has no way to observe whether those changesguide→
- Shared runner, queueA shared runner queue is the default CI infrastructure configuration at L1: a fixed pool of CI runners (virtual machines or containers) shared across all developers, all teams, andguide→
Criteria - what to measure
- 01A CI pipeline runs on pull requests
- 02CI results are reported after the pipeline completes
- 03CI runs on every PR (not just on manual trigger)
- 04Shared runner queue exists even if slow
L2 · Stage 02Guided
Practices - what it looks like3 guides
- CI < 10 minutesCI under 10 minutes is the first meaningful milestone on the path to AI-native delivery infrastructure.guide→
- Basic cachingBasic caching in CI refers to storing dependency packages, compiled artifacts, and Docker layers between pipeline runs so they don't need to be re-downloaded or re-built from scratch on every commit.guide→
- Dedicated runners per teamDedicated runners per team means each engineering team has its own isolated pool of CI runners, not shared with other teams.guide→
Criteria - what to measure
- 01CI completes in under 10 minutes (median)
- 02Build caching is implemented (dependency cache, build artifact cache)
- 03Dedicated CI runners are allocated per team (no shared queue across all teams)
- 04CI duration is tracked as a metric and reviewed monthly
- 05Cache hit rate exceeds 70%
L3 · Stage 03Systematic
Practices - what it looks like3 guides
- CI < 5 minutesCI under 5 minutes is the Systematic (L3) milestone where CI speed becomes a first-class engineering concern, not a background project.guide→
- Bazel + Remote Caching (EngFlow)Bazel is Google's open-source build system, designed from the ground up for large monorepos and fast incremental builds.guide→
- Incremental builds: only changed fragments; per-worktree pipelines (Cursor 3.6 model); inline security validation before PR finalize (GitHub CodeQL/secret scan for third-party agents)Incremental builds are a build strategy where only the files, modules, or packages that have changed since the last build are recompiled, and only the tests that cover changed code are re-executed.guide→
Criteria - what to measure
- 01CI completes in under 5 minutes (median)
- 02Remote caching is implemented (Bazel remote cache, EngFlow, Gradle Enterprise)
- 03Incremental builds run only changed modules or fragments
- 04P95 CI duration is under 8 minutes
- 05Build system supports hermetic builds (reproducible outputs regardless of machine)
L4 · Stage 04OptimizedMost teams aim here
Practices - what it looks like3 guides
- CI < 2 minutesCI under 2 minutes is the Optimized (L4) milestone and represents a qualitative shift in how CI is used.guide→
- Ephemeral microVM sandboxes: agent has own hardware-isolated environment (AWS Lambda MicroVMs / self-hosted E2B-Daytona in your VPC, 10s spin-up)An ephemeral sandbox is a short-lived, fully isolated environment created specifically for a single agent task and destroyed when the task is complete.guide→
- CI as Sandbox: 50 attempts in 5 min without blocking team; CI validation pushed into the coding loop (CircleCI Chunk Sidecars); scheduled / async agents land PRs overnight (Routines, Cursor /loop, CI auto-fix)"CI as Sandbox" is a configuration pattern where the CI system is intentionally designed to support rapid, high-frequency iteration by AI agents, isolated from the normal developer CI workflow.guide→
Criteria - what to measure
- 01CI completes in under 2 minutes (median)
- 02Ephemeral sandbox environments spin up in under 10 seconds for agent CI loops
- 03Agent sandbox CI supports 50+ iteration attempts in 5 minutes without blocking team CI queue
- 04P95 CI duration is under 3 minutes
- 05CI feedback latency (from push to result) is tracked and reported
L5 · Stage 05Autonomous
Practices - what it looks like3 guides
- Sub-minute feedbackSub-minute CI feedback is the Autonomous (L5) frontier - a pipeline that returns meaningful quality signal to an agent in under 60 seconds.guide→
- Self-driving CI: auto-scaling per agent loadSelf-driving CI is a CI system that observes its own load, predicts demand, and scales its infrastructure automatically without any human intervention.guide→
- Production feedback → CI auto-adjusts test suite"Production feedback drives CI test suite adjustment" is an L5 pattern where the CI test suite is not a static artifact maintained by engineers but a dynamic system that evolves baguide→
Criteria - what to measure
- 01CI provides sub-minute feedback for standard changes
- 02CI auto-scales runner capacity based on agent load (no manual capacity planning)
- 03Production feedback loop auto-adjusts the CI test suite (adds tests for observed failures, removes redundant tests)
- 04CI runner utilization stays between 50-80% (auto-scaling prevents both waste and queuing)
- 05Test suite evolution is auditable (each auto-added/removed test has a provenance record)
From the Field
Recent releases, projects and discussions the AI Radar classified into this capability.
release
rust-v0.117.0-alpha.21
release
article
[Kelsey Hightower] This generation of AI tooling will not change the CI/CD process. People will continue setting up dominos of shell scripts and using triggers to knock them over.
article
¡Hola, soy DORA. Why hasn't AI improved my metrics?
release
[email protected]
article
CircleCI Introduces Chunk Sidecars to Bring CI Validation Directly Into AI Coding Workflows
release
release
Which level is your team at in CI/CD Pipeline?
The criteria above are what we check in a facilitated assessment. Run it yourself to place this capability, see which gates you have passed, and compare it against the other 3 in Delivery Management.
Start the assessment →