# Sub-minute feedback

> Sub-minute CI feedback is the Autonomous (L5) frontier - a pipeline that returns meaningful quality signal to an agent in under 60 seconds.

Perspective: delivery
Source: https://visdom-maturity-matrix.virtuslab.com/guides/delivery/sub-minute-feedback

## What It Is

Sub-minute CI feedback is the Autonomous (L5) frontier - a pipeline that returns meaningful quality signal to an agent in under 60 seconds. This is not a theoretical aspiration; engineering organizations operating at L5 have demonstrated it in production. The mechanism is a combination of pre-warmed runner pools (zero startup latency), distributed build execution across many machines, highly selective test impact analysis (running only the 10-30 tests directly affected by a change), and pre-computed build artifacts that eliminate compilation steps entirely.

At sub-minute feedback, the CI loop becomes so fast that it changes the fundamental architecture of how agents work. Agents no longer submit a change and wait - they submit, receive feedback, and loop continuously at machine speed. An agent completing 60-80 iteration cycles per hour can work through a complex feature implementation, discover its own edge cases, write tests for them, and converge on a working solution - all without human intervention during the iteration phase. The human reviews the final output, not the intermediate steps.

The distinction between "fast CI" and "sub-minute feedback" is important. Sub-minute feedback does not mean the entire CI suite runs in under 60 seconds - that would require a test suite of trivial size. It means the agent receives a meaningful, actionable quality signal in under 60 seconds. That signal might be: lint passed, type checking passed, and the 23 tests directly affected by your change passed. The full integration suite runs asynchronously and catches regressions at a slower cadence. The sub-minute signal is sufficient for the agent's inner loop; the asynchronous full suite is the safety net.

Achieving sub-minute feedback requires vertical integration of the build system, the CI platform, the test infrastructure, and the caching layer. This is not something that can be bolted onto an existing GitHub Actions workflow with a few YAML changes. It requires a platform engineering investment - typically a dedicated team that owns CI infrastructure as a product. Teams at L5 treat CI speed with the same engineering rigor they apply to production latency.

## Why It Matters

- **Agents reach autonomous operation** - at sub-minute feedback, agents close their own iteration loops without human involvement in each step, enabling true autonomous task completion
- **Pipeline throughput scales linearly with agent count** - a fleet of 20 concurrent agents can each iterate 60+ times per hour, generating the kind of throughput that changes delivery economics
- **Test-driven agent development becomes natural** - when the red-green cycle takes 45 seconds, agents naturally work in a TDD style: write failing test, implement to pass, repeat - matching the fastest human TDD practitioners
- **Eliminates CI as a limiting factor** - at sub-minute feedback, CI is no longer the bottleneck in the agent iteration loop; agent reasoning time and human review become the limiting factors
- **Demonstrates true AI-native infrastructure** - sub-minute CI is a marker of organizations that have redesigned their infrastructure for AI, not just adopted AI tools on top of human-designed infrastructure

## Getting Started

1. **Build a test impact analysis system with sub-20 test selection** - For sub-minute feedback, the selected test set must be tiny. Invest in a test impact system that maps every code path to covering tests at the function level (not the file level). Tools like Launchable's function-level analysis or custom instrumentation of your test framework can achieve this. The target: any typical agent change should trigger fewer than 50 tests.
2. **Deploy pre-warmed runner containers with zero startup latency** - Use Kubernetes-based runner pools (Actions Runner Controller for GitHub Actions, self-hosted BuildKite agents with persistent containers) where containers are pre-warmed with your full build environment. A pre-warmed container can accept a job in under 2 seconds compared to 60-120 seconds for a cold VM.
3. **Implement remote build execution (RBE)** - Bazel with EngFlow, BuildBuddy, or Google's RBE service distributes build actions across a cluster. A build that takes 2 minutes on one machine takes 15 seconds distributed across 16 machines. This is the infrastructure layer that makes sub-minute build+test achievable for non-trivial codebases.
4. **Pre-compute build artifacts for common configurations** - Instead of rebuilding from source on every CI run, maintain a build artifact cache keyed on precise source hash. When an agent changes one file in a module, only that module's artifact needs rebuilding - all dependent artifacts serve from cache. Bazel's content-addressed caching does this automatically when configured correctly.
5. **Implement asynchronous full suite in parallel** - The sub-minute feedback is the agent's inner loop. The full integration suite must still run, but asynchronously - kicked off immediately when the agent pushes, delivering its results within 5-10 minutes. Design the CI system so agents can act on sub-minute results while the full suite validates asynchronously.
6. **Instrument and alert on feedback latency as an SLO** - Define a service-level objective: 95% of CI runs triggered by agent commits must complete the fast path in under 60 seconds. Alert when this SLO is missed. Treat CI latency with the same operational rigor as API latency - it is infrastructure with users (agents) who are affected by slowness.

> **Tip**: Pre-warmed containers are often the highest-impact change for teams trying to get from 2-minute CI to sub-minute. If your 2-minute CI pipeline spends 45 seconds on runner startup, eliminating startup time gets you to 75-second CI before any other optimization. Start there before attempting the more complex distributed build work.

## Common Pitfalls

**Treating sub-minute feedback as the same as sub-minute full CI.** Sub-minute feedback means the agent gets a fast, meaningful signal. It does not mean the entire test suite runs in 60 seconds. Teams that conflate these targets will either strip out too many tests (false confidence) or chase an impossible goal. Clearly define what the sub-minute feedback signal covers and ensure it's genuinely meaningful.

**Adopting Bazel without organizational commitment.** Bazel provides the distributed build infrastructure that makes sub-minute feedback achievable at scale, but it requires deep investment: BUILD file maintenance, training, and a platform team willing to own the build system. Teams that adopt Bazel without this commitment end up with unmaintainable BUILD files and a build system that fights rather than helps development.

**Under-provisioning the RBE cluster.** Remote build execution distributes work across a cluster of machines. If the cluster is undersized for peak agent load (which generates bursty, simultaneous CI runs), jobs queue on the cluster and the latency benefit disappears. Size the cluster for peak concurrent agent load, not average load.

**Neglecting the asynchronous full suite.** The fast sub-minute path is the agent's inner loop, but it only covers a subset of tests. The asynchronous full suite must actually run and must actually be monitored. Teams that implement sub-minute feedback but let the async full suite become flaky and ignored end up with fast CI that catches only the obvious failures.

**Failing to track false negative rate.** Sub-minute CI with test impact analysis has a false negative rate - sometimes it selects tests that pass even though a change has introduced a bug. Track this rate by comparing sub-minute results to full suite results on a sample of changes. If the false negative rate is above 1%, the impact analysis system needs improvement.

## Bob - Head of Engineering

Bob's team is operating at L4 with 2-minute CI and running 30-40 agent-assisted PRs per day. The productivity gains are real and visible. But he's seeing a ceiling: agents still require human check-ins during long sessions because they can't iterate fast enough to complete complex tasks autonomously. The agents complete the easy parts quickly but stall on the parts that require 15-20 iterations to get right. Bob wants to reach the point where agents can complete full features autonomously, not just the easy subtasks.

Bob should recognize that sub-minute CI is the infrastructure prerequisite for autonomous agent operation and frame the investment accordingly. He needs to present it as: "our agents are currently capable of autonomous operation, but our CI infrastructure requires human supervision because the feedback loop is too slow for agents to close it themselves." The investment - pre-warmed runner pools and distributed build execution - is an infrastructure cost, not a tooling cost. Bob should work with the platform engineering team to scope the work, establish a timeline, and define the SLO (95% of agent-triggered CI runs complete in under 60 seconds). Autonomous agent operation is a product outcome; sub-minute CI is the infrastructure that enables it.

## Sarah - Productivity Lead

Sarah has been tracking agent iteration rate across the team and has clean data: at 5-minute CI, agents average 8 iterations per hour; at 2-minute CI after last quarter's infrastructure work, they average 22 iterations per hour. The relationship is clear and linear. She can model what sub-minute CI would unlock: 60+ iterations per hour, and the resulting completion rate for complex agent tasks.

Sarah should use this model to make a concrete prediction: with sub-minute CI, what percentage of complex agent tasks (currently requiring human intervention to push through the 10-20 iteration phase) would complete autonomously? She should survey current agent users: how many times per session do they need to intervene to "unstick" an agent that's iterating on CI failures? That intervention rate is the cost of slow CI. Sub-minute CI eliminates most of those interventions, and Sarah can estimate the hours recovered per developer per week. A clear before-and-after prediction, validated after implementation, is how Sarah demonstrates the infrastructure ROI and makes the case for the next L5 investment.

## Victor - Staff Engineer - AI Champion

Victor is already running sub-minute feedback for his personal development workflow using a custom local pre-push hook that runs a minimal test set in 20 seconds. He's proven the concept works. But the local hook is a personal tool - it doesn't help the team or give agents the same benefit in CI.

Victor should design the production sub-minute CI system starting from his local proof of concept. The local hook already has the test selection logic (which tests to run for which changes). Porting that logic to a CI step is the core work. Victor should then scope the runner infrastructure: how many pre-warmed containers are needed for the team's current push rate? What does the cluster cost per month? What's the RBE system that fits the team's build setup (EngFlow for Bazel, BuildBuddy as an alternative)? Victor should produce a technical design document, not a blog post: concrete infrastructure choices, cost estimate, implementation timeline, and success metrics (the SLO). That document is the basis for an engineering project, not a research spike.

## Links

- [EngFlow - Remote Build Execution](https://www.engflow.com/)
- [BuildBuddy - Bazel remote caching and RBE](https://www.buildbuddy.io/)
- [Actions Runner Controller - Kubernetes-based GitHub Actions runners](https://github.com/actions/actions-runner-controller)
- [Launchable - Function-level test impact analysis](https://www.launchableinc.com/)
- [Google Cloud - Remote Build Execution](https://cloud.google.com/build/docs/optimize-builds/use-built-in-caching)