Incremental builds: only changed fragments

Incremental builds are a build strategy where only the files, modules, or packages that have changed since the last build are recompiled, and only the tests that cover changed code are re-executed. Rather than rebuilding everything from scratch on every commit - the default behavior of most CI pipelines - incremental builds compute the minimal set of work required to validate a change.

The mechanism varies by build system. In Gradle, incremental compilation tracks which Java source files have changed and recompiles only those files and their transitive dependents. In TypeScript, tsc --incremental maintains a tsconfig.tsbuildinfo file that tracks which source files changed and recompiles accordingly. In Bazel, incremental builds are a core property: every build action declares its inputs, and Bazel only re-executes actions whose inputs have changed. In monorepo setups using tools like Turborepo (JavaScript), Nx (JavaScript/TypeScript), or Pants (Python/Java), the build graph is explicit and only the affected packages are built and tested.

For AI agents generating many small, focused commits, incremental builds are disproportionately valuable. An agent fixing a bug in a single module typically changes 2-3 files in a codebase of thousands. Without incremental builds, CI rebuilds and retests the entire codebase for that 3-file change. With incremental builds, CI rebuilds only the changed module and the modules that depend on it. On a large monorepo, this can reduce CI time from 8 minutes to 45 seconds for a typical agent-generated change. The impact compounds: agents that generate 20 small changes per hour get a 10x CI speedup compared to agents working against full-rebuild CI.

The relationship between incremental builds and test impact analysis is complementary. Incremental builds reduce the compilation and build time by only recompiling changed code. Test impact analysis reduces the test execution time by only running tests affected by changed code. Mature CI pipelines at L3-L4 implement both: incremental compilation followed by targeted test selection.

• CI time scales with change size, not codebase size - a 3-file change in a 500-module monorepo should take seconds to validate, not minutes; incremental builds make CI time proportional to the change, not the total codebase
• Agents benefit disproportionately - agents generate many small, focused changes; incremental builds turn each of those small changes into a fast CI run, dramatically increasing the iteration rate
• Enables monorepo CI to stay fast as codebases grow - full-rebuild CI degrades as codebases grow; incremental build CI stays fast because it only rebuilds the changed sub-graph
• Reduces compute cost proportionally - fewer compilation and test execution actions mean lower CI compute costs; for teams with many agent-generated commits per day, the cost savings are substantial
• Makes the "change small things often" pattern sustainable - incremental builds reward the small, focused commit style that AI agents naturally produce, reinforcing the pattern rather than punishing it

1. Identify your build system's incremental build support - Gradle (Java/Kotlin): enable --build-cache and org.gradle.incremental=true. TypeScript: use tsc --incremental or ts-node --transpile-only for faster type checking. Rust: Cargo is incremental by default when CARGO_INCREMENTAL=1. Go: the build cache is incremental by default. JavaScript monorepos: consider Turborepo or Nx for package-level incremental builds.
2. Preserve the incremental build cache across CI runs - Incremental build state is stored on disk (.gradle/, .tsbuildinfo, target/ directory for Rust). Configure your CI cache to persist these directories between runs, keyed on the relevant source files. Without cache persistence, incremental builds restart from scratch every CI run and provide no benefit.
3. For monorepos: implement affected package detection - Turborepo's --filter=[HEAD^1], Nx's affected:test, and Pants' :: with change detection all determine which packages are affected by a commit and run builds and tests only for those packages. Setting up these tools is a one-time investment that immediately benefits all subsequent CI runs.
4. Validate incremental build correctness - Incremental builds can have correctness bugs: a change that should trigger a downstream rebuild doesn't. Run the full build weekly (or on merge to main) to catch any incremental build correctness issues. Compare the incremental build results to the full build results and investigate any discrepancies.
5. Configure your CI system to pass build state between jobs - If your CI pipeline splits into multiple jobs, each job needs access to the incremental build state. Pass build outputs between jobs using CI artifacts or a shared build cache. Don't split the build across jobs in a way that forces each job to recompile independently.
6. Measure the before-and-after impact on CI time distribution - Track p50 and p95 CI time before and after enabling incremental builds. The p50 should drop significantly (typical changes benefit most). The p95 may not drop as much (changes to widely-depended-on modules still trigger large sub-graph builds). The distribution tells you where the remaining optimization opportunity is.

Incremental build false negatives. Not all build systems correctly detect all changes that should trigger a rebuild. If your build system uses file modification timestamps rather than content hashes to detect changes, fast file copies or git operations that don't change content can confuse it. Use content-hash based change detection (Bazel does this by default; Gradle supports it with --no-rebuild and correct task input declaration).

Cache invalidation bugs masking incremental build bugs. If your CI cache is misconfigured and always misses, incremental builds appear to work correctly because they always run from scratch - but you're not getting the speedup. Check whether your CI runs are actually restoring incremental build state by adding a log step that reports the size of the cache restore.

Applying incremental builds without correct dependency declarations. Build system tools infer dependencies between targets from explicit declarations (Bazel BUILD files, Turborepo turbo.json, Nx project.json). If a target has an implicit dependency that isn't declared, incremental builds won't rebuild it when its upstream dependency changes, producing incorrect results. Complete and accurate dependency declarations are required for correct incremental builds.

Over-relying on incremental builds and neglecting full builds. Incremental builds skip work by design. Over time, accumulated incremental build state can become stale or incorrect. Teams that never run full builds lose the safety net of a clean, from-scratch validation. Run full builds on merge to main and nightly on the main branch to catch anything that slips through incremental validation.

Implementing incremental builds in one language without others. In polyglot codebases, implementing Gradle incremental builds for Java while leaving Python builds as full rebuilds creates a two-speed CI. The total CI time is dominated by the slowest full-rebuild component. Address all languages in the build or the improvement is partial.