v1.6September 1, 2026/covers August 2026

September 2026: The Factory Floor

July said verification was the limit on how much you can delegate. August built the machinery underneath it, and sent the invoice. 83 of 240 matrix items were rewritten, and three surplus rows were removed.

At a glance

Autonomy is judged by your written policy, not by who clicks approve

Why: Claude Code made auto mode the default on August 14, six weeks after making Manual the default. The numbers behind the reversal: developers approve 97% of permission prompts, and in a 1,053-person study humans caught 13.6% of dangerous commands where a classifier caught 89%.

Coding Agent Usage L2, Governance L2

Agent runs are scored by how they end, not by an approval percentage

Why: Vercel published a working software factory in which every agent run terminates as success, flawed, blocked or manual, and each failing class routes to a different repair. Only success ships.

Metrics L4

AI-written code now has a measured running cost

Why: A study of 3.52M changes in one enterprise codebase found AI-generated code carries higher coupling and copy overhead, showing up as 5-8% more compute consumed in production. Meta's telemetry pairs a 220% rise in changes with a 40% rise in incidents.

Metrics L3 and L5, Observability L4

Opening a repository is now treated as running its code

Why: The ChainDrop compromise planted persistence in .vscode/tasks.json and .claude/settings.json, so the payload ran on folder open with no install step. The package signatures were valid; they proved the source, not its safety.

Agent Runtime L2, Governance L3

An agent pull request needs a named human owner to merge

Why: Across 8.1M pull requests, agent-authored ones merge at 79% in the strongest organisations and 37% in the rest. The variable is not tooling; it is whether a person's name is on the change.

Code Review L4, Merge & Deploy

Test process is no longer prescribed to agents; the outcome is measured instead

Why: A controlled comparison found agents told to write tests first scored consistently worse at three to eight times the token cost, because they harden a bad design around the first test. The sample is small and the author says so, but mutation score is the better instrument either way.

Testing Strategy L3 and L4

What changed in the model

Three changes to how the model is written. Each one changes how your own score should be read.

No capability is counted twice

Some practices were claimed by two areas at once, so the same strength could lift two scores. Each capability now belongs to exactly one area. If your score moves slightly against last month, this is the most likely reason, and the new number is the honest one.

Criteria describe the capability, not the tool

Where a named product or framework had become the bar, teams that solved the same problem another way scored lower for no good reason. A dependency-graph build with a shared cache is the criterion; Bazel, Nx and Turborepo are all ways to meet it. The same applies to delivery metrics, where DORA is now one option among equivalents.

Thresholds no longer assume you are a large company

Bars like "50+ pull requests a day" or "400+ tools" measured headcount rather than maturity, and a small team could not reach them however well it worked. Those now read against your own baseline before agents. Percentages of your own organisation were left as they were, because they already scale.

Every change, by area and level

2026-07 2026-08 - July taxonomy preserved unchanged.

Development

Coding Agent Usage

L2Autonomy is set by a written rule, not a per-prompt click: Claude Code auto mode became the default Aug 14 behind a classifier, but permissions.deny and ask still override it and Enterprise/API stay opt-in

L3August model refresh: Claude Opus 5, GPT-5.6 on Codex, Grok 4.6 (500K context, $2/$6), Meta Muse Code ($1.25/$4.25); open-weight workers now GLM-5.3, Qwen3.8-Max, DeepSeek V4 Pro

L4Routing must be re-costed on a schedule: the cheap tier inverted in four weeks, with Sonnet 5 locked at $2/$10 and DeepSeek V4 Pro shipping with a price increase

L4The parallelism ceiling is orchestrator context pollution, not token cost: cap batches at 2-4, restrict status polling, no concurrent repo-wide git operations

Context Engineering

L2Agent config belongs in the repo: across 441 repos, those without committed AI configuration showed twice the cognitive-complexity growth (+53% vs +27%) and 1.7x the static-analysis warnings

L3Context budgeting now carries a standing eviction rule - lint leakage 62%, context bloat 42%, skill leakage 35% across 100 repos, and Anthropic cut over 80% of Claude Code's system prompt with no measurable quality loss

L373.8% of agent config files are committed once and never touched again - the failure mode is accretion, not absence

Code Review & Quality

L4Auto-approval is driven by a risk classifier trained on your own incident history (Zalando: 33% of PRs auto-approved, 20-40% lead-time reduction), and every agent PR carries a named human owner

L4Ownership is the hidden merge gate: across 8.1M PRs, agentic PRs merge at 79% in elite orgs and 37% in the rest, and the variable is a named human owner

L5Fleet self-review is bounded - a drafting agent never approves its own work, and LLM defect detection degrades across successive review rounds

L1Model upgrades do not buy security: average GenAI security pass rate flat year on year at 56%, with coding-specialised models no better than general-purpose ones

Testing Strategy

L3Test process is no longer mandated to agents - TDD inside the agent loop lost to unconstrained agents at 3-8x more tokens, because agents harden bad design around the first test (five-batch study, cite as provocation)

L4Mutation testing on high-risk paths is the real coverage signal: one component measured 100% line coverage at 61% mutation strength

L1Acceptance rate retired as a quality signal - 31% of accepted AI completions are deleted outright, most within 15 minutes

Delivery Management

CI/CD Pipeline

L3Agent CI is internet-facing: one public GitHub issue reached CI secrets in three vendors' own repos (Gemini CLI CVE-2026-12537 at CVSS 10.0). No github.event.* interpolated into run:, agent passes split into separate jobs with per-job token scope

L5Every production failure becomes a permanent regression test - the feedback layer of the six-layer agent reliability model

Merge & Deploy

L3Agent output lands as a stack of dependent, independently reviewable branches rather than one 1,000-line PR

L4AI-assistance disclosure enforced in CI, not by convention: Linux 7.2 carried 1,111 Assisted-by commits and at least one maintainer strips the tags on the way in

Metrics

L4Auto-Approve Rate replaced by four-state run classification - success / flawed / blocked / manual - where each non-success class routes to a different fix and only the classification rate justifies widening the automation boundary

L3Cost per iteration now includes the compute the generated code itself burns in production, measured at +5-8% across 3.52M changes

L5Throughput scorecards paired with incidents-per-merged-change and firefighting hours: Meta saw changes +220% and features +36% against incidents +40% and firefighting time +70%

L2New anti-pattern: metering adoption by token spend or seat activity is Goodhart-complete and gamed within weeks

Governance & Compliance

L2The control is a declarative, version-controlled deny/ask ruleset - developers approve 97% of permission prompts and caught 13.6% of dangerous commands where a classifier caught 89%

L2EU AI Act restated: Digital Omnibus in force July 27 as Regulation (EU) 2026/1744. Article 50 transparency applies from Aug 2 2026; high-risk stays deferred to Dec 2027 / Aug 2028

L3Repository-supplied agent config (.claude/, .vscode/, .cursor/, .gemini/, build.rs) on a mandatory-diff path - ChainDrop executed on folder open with no install step

L4Skills and MCP servers allowlisted and pinned rather than scanned: all eight open-source skill scanners were bypassed by one file, and 73.2% of confirmed-malicious skills used undocumented shadow features. Assessed against OWASP Agentic Skills Top 10 v1.0

Organization

AI Adoption Model

L3Platform team as a central proxy or agent registry handing out identity, cost tracking, sandboxing, observability and evals by default - declaring an agent drops from weeks to minutes

L3Enablement as a named programme with countable attendance (a standing guild, guided hackathons, hands-on labs), and no mandated tool - the platform is standard, the choice is free, outcomes are measured

L3Zalando's honest harms travel with the pattern: complexity inflection points, ~5,000-character agent commit messages, ballooning PR sizes, and engineers gaming the risk classifier by splitting PRs

Knowledge Management

L4Agent Plugins 1.0 packages skills plus MCP servers into one vendor-neutral installable (AWS, Anysphere, Microsoft, OpenAI, Vercel, Google), then treated as maintained assets with a review cadence and an eviction rule

L4MCP servers assessable from published metadata before a client connects - the Server Card .well-known proposal

Team Structure & Roles

L3Judgment relocates rather than disappears: problem selection, architecture, the quality bar, which signals to trust, and shipping authority stay human even when authorship does not

L3The harness, not the model, is the asset that survives a vendor swap

L4Span of control is bounded by the orchestrator's context, not tokens - batches of 2-4, restricted polling, overlapping file ownership read as a signal to consolidate

Tech Debt & Modernization

L3Refactoring has a measurable payoff: one refactor cut input tokens for every future change to that code by 83%, roughly $0.40 per change, compounding

L4The permanently-deferred migration is now a two-week job, so the backlog is a choice rather than a constraint

Infrastructure

Agent Runtime & Sandboxing

L2Untrusted repositories opened with auto-run hooks disabled - ChainDrop planted persistence in .vscode/tasks.json (runOn: folderOpen) and .claude/settings.json (SessionStart), executing before any install

L2Short-lived credentials with spend caps and baseline alerts: stolen keys reach gray-market proxies within minutes, and one case reached about $1M before containment

L3Egress denied by default with destinations allowlisted, and the evaluation environment inside the security boundary - UK AISI recorded 19 unsanctioned actions across 122 runs, one fabricating identities to pressure a maintainer

L5A replaceable harness as a maturity criterion: model adapter, tool registry, session log and the agent loop itself swappable without a rewrite

MCP & Tool Integration

L2The stateless MCP core is shipping - servers drop sticky sessions and run serverless - and Roots, Sampling and Logging are on a 12-month clock

L3Central IdP auth moving toward federated workload identity and sender-constrained tokens (CIMD, DPoP, Workload Identity Federation)

L4Injection resistance tested in the IDE configuration you actually ship: the same model held at 0% compliance via API and 100% inside an editor

Build System

L4Worktree isolation per agent now ships as a vendor default rather than being assembled by hand

Observability & Feedback Loop

L4Diagnosis stays human: agents are superhuman at observing and unreliable at orienting, where correlation gets mistaken for causation

L4Incident rate and firefighting hours tracked against change volume, because the two move in opposite directions

The month in numbers

Context behind the edition. The figures that drove specific changes are cited with those changes above.

37%/6%

report any / significant EBIT impact from AI - both flat year on year

McKinsey

56%

security pass rate for generated code, unchanged in a year - model upgrades do not buy it

Veracode

90%/68%

of developers use coding agents weekly / daily, across 15,000 surveyed

JetBrains

31%

of accepted AI completions are deleted outright, most within 15 minutes

arXiv

26%

of AI-written vulnerability patches fully fix the flaw without changing behaviour

1Password

1,111commits

in Linux 7.2 tagged Assisted-by, with some maintainers stripping the tag

LWN

Updated Guides

83 of 240 matrix items were rewritten this edition and three surplus rows were removed. 72 guides changed; the 44 below are the ones rewritten around new August evidence.

Development

Agent in IDE (auto mode)

Auto mode became the default Aug 14 behind a classifier - but deny/ask rules still override it

Development

CLI agents as primary

August model refresh and the four-week price inversion; Grok 4.6, Muse Code, DeepSeek V4 Pro up

Development

One-shot unattended agents

Four-state run classification; goals and loops; a drafting agent never self-approves

Development

3-5 parallel agents per developer

The ceiling is orchestrator context pollution, not tokens - cap batches at 2-4

Development

CLAUDE.md basics

Repos without committed agent config showed twice the cognitive-complexity growth

Development

Context budgeting

A standing eviction rule: 62% lint leakage, 42% bloat, and Anthropic cut over 80% of its system prompt

Development

Policy-based auto-approval

Risk classifier trained on your own incidents (33% auto-approved) plus a named human owner

Development

Agent fleet self-reviews

LLM defect detection degrades across review rounds; the verification ratchet has a floor

Development

Test-oracle reliability

Process no longer mandated to agents; 31% of accepted completions deleted within 15 minutes

Development

Mutation testing validation

100% line coverage at 61% mutation strength - the number that ends the coverage argument

Delivery

Incremental builds

Agent CI treated as internet-facing: no github.event.* in run:, split jobs, scoped tokens

Delivery

Production feedback into CI

Every production failure becomes a permanent regression test

Delivery

Policy-based merge rules

Agent output as a stack of dependent, independently reviewable branches

Delivery

Canary / progressive deployment

Disclosure enforced in CI - Linux 7.2 had 1,111 Assisted-by commits and maintainers stripping tags

Delivery

Licenses vs usage rate

Never token spend or seat activity as an adoption target - both are gamed within weeks

Delivery

PR throughput per developer

Acceptance rate retired: 31% of accepted completions are deleted outright

Delivery

Cost per iteration

Now includes the compute the generated code burns in production, measured at +5-8%

Delivery

Run classification (was auto-approve rate)

Rebuilt around success / flawed / blocked / manual, each routing to a different fix

Delivery

Cost per feature

Paired with incidents-per-merged-change and firefighting hours, which move the other way

Delivery

Official AI tool policy

A declarative deny/ask ruleset, not a human clicking approve - 97% of prompts are approved

Delivery

EU AI Act awareness

Article 50 transparency applies from Aug 2; high-risk stays deferred to Dec 2027 / Aug 2028

Delivery

Policy-as-code

Repo-supplied agent config on a mandatory-diff path; org-wide default model now enforceable

Delivery

Automated compliance checks

Allowlist and pin instead of scanning - all eight skill scanners were bypassed by one file

Organization

Platform team owns AI tooling

Zalando and Ramp: a central proxy or registry handing out identity, cost, sandbox and evals

Organization

IDP with an AI layer

Enablement as a programme with countable attendance, not a launch email

Organization

Standardized agent setup

No mandated tool: the platform is standard, the choice is free, outcomes are measured

Organization

Context fabric

Servers assessable from published metadata before a client ever connects

Organization

Skills and plugins as governed assets

Agent Plugins 1.0 packaging, plus a review cadence and an eviction rule

Organization

Training and enablement

Instruct confidently, then verify confidently - which is not reading every line

Organization

Review shifts up the lifecycle

Judgment relocates: five points that stay human even when authorship does not

Organization

Platform / harness engineer

The harness, not the model, is the asset that survives a vendor swap

Organization

Span of control

Bounded by orchestrator context pollution - batches of 2-4, restricted polling

Organization

Continuous modernization

Refactoring with a measurable payoff: -83% input tokens on every future change

Organization

Cross-repo migration agents

The permanently-deferred migration is now a two-week job

Infrastructure

Basic sandboxing

The repository is an execution surface - ChainDrop ran on folder open, no install needed

Infrastructure

Scoped agent credentials

Short-lived keys, spend caps, baseline alerts - stolen keys are resold within minutes

Infrastructure

Network isolation

Egress denied by default, and the evaluation environment inside the security boundary

Infrastructure

Agent fleet on dedicated compute

The harness is unbundling - can you swap the model without rewriting it?

Infrastructure

Basic tool authorization

Stateless MCP core shipping; Roots, Sampling and Logging on a 12-month clock

Infrastructure

RBAC per MCP tool

Toward federated workload identity and sender-constrained tokens

Infrastructure

MCP governance

Server Card assessment before connecting; test injection resistance in the IDE you ship

Infrastructure

Agent-specific build profiles

Worktree isolation per agent now ships as a vendor default

Infrastructure

Anomaly to agent investigation

Incident rate and firefighting hours tracked against change volume

Infrastructure

Self-healing known patterns

Diagnosis stays human: superhuman at observing, unreliable at orienting

How the matrix is built

The shape

4 perspectives x 4 areas x 5 levels x 3 items = 240, and 240 is also the guide count. Every item owns exactly one guide and no guide is claimed twice. Levels run Assisted, Delegated, Systematic, Governed, Self-improving.

Two layers

The matrix is what we show: one sentence per item, in a practitioner's language. The maturity gates are what we assess: Must, Should, Prerequisites and Evidence per area and level, and they decide the workshop score. Both layers must say the same thing; a gap between them is a defect, not a nuance.

Rules for writing a criterion

Describe the capability. A vendor or a methodology belongs in parentheses as an example, never as the bar.
A threshold must not encode company size. A percentage of your own organisation scales; an absolute volume measures headcount.
L1 describes a floor a team actually stands on, never an absence. Deficit wording is blocked by a test.
Each capability has one owning area, so nobody scores twice for one practice.

How it is kept honest

Every edition is checked automatically against these rules before it ships: that the matrix and the assessment still say the same thing, that no capability is claimed by two areas, and that nothing is displayed which the assessment never actually scores. Previous editions are never edited, so a score from an earlier month stays comparable to what it meant at the time.

Known and deliberate

The level names promise a delegation and control arc, but CI/CD Pipeline and Build System still ladder on latency: their L4 criteria are wall-clock targets with no governance in them. Re-laddering those two areas would move existing workshop scores, so it is a decision about the model rather than a tidy-up.

What Didn't Change (and Why)

Matrix structure (5 levels, 4 perspectives, 16 areas) - Stable; the month changed doctrine and evidence, not the shape of maturity.

Stripe Minions as L5 north star - Still the throughput reference; Bun-in-Rust joins it as the rewrite reference.

MicroVM sandboxes as the runtime baseline - Unchanged as baseline - what changed is the doctrine around them (assume escape).

Multi-agent fleets as product default - Confirmed; July added collision data and merge queues, not a new direction.

Skills / AGENTS.md / spec-driven development - Direction holds; Anthropic just simplified the guidance (less prompt, more trust).

Sources

OpenAI: Separating Signal from Noise

SWE-Bench Pro recommendation formally retracted (~30% of tasks broken)

Cursor: reward hacking audit

63% of resolutions retrieved, not derived; 87.1% → 73.0% strict harness

Building to the Test (arXiv)

Agents ship dead code passing a visible 222-test oracle

Hugging Face: July security incident

The first runaway agent - eval model breached production via sandbox zero-day

Willison: the first known runaway AI agent

Analysis of the ExploitGym → HF breach

Pillar: The Week of Sandbox Escapes

Escapes plant files trusted host tools later read - four failure modes

Claude Code v2.1.200: default Manual

Default permission mode flipped from Auto to Manual (July 3)

GitHub: enterprise managed settings

'Any client outside the policy is a gap' - policies cover app + cloud agent

MCP 2026-07-28 release candidate

Stateless core, OAuth/OIDC auth, deprecation policy - biggest revision ever

Azure DevOps MCP flaw

Hidden PR comments hijack AI review agents (official Microsoft server)

AWS Kiro CVE-2026-10591

Poisoned web page makes the agent rewrite its own MCP config

Addy Osmani: Own the Outer Loop

Quality, Verdict, Answerability - the human's floor

Addy Osmani: Software Factories, Light and Dark

Back pressure and comprehension debt

Pragmatic Engineer: What is loop engineering?

The legitimizing deep-dive, with the skepticism kept in

Lilian Weng: Harness Engineering

Recursive self-improvement reframed as harness improvement

Fowler: Harness Engineering formalized

agents.md under 200 lines; the discipline gets its name

Cherny: Steps of AI Adoption

Gated → AI-native; each step breaks an org bottleneck

Bun: Rewriting Bun in Rust

535k LOC in 11 days for $165K; TS suite as conformance harness

Anthropic: new rules of context engineering

80%+ of the system prompt deleted; auto-memory over manual curation

RIT: agentic PR review study

78.9% of agentic PRs pass a single reviewer; more review changed nothing

curl: Summer of Bliss

Vulnerability intake suspended five weeks over AI slop

Microsoft rollout study (arXiv)

+24% merged PRs for CLI-agent adopters, sustained over four months

Agent PR conflicts (arXiv)

Cross-vendor pairs conflict at 41.7% vs 19.8% intra-vendor

CNBC: Fable 5 export controls lifted

Restored July 1 after a three-week worldwide blackout

Gibson Dunn: EU AI Act omnibus

High-risk duties deferred to Dec 2027 / Aug 2028

CNBC: the layoff reversal wave

Ford, CBA, IBM reverse AI cuts; 55% of leaders call them a mistake

InfoQ: AWS Claude Apps Gateway

Self-hosted identity/policy/telemetry control plane for Claude agents

Malicious skills defeat scanners

Static skill scanning evaded >90% of the time (HKUST)

Explore August Edition Compare with July