Agent fleet on dedicated compute

An agent fleet on dedicated compute is the infrastructure pattern where AI agent workloads run on a distinct, purpose-built compute layer that is separate from developer laptops, CI infrastructure, and application workloads. Instead of agents sharing resources with other systems, there is a compute cluster specifically sized, configured, and optimized for running agent processes. The fleet is managed as a first-class infrastructure concern with its own capacity planning, monitoring, autoscaling, and operational runbooks.

At L5, agent workloads are substantial enough to warrant dedicated compute. A team running 50-100 simultaneous agent tasks, 24 hours a day, 7 days a week, needs compute that is always available, appropriately sized, and not subject to resource contention from unrelated workloads. Sharing this with CI (which has peak-and-trough load patterns) or with application servers (which have different resource profiles) creates interference that degrades agent reliability and response time.

The dedicated compute layer is typically Kubernetes-based, because Kubernetes provides the primitives needed for agent fleet management: pod isolation, resource quotas, namespace separation, and integration with autoscaling systems. Agent devboxes run as Kubernetes pods in a dedicated namespace, with node selectors that pin them to nodes in the agent node pool. The agent node pool uses instance types optimized for agent workloads: high memory (agents are memory-intensive), fast local NVMe storage (disk I/O is a key bottleneck for concurrent agent workloads), and high-bandwidth networking (large codebases move significant data).

Hardware is only half of what makes a fleet a fleet. The other half is the harness the fleet runs, and August 2026 is when that stopped being a monolith. DeepSeek made deepseek-harness public on 13 August under MIT, command name dsh, on an "everything is a plugin" design: the model adapter, the tool registry, the session log and the agent loop itself are all plugins, each replaceable independently. Shopify had made the underlying argument two weeks earlier, after five months of swapping models: the harness, not the model, is the durable asset. Put those together and the L5 maturity question is not how many nodes you run. It is whether you can swap the model without rewriting the harness - and, increasingly, whether you can swap the loop, the tool registry or the session store the same way when a better one appears.

This matters for a fleet specifically because a fleet amortises the harness across every task you run. If the model adapter is threaded through your scheduler, your cost attribution and your audit logging, then a pricing change at one vendor - and August had several, in both directions - becomes a migration project rather than a configuration change. A fleet built on a pluggable harness treats the model as an input to capacity planning, alongside instance type and storage class, rather than as an architectural commitment.

The dedicated fleet is also the level at which hardware specialization becomes relevant. Cursor's engineering team documented that disk I/O is the hidden bottleneck when running hundreds of agents: each agent reads large numbers of files, and concurrent file reads on a shared disk system create I/O contention that serializes what should be parallel work. Dedicated NVMe SSDs with high IOPS ratings, or a distributed filesystem optimized for concurrent reads, is the solution at this scale.

• No resource contention from unrelated workloads - CI pipelines, application servers, and agent workloads have different resource profiles; mixing them on shared compute creates unpredictable performance characteristics that are hard to diagnose and fix
• Compute can be optimized for agent-specific workloads - agent processes are memory-heavy, disk-read-heavy, and episodically network-intensive; dedicated compute uses instance types and storage configurations matched to this profile rather than to general-purpose workloads
• Capacity planning becomes straightforward - with dedicated agent compute, you can measure utilization, forecast growth, and make capacity decisions independently of other workloads; shared compute makes capacity planning for agents nearly impossible
• Fleet-level observability enables optimization - when all agent workloads run on dedicated compute, fleet-level metrics (tasks per node, disk IOPS per task, memory per task, network bandwidth per task) become meaningful for optimization; this data does not exist when agents are scattered across shared infrastructure
• A pluggable harness keeps model choice a routing decision - when the model adapter, tool registry, session log and agent loop are separable components, re-costing a workload after a price change is a config edit; when they are entangled with the fleet's scheduling and accounting, it is a migration
• Reliability becomes a compute infrastructure SLO - a dedicated fleet can have an SLO (e.g., 99.9% task start within 30 seconds, 99.5% task completion without infrastructure failure) that is distinct from application SLOs; this SLO commitment drives infrastructure investment and is auditable

1. Size the initial fleet based on observed agent load - Before deploying dedicated compute, measure current agent workload: peak simultaneous tasks, average task duration, resource consumption per task (CPU, memory, disk IOPS). Use these measurements to size the initial node pool. A starting point: N simultaneous tasks requires approximately N * (2 CPU, 4 GB RAM, 100 GB NVMe) at baseline resource limits.
2. Create a Kubernetes namespace for agent workloads - Isolate agent workloads from other Kubernetes workloads with a dedicated namespace and resource quotas. Set namespace-level limits on total CPU, memory, and storage that prevent agent workloads from consuming resources needed by application pods.
3. Deploy a dedicated node pool for agents - Create a Kubernetes node pool (in EKS, GKE, or AKS) with nodes that are tainted for agent workloads only. Use node selectors or taints/tolerations to ensure agent pods run only on agent nodes and non-agent pods do not run on agent nodes.
4. Choose storage infrastructure for disk-I/O-bound workloads - For each agent node, provision NVMe-backed instance storage (e.g., AWS i3en or i4i instances, GCP n2-highmem with local SSD) rather than network-attached storage. Or deploy a distributed filesystem (e.g., Ceph, Weka) optimized for concurrent read workloads. Measure IOPS per node and per task to confirm the storage configuration meets the demand.
5. Deploy the devbox manager on the agent cluster - The devbox lifecycle manager (pre-warmed pool, task assignment, environment destruction) should run on the agent cluster, not on a shared management cluster. This co-location minimizes the network distance between the devbox manager and the devboxes it manages.
6. Run the model-swap drill - Pick a workload the fleet runs daily and move it to a different provider's model end to end: adapter, tool registry, session log, cost attribution, evals. Time it. If the answer is "a sprint", the harness is the bottleneck, not the compute, and the fix is to put an interface where the coupling is. Repeat the drill each time a vendor changes pricing, because in August several did.
7. Establish fleet observability - Deploy Prometheus and Grafana on the agent cluster with dashboards for fleet-level metrics: nodes available, pods running, tasks per node, CPU/memory/disk utilization per node, task queue depth, and task completion rates. These dashboards are the operational interface for the fleet.

Under-sizing the initial fleet and then scaling reactively. An under-sized fleet that queues tasks trains developers to not trust the agent infrastructure. Start with more capacity than you think you need and scale down based on observed utilization. Over-provisioning by 50% for the first month is a reasonable approach.

Using network-attached storage when local NVMe is needed. Network-attached storage (EBS, Azure Disk, GCP persistent disk) has unpredictable IOPS under concurrent load. When 50 agents on the same node all read large codebases simultaneously, NAS IOPS limits become the bottleneck. Measure actual disk IOPS under your expected concurrent load before committing to a storage type.

Not separating the agent control plane from the agent data plane. The devbox manager, pool manager, and task queue are the control plane. The devboxes themselves are the data plane. Running them on the same nodes creates resource contention: the control plane components consume CPU and memory that the data plane needs for agent tasks. Use separate node pools for control and data plane components.

Treating the agent fleet as a cost center rather than a productivity center. Dedicated compute has a real cost. If the fleet is evaluated purely as an infrastructure expense, it will be undersized and underinvested. The correct evaluation is: how many developer-hours of productivity does the fleet enable? A fleet that costs $10,000/month and enables 500 developer-hours of agent productivity per month is delivering developer time at $20/hour - significantly cheaper than human labor.

Building the fleet around one vendor's harness. A fleet is a large, long-lived investment; the harness running on it changed shape twice in 2026 alone, and open, plugin-based harnesses now exist where even the agent loop is replaceable. If the scheduler, the cost attribution and the audit trail all assume one vendor's session format and one vendor's model API, the fleet inherits that vendor's roadmap and pricing. Define the seams - model adapter, tool registry, session log, loop - before the fleet is load-bearing, not after.

No cost attribution to teams or projects. When agent compute is shared infrastructure with no allocation, teams do not see the cost of their agent usage and have no incentive to optimize. Implement cost attribution at the team or project level using Kubernetes labels and cloud cost allocation tools. Teams that see their agent compute costs make better decisions about task efficiency and cleanup.