Anthropic's CI hit 25x: your Azure Pipelines cost math

Anthropic's engineering team published a number this week that belongs in every platform team's planning document: CI jobs grew 25x over six months. Not traffic, not storage, not model spend. Continuous integration jobs. The cause is stated plainly in the same post: Anthropic engineers now ship 8x as much code per quarter as they did between 2021 and 2025, Claude authors 80% of that code, and test volume across the codebase grew 10x. Agentic coding does not just change who writes the code. It changes the shape of the load hitting the system that verifies it.

Most Swedish and EU engineering teams are somewhere earlier on that curve, running Claude Code, Copilot or Codex against an Azure DevOps or GitHub Enterprise estate. The useful question is not whether your CI job count will rise. It is which line on your bill moves when it does, and whether the architecture underneath survives the move. Those two answers differ sharply depending on whether you run Azure Pipelines or GitHub Actions, because the two platforms bill fundamentally different things.

What actually broke at Anthropic

The service under strain was test impact analysis: rather than running every test on every change, a selector decides which tests are relevant to a given pull request, based on historical test results and package relevance. It is the standard defence against a test suite that outgrows the time developers will wait. Anthropic's implementation had two parts. A listener recorded results from every CI run. A selector read that history to decide what to run next.

The listener was a single process with a single writer. That detail is the whole story. It could not scale horizontally, so when job volume climbed the listener fell behind the stream of incoming results, and a lagging listener produces failure modes that are worse than a slow pipeline. Bad changes merged because the tests that would have caught them were never selected. Flaky tests blocked merges they should not have blocked. New and newly-fixed tests did not run at all, so regressions slipped through in exactly the area where someone had just done the work to prevent them.

The sequence of mitigations is the part worth pinning to a wall. Anthropic reports that moving the listener to a bigger machine bought 70 days of relief. Sharding per package bought 29 days. Daily restarts bought less than one day. Each fix was cheaper than a rewrite and each one bought less time than the one before, which is what exponential load does to vertical scaling.

The rewrite replaced the singleton with stateless listener workers, so any worker can process any result, backed by an in-memory data store with journal-based event logging and a separate consumer that rolls up per-test history every few seconds. One engineer delivered it in three weeks, against roughly a quarter for the original. The team's recommendation to everyone else is a planning assumption rather than an architecture: assume your architecture will be at 25x load within two quarters, and design for 10 to 20 times the scale you think you need.

Two billing models, two very different 25x outcomes

GitHub Actions bills minutes. Azure Pipelines bills concurrency. Under flat load the distinction is an accounting curiosity. Under agent-driven load it decides whether a 25x job increase produces a 25x bill increase or something closer to flat.

GitHub Actions: the bill tracks job count

GitHub-hosted runners are charged per minute against a monthly included allowance. Current list rates and allowances:

Included minutes per month are 2,000 on Free, 3,000 on Pro and Team, and 50,000 on Enterprise Cloud. Public repositories remain free.

Now apply the Anthropic multiplier to a mid-sized team on Enterprise Cloud. Assume 2,000 CI jobs a month averaging six minutes on Linux 2-core x64 runners. That is 12,000 minutes, comfortably inside the 50,000-minute allowance, so the Actions line on the invoice reads zero and nobody has ever needed to think about it. At 25x, the same team runs 50,000 jobs and 300,000 minutes. Subtract the allowance and 250,000 billable minutes at $0.006 is $1,500 a month, or $18,000 a year. On Windows runners at $0.010 the same workload is $2,500 a month.

The jump from zero to a five-figure annual line item is what makes this a governance problem rather than a budget variance. Nobody set an alert on a number that was zero.

Azure Pipelines: the bill tracks peak queue depth

Azure DevOps sells parallel jobs, not minutes. A Microsoft-hosted parallel job costs $40 a month with no monthly minute limit and a 360-minute cap per job. A self-hosted parallel job costs $15 a month with no time limit at all, and you supply the compute. The free tier gives private projects one Microsoft-hosted job capped at 1,800 minutes a month and 60 minutes per run, which you must explicitly enable by linking an Azure subscription, plus one self-hosted job and one more for each active Visual Studio Enterprise subscriber in the organisation.

Because you are buying concurrency, the number you need to forecast is not total minutes but how many jobs want to run at the same moment. Little's law gives it directly: required concurrency equals arrival rate multiplied by mean job duration. Take the same 50,000 jobs of six minutes each. Across a full month of 43,800 minutes that averages under seven concurrent jobs, which sounds relaxed until you notice that CI does not arrive uniformly. Concentrate the same volume into roughly 160 working hours, or 9,600 minutes, and the working-hours average is about 31 concurrent jobs, with real peaks well above that.

That is the number that costs money, and it is also the number you can negotiate. Every minute of queue latency your developers will tolerate reduces the concurrency you must buy. Going from 5 to 12 Microsoft-hosted parallel jobs adds $280 a month and accepts longer queues at peak. Going from 5 to 31 adds $1,040 a month and keeps queues short. Neither number is right in the abstract, but you should know which one you are choosing.

Two operational details bite at this scale. Parallel jobs are purchased at organisation level and cannot be partitioned to a specific project or agent pool, so one team's agent-generated burst consumes capacity the next team was relying on. And new Azure DevOps organisations are capped at 25 Microsoft-hosted parallel jobs, with increases requiring a support request and subject to regional capacity. If your forecast crosses 25, open that request before you need it, not during the sprint where CI queues are already an incident.

Check where you actually are using the pool consumption report on the Analytics tab of your agent pool, which graphs running and queued jobs against your parallel job count for the previous 30 days. If the running line sits flat at the ceiling while the queued line climbs, you are already rationing.

Do not build the plan on "self-hosted is free"

The obvious response to per-minute billing is to move everything to self-hosted runners. That reasoning was tested in public over the last nine months and the outcome should temper it.

On 16 December 2025 GitHub announced a $0.002 per minute Actions cloud platform charge applying to all workflows, alongside a reduction in GitHub-hosted runner prices of up to 39% from 1 January 2026, with the new hosted rates already including the platform charge. The same announcement extended that per-minute charge to self-hosted runners from 1 March 2026. After substantial community pushback GitHub updated the post: "We're postponing the announced billing change for self-hosted GitHub Actions to take time to re-evaluate our approach." The hosted price reduction went ahead as planned.

So self-hosted Actions minutes are not billed today, and GitHub has not published what replaces the postponed model. A three-year infrastructure plan whose economics depend on that charge never arriving is a plan with an unhedged assumption in it. Model both cases. At 300,000 minutes a month, a reinstated $0.002 platform charge is $600 a month on top of the Azure compute you are already paying for the runners themselves.

Azure Pipelines has no equivalent exposure. Self-hosted agents are billed by concurrency at $15 per parallel job with unlimited minutes, and you can register any number of agents against those slots at no additional Azure DevOps cost.

Test selection is the lever that beats both bills

Buying capacity absorbs the load. Test impact analysis reduces it, which is why Anthropic invested in the service rather than simply provisioning more runners. If a pull request touching one package runs 4% of the suite instead of 100%, the minutes bill and the concurrency requirement both fall by roughly the same factor, and they fall before you spend anything on compute.

You do not need Anthropic's architecture to start. The cheap first version is a dependency-graph filter: map changed paths to owning packages, then run that package's tests plus its reverse dependencies, with a full suite on the merge queue and on a nightly schedule. In Azure Pipelines this is job-level path filtering plus a selection step:

jobs:
  - job: selected_tests
    pool: my-managed-pool
    steps:
      - script: |
          git fetch origin $(System.PullRequest.TargetBranch) --depth=1
          CHANGED=$(git diff --name-only FETCH_HEAD...HEAD)
          node tools/select-tests.js "$CHANGED" > tests.txt
        displayName: Select impacted tests
      - script: npm run test -- --listFile=tests.txt
        displayName: Run impacted tests

  - job: full_suite
    condition: or(eq(variables['Build.Reason'], 'Schedule'), eq(variables['Build.SourceBranch'], 'refs/heads/main'))
    pool: my-managed-pool
    steps:
      - script: npm run test
        displayName: Run full suite

The history-based version Anthropic built is a later step, and their design notes apply if you get there. Keep state out of the processes that ingest results, because a single-writer ingest service is the component that fails first. Instrument it well enough that an agent can watch the metrics and optimise incrementally. Avoid single-instance critical services you cannot measure.

One warning specific to agent-heavy teams. Test selection systems that learn from history assume the code changes slowly relative to the test history. When agents are authoring 80% of the code, history ages faster, and a selector tuned on last quarter's patterns will under-select on the packages that changed most. Keep the scheduled full run, and treat its failures as feedback on the selector rather than only on the code.

Managed DevOps Pools and the Sweden Central option

If the concurrency answer points to self-hosted capacity, Managed DevOps Pools is the current Azure path and it reached general availability with Sweden Central and Norway East among the supported regions. The agents run in a Microsoft-managed subscription rather than yours, which sits between fully hosted agents and running your own scale sets.

Three capabilities matter under agent-driven load. Pools can maintain agent state for up to seven days, so warm caches survive between runs, which directly cuts the minutes that dominate a per-minute bill. Jobs can run up to two days, against the 360-minute cap on paid Microsoft-hosted agents. And pools scale to thousands of simultaneous agents while letting you pin them to the region closest to your dependencies. The GA release also added Key Vault integration during provisioning, proxy support, visible agent IP addresses in the Initialize step for firewall debugging, and Ubuntu 24.04 images.

The Swedish and EU angle

CI is rarely in scope when Swedish organisations map data residency, and the 25x curve is a good reason to revisit that. Build agents check out your entire source tree, hold secrets in memory, and frequently touch production-shaped test data. A pipeline running on hosted agents in an unspecified region is a data flow, and multiplying its volume by 25 multiplies the number of times that flow occurs. Managed DevOps Pools in Sweden Central or Norway East gives you a defensible regional answer for the build tier without taking on scale set operations, and the visible agent IP makes it straightforward to document which egress points your CI actually uses.

For NIS2-scoped organisations, the Anthropic failure modes translate directly into control language. A lagging test selector meant bad changes merged and new tests did not run, which is a degradation of a change-control mechanism your risk function very likely believes is binary. If you rely on test selection anywhere in the path to production, the selector is part of your change-control evidence and it needs monitoring and an alert on ingest lag, not just a dashboard.

Procurement gets a cleaner story from the concurrency model. An Azure DevOps parallel job is a fixed monthly price for unlimited minutes, which produces a stable annual figure in an upphandling budget. A per-minute model produces a figure that moves with how much code your agents write next quarter, and few public-sector budget processes accommodate that gracefully. If you run both platforms, that difference alone is worth raising before the next framework renewal.

What to do this quarter

• 1. Measure the current slope. Pull CI job counts and total minutes for the last six months. The trend line matters more than the absolute number, because the planning assumption is 25x within two quarters.
• 2. Price your own 25x. On Actions, multiply billable minutes by your runner rate after the included allowance. On Pipelines, use arrival rate times mean job duration over working hours to get required concurrency, then multiply by $40 or $15.
• 3. Check the pool consumption report. If queued jobs are climbing while running jobs sit at the ceiling, you are already rationing CI and calling it normal.
• 4. Raise the 25-job cap early. New Azure DevOps organisations are limited to 25 Microsoft-hosted parallel jobs, and increases require a support request subject to regional capacity.
• 5. Ship path-based test selection. Start with the dependency-graph filter, keep a scheduled full suite, and only build history-based selection once the simple version is saturated.
• 6. Audit for single-writer services. Find every component in your CI path that runs as one process with one writer. That is where the 70-day, 29-day, one-day mitigation ladder begins.
• 7. Model the self-hosted charge both ways. GitHub postponed the $0.002 per minute self-hosted charge without replacing it. Budget for its return rather than assuming it is gone.
• 8. Put CI in the data-flow map. Decide the region for your build tier deliberately, and use Managed DevOps Pools in Sweden Central or Norway East if residency is part of the answer.

Conclusion

The 25x figure is the most concrete public evidence yet that agentic coding moves cost and architectural risk downstream, out of the model bill and into the verification tier. The cost half is arithmetic you can do this week from published list prices. The architectural half is harder, because vertical scaling will keep working just well enough to defer the rewrite, at 70 days, then 29, then one. Anthropic's engineer finished the redesign in three weeks. Finding the single-writer service in your own pipeline takes an afternoon.