On this page · 13 sections
- Rule 1: know what a minute costs before you argue about coverage
- Rule 2: the rounding rule shapes your job design
- Rule 3: treat flakiness as the primary failure mode, because it is
- Rule 4: automate the money paths first, not the easy paths
- Rule 5: put the pyramid back the right way up
- Rule 6: make the suite tell you why, not just that
- Rule 7: budget the suite like infrastructure, because it is
- What a suite costs to run: a worked comparison
- India-specific considerations
- The honest summary
- FAQ
- How eCorpIT can help
- References
Summary. GitHub reduced the net cost of GitHub-hosted runners by up to 39% on 1 January 2026, driven by a roughly 40% price cut across runner sizes paired with a new $0.002 per-minute Actions cloud platform charge folded into the meter. The separate self-hosted billing change was postponed. What did not change is the spread between machines: a Linux 2-core minute is $0.006, a Windows 2-core minute is $0.010, and a macOS minute is $0.062, which is over 10 times the Linux rate. GitHub also rounds every job up to the nearest whole minute, so 100 twenty-second jobs bill as 100 minutes rather than 34. Meanwhile the reliability problem is older and worse than the cost problem. Google's testing team reported that about 1.5% of all test runs produced a flaky result, almost 16% of its tests had some level of flakiness, and about 84% of observed pass-to-fail transitions involved a flaky test rather than a real bug. A regression suite that is slow, flaky and billed on the wrong machine is a tax. These are the seven rules we use to make one pay.
Rule 1: know what a minute costs before you argue about coverage
Coverage debates are cheaper to settle with a number. Here are the published rates from the GitHub Actions runner pricing reference, effective since the January 2026 change.
| Runner | Per-minute rate | Relative to Linux 2-core |
|---|---|---|
Linux 1-core (x64), actions_linux_slim |
$0.002 | 0.33x |
Linux 2-core (arm64), actions_linux_arm |
$0.005 | 0.83x |
Linux 2-core (x64), actions_linux |
$0.006 | 1x |
Windows 2-core (x64), actions_windows |
$0.010 | 1.67x |
macOS 3-core or 4-core, actions_macos |
$0.062 | 10.3x |
| Linux 8-core larger runner | $0.022 | 3.67x |
macOS 12-core, macos_l |
$0.077 | 12.8x |
Work an example. A suite that takes eight minutes on a Linux 2-core runner costs $0.048 per run. At 50 runs a day across a team, that is roughly $72 a month. The identical suite on a macOS runner is $0.496 per run, or about $744 a month. Nothing about the tests changed. The bill went up more than tenfold because of the machine underneath them.
That does not make macOS wrong. If you ship an iOS app you need macOS runners, and the arm64 macOS 5-core macos_xl at $0.102 or the 12-core macos_l at $0.077 may still be the right call because a faster machine can bill fewer minutes. It makes running your Node unit tests on macOS wrong.
Two constraints worth knowing before you plan around larger runners: they are only available to organisations and enterprises on GitHub Team or GitHub Enterprise Cloud, and included minutes cannot be used for larger runners.
Rule 2: the rounding rule shapes your job design
GitHub rounds the minutes and partial minutes each job uses up to the nearest whole minute. This single sentence in the pricing docs quietly invalidates a popular design.
Splitting a suite into many small parallel jobs feels efficient. It is, for wall-clock time. It is not, for billing. One hundred jobs that each take twenty seconds bill as one hundred whole minutes. The same work in ten jobs of about three and a half minutes bills as forty. Same tests, same machine, 60% less spend.
So the shape to aim for is a moderate number of jobs that each run for several minutes, not a swarm of tiny ones. Parallelise to hit your wall-clock target and then stop, rather than parallelising because the matrix syntax makes it easy.
Rule 3: treat flakiness as the primary failure mode, because it is
The most useful public data on this is still Google's, published on the Google Testing Blog by John Micco of Google's testing team in May 2016. The numbers are old and they remain the most honest ones available, because Google published them against its own corpus at a scale most organisations never reach.
Micco defines the term precisely: "We define a 'flaky' test result as a test that exhibits both a passing and a failing result with the same code."
Three findings matter.
About 1.5% of all test runs at Google reported a flaky result, and the rate was continual: Micco notes the insertion rate was about the same as the fix rate, so the organisation was "stuck with a certain rate of tests that provide value, but occasionally produce a flaky result". Almost 16% of Google's tests had some level of flakiness.
The third finding is the one that should change your behaviour. In post-submit testing, "about 84% of the transitions we observe from pass to fail involve a flaky test". Five out of six times a build goes red, it is noise.
Google's own arithmetic makes the cost concrete: an average project holds around 1,000 tests, and at a 1.5% flaky rate about 15 will fail on any given run, each needing investigation by a build cop or a developer. That is the tax. It is paid in attention, not dollars, which is why it rarely appears in a budget and always appears in the retro.
Micco is candid about the trap in the obvious mitigation. Marking a test flaky so it only reports failure after three consecutive failures reduces false positives, but "encourages developers to ignore flakiness in their own tests". His worked example: a 15-minute integration test marked flaky and genuinely broken by a submission is not discovered for three executions, or 45 minutes, and even then someone has to decide whether it broke or merely flaked three times running.
The rule that follows is unpopular and correct. A test that flakes is quarantined out of the blocking suite the same day, with a ticket, and either fixed or deleted inside a sprint. A suite where red does not mean stop is not a suite. It is a rumour.
Rule 4: automate the money paths first, not the easy paths
Teams automate what is easy to automate, which is why so many suites have exhaustive coverage of a settings page and none of checkout.
The ordering that works is by cost of failure, not by cost of automation.
| Area | Automate? | Reasoning |
|---|---|---|
| Payment, billing, subscription state | Always, first | A failure here loses money and trust in the same minute |
| Auth, permissions, tenant isolation | Always | A cross-tenant leak is an incident, not a bug |
| Core workflow the product is sold on | Always | If this breaks, the product is down regardless of uptime |
| Data export, deletion, retention | Yes | Regulatory exposure, and it is rarely exercised by hand |
| Admin and internal tooling | Rarely | Low blast radius, high churn, cheap to fix forward |
| Cosmetic UI states | No | High maintenance, near-zero failure cost |
| Third-party sandbox behaviour | No | You are testing their mock, not your code |
The last row causes arguments. A test that asserts a payment provider's sandbox returns a particular response is testing the sandbox. Test your handling of the responses instead, including the retry path, which is where agents and impatient users both live. We covered why that path needs an idempotency key in idempotency keys and safe retries for REST APIs.
Rule 5: put the pyramid back the right way up
The failure pattern is familiar: an end-to-end suite that grew because end-to-end tests are the ones that catch real bugs, until the suite takes 40 minutes, flakes constantly, and nobody trusts it.
End-to-end tests catch real bugs because they exercise real integration. They also flake because they exercise real integration: networks, timing, third-party sandboxes, browser rendering. Every source of non-determinism Micco lists, concurrency, non-deterministic behaviour, flaky third-party code and infrastructure problems, is more present at the top of the pyramid than the bottom.
Keep end-to-end coverage for the handful of journeys in rule 4. Push everything else down to integration and unit level, where it runs on a Linux 2-core at $0.006 a minute, finishes in seconds and does not flake because there is no network in it.
An eight-minute suite that people trust changes behaviour. A 40-minute suite that people override does not.
Rule 6: make the suite tell you why, not just that
A red build that says "failed" is a research project. A red build that says which assertion, on which input, with the request and response attached, is a five-minute fix.
Micco's 84% figure is what makes this economic rather than aesthetic. If most red builds are noise, the cost of every red build is the time to classify it. Anything that shortens classification, screenshots and traces on failure, retained artifacts, structured logs, the actual diff, pays back on every single red build, including the false ones.
This is also where the rounding rule from rule 2 and observability meet: retaining artifacts costs storage, not minutes, and storage is the cheaper of the two.
Rule 7: budget the suite like infrastructure, because it is
GitHub's own framing of the January change is that it is "charging fairly for Actions across the board". The company reported that 96% of customers saw no change to their bill, and of the 4% affected, 85% saw a decrease while the remaining 15% faced a median increase of around $13. It also noted that in 2025 developers used 11.5 billion Actions minutes on public projects for free, which GitHub valued at roughly $184 million.
The scale context is useful for anyone deciding whether hosted runners are a serious option. GitHub shipped Actions in 2018; by early 2024 the platform ran about 23 million jobs a day and the architecture could not keep up. After a re-architecture, all Actions jobs have run on the new platform since August, handling 71 million jobs a day, over three times where it started, with individual enterprises able to start seven times more jobs per minute than the old architecture supported.
One correction worth making, because it circulated widely and is out of date: the announced billing change for self-hosted runners was postponed. GitHub's December 2025 update says it is "postponing the announced billing change for self-hosted GitHub Actions to take time to re-evaluate our approach", while continuing the hosted-runner reduction of up to 39% on 1 January 2026. If you moved workloads off hosted runners in a hurry on the strength of the original announcement, the premise changed.
The practical budgeting move is to treat CI spend the way you treat cloud spend: measure it per team, attribute it, and review it. It responds to the same discipline, which we wrote about in cloud FinOps as a managed service.
What a suite costs to run: a worked comparison
Assume 400 tests, a suite that takes eight minutes on Linux 2-core, and 50 runs a day.
| Configuration | Per run | Per month (1,500 runs) |
|---|---|---|
| Linux 2-core, 1 job of 8 min | 8 min at $0.006 = $0.048 | $72 |
| Linux 2-core, 24 jobs of 20 sec each | 24 whole min at $0.006 = $0.144 | $216 |
| macOS 3-core, 1 job of 8 min | 8 min at $0.062 = $0.496 | $744 |
| Linux 8-core larger, 1 job of 4 min | 4 min at $0.022 = $0.088 | $132 |
| Linux 1-core slim, 1 job of 12 min | 12 min at $0.002 = $0.024 | $36 |
Read the second row again. The same 400 tests, on the same machine type, cost three times as much purely because they were split into two dozen short jobs and each one rounded up to a whole minute. That is the rounding rule doing all the work.
The last row is the one teams forget. If a suite tolerates a slower machine, actions_linux_slim at $0.002 a minute is a third of the standard Linux rate, and 12 minutes on slim is cheaper than 8 minutes on standard.
India-specific considerations
For teams building from India against global release trains, two things matter beyond the rate card.
The first is that CI cost is charged in dollars and salaries are paid in rupees, so the ratio between "add a machine" and "add an engineer" is different here than it is for a US team looking at the same pricing table. Throwing a larger runner at a slow suite is often the correct call, and the instinct to optimise engineer time over compute time is worth checking against the actual numbers rather than a US-shaped intuition.
The second is data. Test fixtures have a habit of being real customer data with the names left in. Under the Digital Personal Data Protection Act, 2023, the entity determining the purpose and means of processing personal data carries obligations including erasure on consent withdrawal and breach notification to the Data Protection Board of India, as Anirudh Burman's analysis for the Carnegie Endowment sets out. A production dump in a CI fixture is processing personal data on a machine you do not own, with logs retained by a third party. Synthesise fixtures. We design pipelines aligned with DPDP requirements; the compliance determination itself sits with the operator and their counsel.
The honest summary
A regression suite is not free and it was never meant to be. It is a bet that catching a defect before release costs less than catching it after. That bet pays when the suite is fast enough that people wait for it, trustworthy enough that red stops the train, and cheap enough that nobody proposes deleting it in a cost review.
Most suites fail the second condition first. Google, with the resources to solve almost anything, still reported 84% of pass-to-fail transitions as flaky and an insertion rate matching its fix rate. If that is the ceiling, the answer is not a heroic push to eliminate flakiness. It is a quarantine discipline that keeps the blocking suite clean while the flaky ones get fixed on their own schedule.
The most expensive test suite is the one everyone has learned to ignore.
FAQ
How eCorpIT can help
eCorpIT is a Gurugram-based technology organisation, founded in 2021, CMMI Level 5 appraised and MSME certified, with senior-led engineering teams and partnerships including AWS, Microsoft and Google. We build regression suites the way this article describes: money paths first, the pyramid the right way up, a quarantine discipline that keeps red meaningful, and job shapes chosen against the actual rate card rather than the default matrix. We will also tell you which of your existing tests to delete, which is usually the fastest win available. If your release train is blocked on a suite nobody trusts, talk to our team. Related reading: managed Kubernetes and AI platform services and application modernization from monolith to microservices.
References
- GitHub Docs, Actions runner pricing
- GitHub Docs, Billing and usage for GitHub Actions
- GitHub Docs, GitHub Actions billing
- GitHub Docs, Actions limits
- GitHub Docs, Larger runners
- GitHub, Actions pricing calculator
- GitHub Docs, View and download usage reports
- GitHub Docs, Estimate Actions costs
- John Micco, Google Testing Blog, Flaky Tests at Google and How We Mitigate Them
- GitHub Blog, Let's talk about GitHub Actions
- GitHub Roadmap, GitHub Scale Set Client
- GitHub Docs, Migrate from self-hosted to GitHub-hosted runners
- Anirudh Burman, Carnegie Endowment for International Peace, Understanding India's New Data Protection Law
Last updated: 16 July 2026.
