On this page · 11 sections
- The problem with flaky tests
- What changed in Xcode 27 and Swift 6.4
- What we saw when we ran it
- Using --repeat-until fail to reproduce a failure
- Using --repeat-until pass with care
- Which tests are repeated
- A known issue in Swift 6.4.0
- A practical debugging workflow
- What this feature does not solve
- How eCorpIT can help
- References
Summary. Swift 6.4, which ships in Xcode 27, adds two options to swift test: --repeat-until pass or --repeat-until fail, and --maximum-repetitions. Each Swift Testing test case then repeats on its own until it meets the condition or reaches the limit. The options work only for Swift Testing tests, not XCTest. The limit counts the first run. Without a limit, --repeat-until has no upper bound. In our own test run on Xcode 27.1, a test that failed twice and then passed still left swift test reporting failure. This guide explains how the options behave, what we saw and how to use them to chase flaky tests.
Flaky tests are hard to diagnose because a single passing run does not prove the behaviour is stable. Re-running a failing test by hand helps, but it is slow and hard to standardise in command-line and CI workflows.
About this article. This guide is based on Apple's Xcode 27 release notes, the SwiftPM 6.4 release notes, Swift Testing proposal ST-0024, Apple's WWDC26 session on what's new in Swift, and our own test run with Xcode 27.1, as checked on 17 October 2026. Researched with AI assistance and reviewed by our editorial team.
The problem with flaky tests
A deterministic test gives the same result when its inputs and environment are the same. A flaky test can give different results across runs even though nothing has intentionally changed. Tests are often sensitive to:
- asynchronous work and concurrency;
- timing;
- shared resources and file-system state;
- external services and networking;
- randomly generated data;
- other environmental conditions.
Repetition is a test execution control, not a new assertion mechanism. It helps you see intermittent behaviour more often; it does not explain it.
What changed in Xcode 27 and Swift 6.4
Apple lists the change in the Xcode 27 release notes under Swift Package Manager, New Features: swift test now supports repeating test cases until a condition is met, using --maximum-repetitions and --repeat-until with a value of pass or fail, and only the test cases that match the repeat condition are repeated. Xcode 27 includes Swift 6.4, and the feature comes from the open-source Swift 6.4 toolchain: it was specified in Swift Testing proposal ST-0024 and is carried out by Swift Testing's own runner.
The command looks like this:
swift test --repeat-until pass --maximum-repetitions 10
swift test --repeat-until fail --maximum-repetitions 100
| Option | What it does |
|---|---|
--repeat-until pass |
A failing test case repeats until it passes or reaches the limit; passing cases run once |
--repeat-until fail |
A passing test case repeats until it fails or reaches the limit |
--maximum-repetitions N |
Sets the most times each case runs, counting the first run |
--maximum-repetitions N on its own |
Runs every test case exactly N times |
--repeat-until with no maximum |
No upper limit: cases repeat until they meet the condition |
The SwiftPM 6.4 release notes describe the limit and the behaviour of each condition. Apple's WWDC26 session on what's new in Swift adds that when you repeat until the tests pass, only failing tests are re-run.
Swift Testing only
The help text for both options says they are supported only for Swift Testing test suites. XCTest test methods in the same swift test run are not repeated. If your package still has XCTest tests, migrate the flaky ones to Swift Testing before you rely on these options; Apple's WWDC26 session on migrating to Swift Testing covers how.
Swift Testing itself has been available since Xcode 16. There is no new @Test trait or attribute. ST-0024 lists a repetition trait only as a possible future direction, so repetition today is chosen on the command line.
What we saw when we ran it
We built a small Swift package with three Swift Testing tests and ran it with Xcode 27.1 (build 27A9275), whose toolchain reports Swift 6.4. One test failed on its first two runs and passed on the third, one always passed, and one failed only on its third run. Each test counted how many times it ran.
| Command | Fails twice, then passes | Always passes | Fails on third run | Exit status |
|---|---|---|---|---|
swift test |
1 run | 1 run | 1 run | Failure |
--repeat-until pass --maximum-repetitions 5 |
3 runs | 1 run | 1 run | Failure |
--repeat-until fail --maximum-repetitions 5 |
1 run | 5 runs | 3 runs | Failure |
--maximum-repetitions 3 |
3 runs | 3 runs | 3 runs | Failure |
--filter alwaysPasses --repeat-until fail --maximum-repetitions 4 |
Not run | 4 runs | Not run | Success |
Four things stand out:
- Each case repeats on its own. In the fail run, the test that failed first stopped at once while the others kept going.
- The limit includes the first run. A maximum of 5 meant five runs in total, not one run plus five repeats.
- A pass after retries is still a failure. With
--repeat-until pass, the flaky test passed on its third run, butswift teststill exited with failure because the earlier runs had failed. Do not use it to turn a red CI build green.
- Fail mode repeats your healthy tests. Every passing test runs up to the limit, so narrow the run with
--filter.
Using --repeat-until fail to reproduce a failure
This is the more useful mode for investigation. Point it at the suspect test and give it a generous limit:
swift test --filter LoginTests --repeat-until fail --maximum-repetitions 100
Each matching test case keeps running until it fails or reaches 100 runs. Without --filter, every passing test in the package repeats up to the limit, which can turn a quick suite into a long one.
Once the failure appears, investigate the usual suspects:
- concurrency behaviour and shared mutable state;
- timing assumptions and asynchronous completion;
- resource clean-up between tests;
- test ordering;
- environment-dependent behaviour.
The repetition turns a hard manual reproduction into a repeatable command. It does not find the root cause.
Using --repeat-until pass with care
swift test --repeat-until pass --maximum-repetitions 10
This re-runs only failing test cases and shows whether a failure is intermittent. Two cautions apply. A test that eventually passes still fails the overall run, as our results show. And a pass after a retry is evidence of instability, not proof the test is healthy:
Pass after retry
≠
Problem solved
Which tests are repeated
Repetition works per test case, not per suite or per run. For a parameterised test, each argument is its own test case, so only the argument that misbehaves repeats. A Swift Testing suite like this one is what the options act on:
import Testing
@Suite struct LoginTests {
@Test func validLogin() async throws { /* ... */ }
@Test func invalidPassword() async throws { /* ... */ }
@Test func expiredSession() async throws { /* ... */ }
}
The same change reached Xcode's test plans. The Xcode 27 release notes say the Test Repetition Mode setting now repeats only individual Swift Testing test cases. Xcode and xcodebuild have long offered their own repetition, through the Run Repeatedly command and the -test-iterations and -run-tests-until-failure options; what is new in Swift 6.4 is the swift test path for packages.
A known issue in Swift 6.4.0
Swift Testing has a reported bug in which warnings and known issues are treated as failures by the repetition logic, so pass mode can repeat when it should not and fail mode can stop early. The fix was merged to the 6.4.1 and 6.4.2 release branches. The toolchain in Xcode 27.1 reports itself as Swift 6.4 with a 6.4.0 build number, and Apple's release notes do not name the patch version, so expect this behaviour until a later toolchain ships. If your tests record warnings or use withKnownIssue, check the results by hand.
A practical debugging workflow
- Run the tests normally. If a test fails every time, investigate that failure directly.
- Reproduce the flake. Run
swift test --filter <suspect> --repeat-until fail --maximum-repetitions 100.
- Capture the evidence. Keep the failure message, logs, the state of fixtures and any environment details.
- Make the test deterministic. Remove the timing, shared state or environmental dependency that caused it.
- Confirm the fix. Run the same repeat-until-fail command again and check it reaches the limit without failing.
What this feature does not solve
- It does not find the root cause. If a test fails on run 37, you still have to work out why.
- It does not make flaky tests reliable. It changes how often you see the behaviour, not its source.
- It does not replace assertions. Your expectations still decide whether a test passes.
- It does not mean every test should be retried. Retries add run time and can hide instability if used as a permanent workaround.
- It does not cover XCTest. Only Swift Testing tests repeat.
- It does not remove the need for isolation. Tests that depend on shared state or external services stay fragile until you fix the dependency.
Our Xcode 27 and Swift 6.4 overview covers the other changes in this release, and our guide to choosing a software testing partner covers test strategy more broadly.
How eCorpIT can help
eCorpIT is a Gurugram-based technology organisation, founded in 2021, assessed at CMMI Level 5 and MSME certified, with senior-led engineering teams working across AWS, Microsoft and Google platforms. Our iOS teams migrate test suites to Swift Testing, track down flaky tests and set up CI runs that use repetition for diagnosis without hiding failures. Talk to us at /contact-us/.
Last updated: 17 October 2026.
