In Apple’s WWDC 2018 talk on What’s New in Testing, they announced a big new feature: parallel testing. Parallel testing runs your test suite on multiple simulators at once, in parallel, so it takes significantly less time to run. And all you have to do is check a box in Xcode 10.
If you don’t already have tests, you can write your first ones today with How to write unit tests in Swift 4. And if you want to take your unit testing further, check out Unit Testing in Swift.
Below are my notes from the talk, which introduced parallel testing, code coverage improvements, and test randomization.
Coverage
Smaller coverage files
Code coverage setting per target: all targets or some targets
xccov: new command line tool makes it easy to get coverage in CI
- output: human-readable or JSON
Coverage Data
- lives in Derived Data
Source Editor: Show/Hide code coverage
Demo:
- enable code coverage
- Report navigator: coverage report
- Edit scheme: gather coverage for some targets, add the targets you want
- Report navigator: report only shows the covered targets
- View coverage in editor to see what’s not covered
Xcode 10 features
- test selection: 1000 unit tests, 10 UI tests
Test selection with schemes: “tests to skip” in Xcode 9
Xcode 10 adds “tests to run”
Test ordering
Alphabetical
- Default
- Deterministic
- Can hide implicit dependencies
Implicit dependencies are bad and should be avoided
Each test should set up and tear down its own state
To ensure there are no implicit dependencies, Xcode 10 has randomization
- enable in scheme editor
Parallel Testing
Waiting for tests to finish is bad
Xcode 9 has parallel destination testing with only xcodebuild
Now, Xcode has that in the IDE
Classes execute in parallel on simulators
Demo
- See how long tests took to run in Report Navigator
- Edit Scheme > Test action > Options > Parallel
- Xcode launches multiple copies of Mac app to test in parallel
- 14s down to 5s total execution time
- iOS: multiple clones of the simulator (iPhone X)
- Each runner executes a different test class in the suite
- Xcode: Report Navigator: Test Log shows runners and class that’s executing
Demo Recap
- enabling parallelization
- view results in test log and report
- Instances of Mac app running tests
- Instances of iOS Simulator running tests
xcodebuild
- Override number of workers
- force parallel testing on or off
Tips and Tricks
- split long-running test class into two classes (testing will never run faster that the slowest class)
- Put performance tests into own bundle with parallelization disabled
- Understand which tests are not safe for parallelization (if they access shared resource like file or database)
- More info in Testing Tips & Tricks talk
Summary
- Code coverage with xccov
- Test selection and ordering
- Parallel testing
Have a great day!