iOS Mobile Testing: Parallel Execution with Fastlane

How to reduce test execution time by running in parallel on physical devices and simulators? How to generate test reports with screenshots and run tests via a command-line interface?

Getting started

Fastlane is the easiest way to automate beta deployments and releases for your iOS and Android apps. 🚀 It handles all tedious tasks, like generating screenshots, dealing with code signing, and releasing your application.

Fastlane for iOS

fastlane can be installed in multiple ways. The preferred method is with Bundler. fastlane can also be installed directly through Homebrew (if on macOS). It is possible to use macOS’s system Ruby, but it’s not recommended, as it can be hard to manage dependencies, and causes conflicts.

Please follow the steps mentioned on the official site of fastlane

Now we have Fastlane tools installed. You should have fastlane directory in your project with Appfile and Fastfile created.

Executing tests via command line

In order to run tests on the Continuous Integration Server, we need to execute them via a command-line interface. There are multiple approaches for this task, in this tutorial we’ll be using the Fastlane Scan action. Let’s set it up:

  • Navigate to created fastlane directory cd fastlane/
  • Open Fastfile in text editor
  • Set scheme according to your project scheme name
  • Specify test device
lane :uiTest do
scan(
scheme: 'ExampleFastLaneiOSUITests', # Project scheme name
clean: true, # Clean project folder before test execution
device: 'iPhone 8 plus' # Simulator for testing
)
end

Well done, we have successfully configured our project to execute tests and Fastlane tools will do the rest for us. To run tests:

  • Execute fastlane uiTest
  • Wait for tests to be executed
Test execution in Simulator

You will find HTML and Junit reports generated in fastlane/test_output directory.

Improving Test Reports

Fastlane allows generating only simple test reports which do not include screenshots and device logs. When building the Test Automation framework we need to make sure, that the test report has enough information, so we don’t have to re-run tests manually and can analyze test failures more efficiently

The better approach is to use XCTestHTMLReport.

When we have it successfully installed the next step will be to add a helper method, so we can generate a report as a part of the test:

desc "Generate test reports"
def generate_report
puts "Generating Test Report ..."
sh 'xchtmlreport -r test_output/ExampleFastLaneiOSUITests.xcresult -i'
puts "Test Report Succesfully generated"
end

Running tests in parallel on multiple devices

When testing Mobile Applications we need to verify that the Application works as expected on all supported devices and OS versions. Fortunately, we can automate this part with minimal effort. Parallel testing on simulators can be configured in Fastfile, we need to specify devices for testing

TEST_SIMULATORS = ['iPhone 8','iPhone SE (3rd generation)','iPad mini (6th generation)']
lane :uiTest do
scan(
scheme: 'ExampleFastLaneiOSUITests', # Project scheme name
clean: true, # clean project folder before test execution
devices: TEST_SIMULATORS
)
end
Parallel Test Execution on Simulators

Unfortunately, testing on simulators won’t be always the best approach, cause some issues won’t be appearing. Testing on real devices will provide us with more accurate test results. Also, we need to disable Auto-Lock so that the device will always be ready for test execution:

  • Navigate to the Display & Brightness section
  • Set Auto-Lock to Never

To make sure, that system notification won’t affect test execution we need to enable Don’t disturb mode :

  • Navigate to the Do Not Disturb section
  • Enable Do Not Disturb mode
  • Set Silence to always

The device is configured for test execution now. To execute tests on real devices we need to specify UDID. You can get it in multiple ways, for example, this. Then we just need to set the physical device as the destination for testing.

REAL_DEVICES = [
'platform=iOS,id=2a31ef65jc84c16er657af3f29901c20917g37',
'platform=iOS,id=78a91ef5bf2036fa49ec3df1af356jh5676390'
]private_lane :real_device_test do
scan(
scheme: 'ExampleFastLaneiOSUITests', # Project scheme name
clean: true, # Clean project folder before test execution
destination: REAL_DEVICES, # Devices for testing
result_bundle: "TestResults" # To generate test reports
)
generate_report
end

Tests will be executed on configured devices and a test report will be captured for each device.

Notes:

Sometimes you will need to add the following additional parameters

  • fail_build: false # Otherwise following steps won’t be executed
  • disable_concurrent_testing: true # to stop parallel execution and enable sequential testing order
  • testplan: ‘’ #define the test plan you wish to execute (eg: smoke, regression, etc…)

Check out the list of all available parameters

Generated HTML report will have test results for each device and test logs with screenshots.

Test Report example

Conclusion

In this article, we have learned how to set up Fastlane for your iOS project and execute Xcode UI Tests in parallel on physical devices and simulators.

It will allow us to set up our test execution on a Continuous Integration server and attach advanced test reports, as well as increasing of test coverage and reduce manual compatibility testing effort. Complete Test Framework setup example with Application Source Code is available on GitHub.

If you need to build an advanced fastline pipeline for deployment, please refer “How to build the perfect fastlane pipeline for iOS

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Dilshan Fernando

Quality Engineering | Test Automation Engineer | AWS Certified Solutions Architect | Problem Solver