Parallel testing is your ticket to faster testing and a quicker turn around in deployments. When testing websites or applications, it is important to remember that time is a factor- you always have a finite amount of time to test before deployments or to increase your coverage. Testing 100% of an application is a noble cause, but no developer wants to spend more time testing than developing their product. Parallel testing lets you get more testing done in a tighter window.
With Continuous Integration, testers and developers are asked to be constantly writing new test scripts for different features and test cases. These scripts take time to run, and the more test cases running against an increasing number of environments can spell doom for a deployment. Think that testing is only going to take 2 days? Wrong, your tests have been firing off for a week, and they still have a day left. So how do we speed up testing and do more QA’ing with less time between deployments? The answer is Parallel Testing.
Sequential Test Execution:
Parallel Test Execution:
Instead of running tests sequentially, or one after the other, parallel testing allows us to execute multiple tests at the same point in time across different environments or part of the code base. You can do this by setting up multiple VMs and other device infrastructure or by using a cloud test service like CrossBrowserTesting.
Parallel Environments
The growing number of devices and browsers your customers are using can be a challenge when trying to test quickly and efficiently. Let’s imagine a real world example: in release 3 of your product, you have 8 hours of sequential regression testing to perform before the team feels confident to deploy.
By release 5, this may be twice the amount of hours you’ll need to run your tests and as a bonus, your product is getting popular and is being used by more users on an increasing number of different devices. Before you were testing for only Chrome and FireFox, but now you see that you need Android and iOS devices, Safari and multiple versions of Internet Explorer. So you have 16 hours of tests and 10 different devices or browser to cover. This would take us 160 hours for complete test coverage before our deployment. With parallel testing environments, we can run our 16 hours of tests on 10 different devices at the same time, saving us 146 hours of testing time.
Parallel execution also has the distinct advantage of isolating test cases and runs to one specific OS or browser, allowing for the testers and developers to dedicate their meaningful resources of serious problems with cross-platform compatibility.
Parallel Testing In CBT
The maximum number of automated Selenium and Javascript unit tests (API initiated tests) that can be run concurrently is limited according to your subscription plan.
We understand the importance of running as many tests concurrently as possible to increase the velocity of your continuous integration flow. To support this, we do not place arbitrarily low constraints on the number of parallel tests.
A major advantage of our service is that we provide a combination of real operating systems and an extensive array of physical phones and tablets. While we do maintain a large number of mobile devices, they are not unlimited. Ensuring the load is spread across the configurations is important in providing availability for all our customers.
If you want a toolkit to help you, we like TestNG for Java, there is a GitHub repo below that you can look at to get started
Python Examples
NodeJs Examples
Java Examples
Ruby Examples
More information about TestNG can be found here: http://testng.org/doc/index.html