Introduction
In today’s DevOps age, Continuous Testing is a critical component for any company planning on releasing new software. Releasing new software or even a new version of an existing product is often a daunting task – it requires time, it’s risky and more often than not, is not tested before release. This last bit is crucial, because it helps the company understand if the software is working properly or not. And, isn’t that the main goal of every software? A bug-plagued product is of no worth to the company.
Business Problem
Studies show that a product company that releases a new version every quarter faces routine challenges such as:
- The product released had untested features and regression bugs, resulting in compromised customer experience.
- Many issues fixed in the development stage were delaying the move to production.
Many issues fixed in the development stage were delaying the move to production.
This is where Continuous Testing comes in! Continuous Integration and Testing helps in overcoming these problems.
Wikipedia defines Continuous Testing as, “Continuous testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate.”
With this article, we will describe how Continuous Testing can be implemented in a product engineering group.
Role of Continuous Testing
As enterprises continue to evolve their processes and practices – so they can reduce the time from the development to the production stage – Continuous Testing plays a key role in facilitating this.
Continuous Testing in its most advanced form is seamless integration of code repository, build, unit tests, functional tests, and deployment to stage / production server. However, organizations cannot reach the final stage in one go. They have to implement the process one stage at a time, to avoid disruption in operations during the implementation process.
Continuous Testing will also help save production costs, because a bug detected during the testing phase will prove to be less expensive to fix than one caught during the production phase.
With Continuous Testing, a software company can avail these following key benefits:
- Early discovery of bugs and defects with the software.
- Quick TAT, as processes are automated
- Improvement in the quality of the product
- Reduced production cost
- Increased time management
Recommended Approach
As we mentioned, for a seamless integration, it is important companies follow a recommended progression to ensure they can reap the benefits of Continuous Testing. With that in mind, below is a commended approach we would suggest, that will surely help companies improve their Continuous Testing process.
1. To improve the quality of the code, it is recommended that
- The source code repository (SVN / GIT) is integrated with code quality tools PMD and FindBugs to ensure that the source code maintains the technical quality as per the defined baseline.
2. To shorten the testing time for the monthly release, the following process and tools are recommended
- The code repository is integrated with a CI tool like Jenkins. A build job defined in Jenkins is triggered every time a new code is checked in or updated.
- The development build is automatically run through unit tests using Junit.
- Automate the testing of stable features.
- The automated tests – smoke tests and functionality tests – are run continuously using a tool like Jenkins.
- The issues are reported to the development team and a new build is generated as a result of code changes.
- The release is done once the new features are tested and product is stable
Current Status
In the product group, the process of continuous testing is a combination of manual and automated tests. It is implemented as follows:
- The test sets are divided into Smoke Tests Set and Functionality Tests
- The Functionality Tests are grouped into different batches
- The functionality tests batches are scheduled to run alternate day
- The smoke tests are scheduled to run every day.
- The tests run continuously and on the test environment.
The tests sets are basically covering the regression tests and will make sure that there are no regression bugs when a new release is planned. For testing new features for every release, manual testing approach is adopted.
The product group has reduced the regression test time by 2 weeks for every release. It has also removed the manual effort for regression testing by using Selenium test automation tool.
Conclusion
To sum up, for continuous delivery to be a success, companies need to set up a favorable environment where the Continuous Testing model can work. Continuous Testing has been a part of the development process for years now, but with growth in technology and its subsequent evolution, Continuous Testing has come to the forefront and its importance is being recognized. The process itself has also evolved to suit the changing requirements. Hence, it is imperative that Continuous Testing be implemented diligently in software companies. Why? Because “All code is guilty, until proven innocent.”