“Relevance Lab celebrates successful completion of 5 years since inception. It is a real proud moment since the start in 2011 with a handful of passionate team members and a dream to pursue a journey. With a presence in US, India and Singapore the company now has a global footprint, a prestigious customer base, about 300 employees, key partnerships and strong investors. The opportunity of disruption with Cloud, DevOps and Automation is rapidly shaking the 100+ B$ market of incumbents and creating opportunities for niche Software-Defined companies. Relevance Lab with its unique positioning of Product + Services offers platform driven approach with significant advantages against traditional headcount driven services. The key focus on Automation, Optimization and Intelligent Systems is helping win large and transformational deals with a very healthy foundation. The company is focused for next 5 years to emerge as a dominant player helping enterprises transforming their Application Management, Infrastructure Management and Service Operations leveraging RLCatalyst, Cloud (IAAS, PAAS, SAAS Models) and Automation.”
When multiple teams and resources are involved in development and operations, getting a clear cut picture on what is happening where is not easy. Though there are many tools like JIRA, Jenkins etc that gives its own unique dashboard, there is no single consolidated dashboard that gives the end-to-end delivery pipeline view. Collecting DevOps statistics from various tools is a challenge and Catalyst has solved this using its dashboard framework, that gives a 360 degree view of development, QA, Continuous Integration and deployment.
The framework has a highly flexible and configurable GUI, where widgets can be configured for different data sources/tools. The default view has multiple widgets for a team to get the 360 view of development and operations
- Project management – JIRA
- Commit – Github, Bitbucket and SVN
- Build – Jenkins
- Quality – Sonarqube
- Deployment – Octopus, XLDeploy and Urbancode
Each one of these widget support multiple sources and one can configure any source according to the project needs. Dashboards can be configured at 2 levels
Team level dashboards on story management, code commits, builds and deployment
- Product or program view
Rolled up view for the product that involves multiple teams – shows flow of commit from dev to prod environments
How it works
The framework runs collectors for each data source that collects and stores data from each source at a predefined interval. The aggregation layer will read these data, convert into meaningful information and send to the GUI layer and charts through the REST API calls.
Once the teams dashboards are configured in the widgets, the data is automatically collected in the back-end and the display gets refreshed every minute.
The JIRA dashboard, the team can see the features that are being developed in the running sprint – The working progress and features that are done
The code repo shows coding activities – who has committed, the trends
The Build dashboard shows the details of last 5 builds, the status of each one of these and the trends
The Quality dashboard shows the code coverage, unit tests, security analysis etc.
Deployment dashboard shows the deployment details per environment.
The framework can be extended to support any type of DevOps tools
The product dashboard gives a roll-up view of the entire product that may include multiple components of each – feature management, code repository, jobs etc. It shows the flow of commits from development to testing to production stages as a pipeline view with appropriate details of unit and functional testing. It also gives the trends on product health and performance.
“Relevance Lab has partnered with ServiceNow to develop an Automation platform for traditional ITSM/ITIL Managed Services Programs. The evolution of “Software-Defined Business” is driving momentum for Automation, Cloud Adoption and DevOps. Automation targets traditional workloads in Infrastructure, Applications and Service Delivery (ITSM/ITIL). Focus is shifting more to Service Delivery Automation as current enterprises have a large volume of Support Tickets and complex setups supported by large number of people with issues on diagnosis, response time, quality and growing complexity. There is need to leverage Orchestration, Cloud Management System and Intelligent Monitoring to bring significant efficiencies and cost optimization in traditional Operations. RL Catalyst is being closely integrated with ServiceNow based Incident/Problem Management, Service Catalogs, CMDB Sync, Auto-Ticket Logging + Remediation and Hybrid Cloud Management. Working with large enterprises Relevance Lab is enabling 30% cost savings leveraging Automation and Cloud Solutions in traditional Infrastructure Management Services (IMS) and Application Management Services (AMS) deals. The RL Catalyst led Service Delivery Automation Platform will soon be made available on ServiceNow Store.”
In today’s DevOps age, Continuous Testing is a critical component for any company planning on releasing new software. Releasing new software or even a new version of an existing product is often a daunting task – it requires time, it’s risky and more often than not, is not tested before release. This last bit is crucial, because it helps the company understand if the software is working properly or not. And, isn’t that the main goal of every software? A bug-plagued product is of no worth to the company.
The Business Problem:
Studies show that a product company that releases a new version every quarter faces routine challenges such as:
- The product released had untested features and regression bugs, resulting in compromised customer experience.
- Many issues fixed in the development stage were delaying the move to production.
Many issues fixed in the development stage were delaying the move to production.
This is where Continuous Testing comes in! Continuous Integration and Testing helps in overcoming these problems.
Wikipedia defines Continuous Testing as, “Continuous testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate.”
With this article, we will describe how Continuous Testing can be implemented in a product engineering group.
Role of Continuous Testing:
As enterprises continue to evolve their processes and practices – so they can reduce the time from the development to the production stage – Continuous Testing plays a key role in facilitating this.
Continuous Testing in its most advanced form is seamless integration of code repository, build, unit tests, functional tests, and deployment to stage / production server. However, organizations cannot reach the final stage in one go. They have to implement the process one stage at a time, to avoid disruption in operations during the implementation process.
Continuous Testing will also help save production costs, because a bug detected during the testing phase will prove to be less expensive to fix than one caught during the production phase.
With Continuous Testing, a software company can avail these following key benefits:
- Early discovery of bugs and defects with the software.
- Quick TAT, as processes are automated
- Improvement in the quality of the product
- Reduced production cost
- Increased time management
Recommended Approach to Continuous Testing:
As we mentioned, for a seamless integration, it is important companies follow a recommended progression to ensure they can reap the benefits of Continuous Testing. With that in mind, below is a commended approach we would suggest, that will surely help companies improve their Continuous Testing process.
1. To improve the quality of the code, it is recommended that
- The source code repository (SVN / GIT) is integrated with code quality tools PMD and FindBugs to ensure that the source code maintains the technical quality as per the defined baseline.
2. To shorten the testing time for the monthly release, the following process and tools are recommended
- The code repository is integrated with a CI tool like Jenkins. A build job defined in Jenkins is triggered every time a new code is checked in or updated.
- The development build is automatically run through unit tests using Junit.
- Automate the testing of stable features.
- The automated tests – smoke tests and functionality tests – are run continuously using a tool like Jenkins.
- The issues are reported to the development team and a new build is generated as a result of code changes.
- The release is done once the new features are tested and product is stable
The above set of steps is repeated for every release.
Current Status of Continuous Testing:
In the product group, the process of continuous testing is a combination of manual and automated tests. It is implemented as follows:
- The test sets are divided into Smoke Tests Set and Functionality Tests
- The Functionality Tests are grouped into different batches
- The functionality tests batches are scheduled to run alternate day
- The smoke tests are scheduled to run every day.
- The tests run continuously and on the test environment.
The tests sets are basically covering the regression tests and will make sure that there are no regression bugs when a new release is planned. For testing new features for every release, manual testing approach is adopted.
The product group has reduced the regression test time by 2 weeks for every release. It has also removed the manual effort for regression testing by using Selenium test automation tool.
To sum up, for continuous delivery to be a success, companies need to set up a favorable environment where the Continuous Testing model can work. Continuous Testing has been a part of the development process for years now, but with growth in technology and its subsequent evolution, Continuous Testing has come to the forefront and its importance is being recognized. The process itself has also evolved to suit the changing requirements. Hence, it is imperative that Continuous Testing be implemented diligently in software companies. Why? Because “All code is guilty, until proven innocent.”
How important do you think Continuous Testing is? Would you recommend it or do you think software companies can afford to give it a miss? Comment below and let us know your thoughts.