The Evolution of Continuous Testing

Faster, stronger, more agile. Organizations are on a mission to streamline processes from operations to IT – and this includes big changes in the testing landscape. I recently had the opportunity to share my thoughts on the topic with TechBeacon’s Christopher Null in his story, “The State of Continuous Testing.” In this blog, I expand on the topic and share a few additional thoughts on how organizations can get it right. 

The Changing Testing Landscape

In the recent past, testing was a time-consuming phase that took place after development. To boost its effectiveness, organizations are shifting to a continuous testing (CT) approach that is an activity embedded in all phases of project development. Simultaneously, a DevOps approach is being leveraged with increasing frequency, taking advantage of its collaborative nature across stakeholder groups. Now, instead of continuous deployment and continuous integration taking place separately, the three practices are being conducted in sync.

Additionally, there is an uptake of approaches like Acceptance Test-Driven Development (ATDD), Specification by Example (SBE), and Business Driven Development (BDD), where analysis and testing go hand in hand. This further helps improve the communication amongst teams, reduce deadlocks and delays towards the later phases of the development process, and augment the multi-disciplinary skill set of the team members.

But, as always with new approaches, the transition has not been seamless. As I shared with TechBeacon, all too often organizations focus on the wrong thing when it comes to testing automation – particularly at the graphical user interface (GUI) level. While automating the process in the GUI itself can seem to be a point-and-click activity, GUI test automation is very expensive if not preceded by the correct set of automated unit and application programming interface (API) tests.

The challenge that organizations face is that these automated units and API tests should be written by a developer or someone with access to the source code. This requires a high level of test awareness and an open and blame-free team atmosphere. Automated unit tests are, regardless of all case studies and evidence, still perceived as time-consuming and lengthening the total development time. Properly written and executed unit tests save a tremendous amount of time in the later phases of the testing process since the stability of the software is already embedded in the process.

For organizations that want to take advantage of continuous testing, avoid these pitfalls, and reap its benefits, there are a few things to keep in mind:  

1) Collaboration and Coordination are Key. 

Ideally, continuous testing is a collaborative effort between developers, testers, and operations stakeholders. It should also be aligned with the scope and the priorities of business stakeholders.

With this in mind, we aim to align continuous testing with continuous integration and continuous delivery, each executed on different levels – unit tests, API/Command Line Interface (CLI) tests, integration tests, system tests, and acceptance tests – while covering the required aspects of functional and non-functional tests, such as maintainability testing, security testing, performance testing, and portability testing. Not all projects require such a rigorous approach, nor do all organizations have the capabilities on an organizational or technical level to cover each of those aspects. 

2) Assess, Tract, and Correct. 

At the outset of any size project, we perform an assessment to identify all obstacles that would prevent a smooth CT process – this, of course, takes into consideration the needs of the project and the business priorities. We also leverage lightweight project governance based on Kanban principles to detect hiccups and remediate issues immediately. This agile process allows our team to successfully initiate, remediate, set up, roll out, support, and run CT processes ranging from automated unit testing (ATDD) to a complete CT cycle in symbiosis with the DevOps process, covering the entire collection of ISO 25010 quality attributes. 

3) Develop an Understanding of Continous Testing. 

It is valuable for our clients and their stakeholders to understand the full CT process. This will help them to better align the software quality effort with the business expectations, to determine how and when to test and to pinpoint possible improvement areas. For example: when business stakeholders grasp the concepts of ATDD, they are able to prepare their test cases in the Gherkin language upfront. This increases the effectiveness and efficiency of the testing efforts, since every code commit or deployment can launch a collection of automated acceptance tests. To enable this, all stakeholders should be kept up-to-date with current methodologies, tool, standards, and approaches in order to leverage these strengths. This is why part of CTG’s approach includes training and coaching on all aspects of the process based on each stakeholder’s needs and skill level. 

Quality and reliability gaps in software can be time-consuming and costly to correct, particularly as software is increasingly integrated in operations. That’s why it’s important for modern organizations to implement continuous testing – and do it right the first time.

Visit CTG’s Quality Assurance and Testing Solutions page to learn more about the types of work we are doing for our clients.