Testing in software is often oversimplified as an activity performed with the only purpose of finding bugs. However, there are plenty of other activities involved. Here we go through some of them.
In software development, it is common to encounter some myths and misconceptions about what software testing means. Sometimes, it is oversimplified as an activity performed with the sole purpose of finding bugs. Another typical view is that it’s the process of verifying that software complies with its specification.
How biased can we be when considering either of these two concepts? How far can we be from reality? In this article, we will enrich these classical approaches by going through some of the many other activities of the testing team in order to agree upon an existing, more complete definition of software testing.
The Magic Word Is Quality
In the words of an American computer scientist, Gerald Weinberg, “Quality is value to some person.” I would not get much value from a computer program that crashes continuously, so it is in the best interest of the person who is providing me with such software to prevent it from happening. In order to avoid undesired behavior reaching end users, software testers dig into the program to find bugs that will later be fixed, if deemed appropriate.
Conformance to specifications follows a similar path. I would not get much value from a computer program if I particularly specified the need for it to work with certain file formats but the application does not handle them properly. To prevent this, the testing team will use the program to check if specifications are met.
In both examples, the software testing team is doing the same thing: It is gathering information about the quality of the software (i.e., about the value it provides or fails to provide to its different stakeholders). Indeed, this will be the goal of the testing team. However, there are other activities equally important as those already mentioned that will contribute valuable information about the quality of the software and the project as well. Let’s see some of the most important.
Finding Bugs (That Matter)
Among the principles of testing, there is one that goes like this, “absence of errors is a fallacy.” This principle is somewhat derived from other principles that state that testing is not exhaustive and that testing shows the presence of bugs, not their absence. Wrapping this up, no software can be assumed to be bug-free, no matter how much testing gets invested in it.
Although it may sound appalling at first, the absence of errors fallacy reveals an essential aspect of software testing. Testers need to not just find any bugs, but rather, find the important bugs, those that matter (and fast!). In order to build this criterion, the testing team needs to learn about the value that the software will give to stakeholders in a given context. There are many aspects considered in this task that will result in assigning the proper priorities to functional and non-functional testing. The learning process in which software testers gain knowledge about the project’s priorities and underlying business operations can be considered as important as the tester’s ability to find bugs.
Another aspect of software testing that is usually missed is reproducing bugs that, unfortunately, passed onto a later stage in development. The worst-case scenario is having them in production when clients are already using the software. In light of such an event, software testers are in charge of reproducing the undesired behavior in the testing environment so that the root cause of the issue can be identified and a fix can be provided. Testers, as technical researchers of a product’s quality, can give a deep insight into the problem and also be able to generate test data faster.
Building Consensus on Specifications
Specifications provide immense value within a software development project. They will serve as the basis for development, test cases, acceptance by the client, and even contractual litigations.
As software testers are in charge of informing about the quality of the project, they learn about its stakeholders and key actors. This knowledge allows testers to look at requirements specifications with a critical eye instead of blindly accept them as the truth.
The analysis of specifications as an activity prior to writing tests is fundamental to the project and is performed by software testers as well. They will question explicit specifications, such as documented requirements, making sure that they do not contradict the existing behavior of the software and that ambiguity is cleared out for all stakeholders. This task is essential to get a clear picture of what stakeholders view as a value in the software the team is developing and that the progress being made on development is not affecting the value already provided to end-users.
Additionally, software testers are involved in the very subtle but not less important task of reviewing implicit specifications. These are broad in scope and can range from the content of meetings with clients to those of a help manual of a past version of the software. Expectations about the value that the software will provide are laid out in these implicit specifications, and it is the testing team’s role not to miss any of them.
Software testing at the specifications’ definition stage is of great benefit. As the cost of changing software increases exponentially over the different stages of the project, having testers identify missing, changing, incoherent, incomplete, or contradictory specifications will save time, money, and heated discussions. That is why it makes sense for the testing team to be involved in projects from the very beginning.
Take a look at this: Software testing best practices and tools to avoid bad testing
Respecting and Improving the Development Process
Software testing is an information-consuming and information-generating activity. The input used by testers needs to be up to date, easily traceable, available to all stakeholders, and comply with other requirements for it to be legitimate within the project. The same requirements apply to information generated by the testing activities. The value of the project’s documentation is proportional to the value obtained by the software whose development was guided by it. Because of this, software testers are obliged to raise concerns whenever the established processes for information exchange are not respected.
Part of the information generated by the testing team can turn into metrics, indicators of how the testing tasks are performed and how the project looks from a quality point of view. Apart from providing them to stakeholders like project managers and clients, the testing team works in the interpretation of these metrics and the continuous improvement of the testing process as measured by these.
Getting the Software Accepted
Rather than just making sure that software complies with specifications, software testers have an active role in getting the software accepted by the stakeholders. The whole point of the acceptance testing level is this. However, instead of just writing and executing acceptance tests, testers accompany the deliverables into the stakeholders’ arms in two ways: showcasing the work done and assisting User Acceptance Testing (UAT).
The first of these tasks include preparing demonstration meetings (demo meetings, for example) in which a just finished work, or even work in progress, gets validated by stakeholders. This entails preparing an environment with the necessary test data to demonstrate how a new feature within the software works and is usually aimed at showing how the business need is fulfilled by this addition. By doing this, the development team can get valuable feedback early on. It also helps to build trust in the product from the stakeholders that can assess the value obtained in the software before being released.
When the software is finally delivered to clients, UAT is usually performed. These are tests run by end-users to accept the provided software and validate it for its use in production. During this phase, software testers assist in the form of generation of test data, aiding with configuration and environment set up, explaining the interactions with current workflows, and mainly helping users by providing any information they require to fulfill their tests.
Supporting Decision Making
Certain decisions on a software development project will depend on the quality of the system under test and the cost of testing as well. Testers will help managers measure the progress of the project and provide information about when a next release can be delivered. They also show if it is safe to deliver it at a certain point in time. Also, it is the tester’s responsibility to inform about the dangers of prematurely delivering a release that is not fit, backing their reasons with credible and well-documented information.
From the end-user point of view, the input of the testing team will also be considered in a variety of cases. Clients can ask if the software is fit for a specific use or if the software meets standards or complies with regulations. Testers can also test for the compatibility of the current software on different platforms.
Do not miss this reading: How to obtain a solid mobile testing
Adding Trust to the Software Delivery Pipeline
For some software projects, time to market is a crucial element that can draw the line between failure and success. This can entail delivering more than one release a day or even having a continuous delivery pipeline where updates are pushed to clients on a per-commit basis. Under these circumstances, information about the software’s quality needs to be provided very fast. Test automation becomes a necessity for projects of these characteristics.
Software testers are jointly in charge of defining the architecture of an automated software testing solution that can deliver test results in the expected time. They will also code the necessary tests and have them integrated into the pipeline alongside software architects. Additionally, they will decide on the best test execution approach, including halting the tests if any critical one failed.
When automated tests fail, testers will retrieve information about the failure, reproduce the behavior, and file any bugs they deem necessary for the version to be patched and finally released. Apart from all of this, automated tests need to be maintained to keep up with the software’s changes and avoid false positives.
Wrapping Up
The activities involved in software testing are many, one of them being finding the important bugs first. Any one-line definition of software testing will surely fall into an oversimplification. However, if such a definition is desired, we cannot omit the word “quality” in it, as the whole point of testing is to learn about a software’s quality and to communicate the findings properly.
In his Black Box Software Testing Courses, Cem Kaner proposed a definition that pairs well with the broad scope of software testing we have just reviewed: “Software testing is an empirical, technical investigation conducted to provide stakeholders with information about the quality of the product or service under test.”
Comments? Contact us for more information. We’ll quickly get back to you with the information you need.
