Software Testing – Third Eye View

A cognitive model is a person’s understanding of how something works, that helps in making quick and good decisions. We all depend on it to understand the complexity throughout. 

In this article, we will discuss a particular cognitive model that can guide your decisions as a Test Engineer. We often confuse models of reality with realism itself. We use them to ease the complications that are around us. Models are an abstraction and reduction of the real world. They are a rough representation of reality but not the reality itself. Hence even the best models are imperfect. We often fail to understand this limitation.

Requirements Are Just Guides

Agile incremental approach helps teams to check if the increment matches the client’s expectations and perception of the software. Today Scrum is the preferred choice for every organization to build and deliver software.

Several teams and organizations claim to be Agile by incorporating a couple of Scrum rituals like standups and retrospectives, without understanding the intention behind them.

When teams fail to understand the basics of Agile they miss out on opportunities to inspect and adapt. 

Below is an example of how misinterpreting can lead to concerns:

A team was working on a complete revamp of the user interface. The aim of the revamp was to improve the user experience. Test engineer was in charge of testing this software UI upgrade.

The complete look and feel of the application were improved to great extent. The application had a date picker that lets the user select the date.

The date picker seems to function fine, however, the date field size was too big which occupied a lot of space. The Test Engineer spoke to the developer and discussed this concern. Test Engineer expressed that date field interfaces need not be too large which is leading to more space more space utilization (real estate) and that can be optimized by including few more similar fields on the same line in the UI. Developer said that the requirement document did not specify the width or length of the date picker and tried to justify that it was not an issue. He seemed to miss that the objective of the revamp was to improve the user experience.

Requirement documentation is a guide that attempts to describe the customer’s needs for the product, but it does not inherently deliver business value. Requirement documentation is a description of the product and not the product itself.

A good tester understands that a requirement document has the below limitations:

•  They can miss certain information.
  • Implicit requirements  – things that users will expect but might not be explicitly captured in the requirement document.
• They can be interpreted differently by people, which may lead to mistakes.
• A requirement document may also go outdated and does not reflect the updated information.

Models are not perfect but useful

The models we use like an architecture diagram, state transition diagram, or a use case diagram do not represent the exact real behavior in reality.
For example, an architectural diagram is a high-level view of the overall outline of the software system, and illustrates how different components in a software system interact with one another.
The architectural diagram helps us understand complex systems and their interactions. However, it’s a simplification, hence does not capture everything given the number of moving parts, and does not perfectly reflect how the real software will operate.

Sometimes the data can be misleading

Organizations collect and analyze data to make informed decisions. Data  is correct but it may not tell the whole fact. We have the tendency to find non-existent patterns by making flawed correlations.
For example, a team felt that there is a connection between the releases and production issues. The data they analyzed also hinted the same thing. They decided to slow down on the number of releases they made. However, the production issues did not reduce despite lesser releases.
Later, upon detailed investigation, the team realized that customers mostly reported legacy issues that had nothing to do with the new releases.

Hence analyze metrics, numbers, and trends from different perspectives.

• The same data can be interpreted in different ways by different people. 
• A sample portion of data may not represent the entirety and can be misleading. 
• Without the context data is meaningless. 

Conclusion

Easy models are less accurate but are easier to understand. As the model continues to be more complete and accurate they become less understandable.  As a tester, be doubtful about requirements and understand their limitations.
No matter how accurate a model seems, remind yourself that it is an imperfect explanation of reality. Knowing this helps you to think through problems and make better decisions.