software testing

Striking the Balance: Shipping on Time with Aggressive Testing

Posted By: Hiba Sulaiman
Posted On: April 29, 2020

If you’re someone who’s trying to find the right way to build and ship software, let me disappoint you by saying that there isn’t one. There is no ideal way of doing so. The methodology with which you should approach a given problem depends entirely on that problem and specific goals of your organization. These goals change with time and so should the methodologies. Some organizations make the mistake of getting too attached to their development and release habits, and in the end, pay for it. Regardless of what approach you opt for, there will always be a trade-off or an opportunity cost. For example, if you look to maximize schedule predictability, you will have to forgo engineer productivity. In the end, it all depends on what risks your organization is willing to take. To understand the right way to build your product, first, you need to understand what matters for your product and how to optimize for that. If your product is highly-priced, businesses are buying it based on their ‘need’. The more expensive it is, the more business-critical it is for your customers and it is more likely that you have to optimize it for functionality, reliability, and predictable schedule. More than being fast, businesses want you to be predictable. That’s why working on a predictable schedule is very important. Another aspect of offering an expensive product is that you have a fewer number of customers, which means that the value of each customer is very high and thus, satisfying them is very crucial. As opposed to the prior discussed case, if the software offered is less expensive, the target market volume is greater, consisting of small businesses or consumers. In this case, schedules are less important and the importance of a single customer is very less. You can afford to deprioritize niche platforms or bugs that only affect a few people. But this, in no way, means that the quality should be ignored. For a consumer product, your focus should be more on the UI/UX of the product. To have a seamless UI/UX, you need to:

Empower your design team
Prototype and iterate before committing to dates
Ensure collaboration between design, engineering, and user testing teams

The right approach also depends on your deployment model. If the product is released on the cloud, this means that you have total control of runtime environment and in the absence of a test matrix, the testing time will be exponentially reduced and so will the volume of bugs. It also gives you the freedom to update whenever you like, the distribution is instantaneous and the universal deleted code goes away permanently. If let’s say, the software is deployed onto the customer’s device, the one-time and future cost of release is relatively higher.

Aggressive Testing

Aggressive testing is all about constantly questioning your product and pushing it to the limit. Testers are required to have out-of-the-box thinking abilities to experiment and play with the application. They need to keep asking themselves, “how else can the product fail?”. After all, it is a proactive approach to finding bugs.

>It’s a different approach than having a predetermined course of action which allows confirmation bias to influence work. It’s about exploring by session-based test management or using high-level test conditions, risks, or ideas. This is how lurking bugs are pursued as there are no constraints.

Users Expect Bug Fixes and New Features

Users expect the software to be free of bugs in return for what they pay. It is impossible to fix all the bugs in one go (at the point of release), which is why regular updates are very important. Users want new features that solve their problems, are usable, and not complicated. But not all the proposed features need to be included in the product. There are certain limitations of software and even though the proposed features may sound like game-changers, it’s not a good idea to change the game that many times. Features should be good enough. That’s why, it is recommended to keep your default response to feature requests, “NO”. And when you do decide to release a feature, make sure that you measure its success before. Both features and bug fixes should be based on user feedback. You should know what users like, dislike, and what problems they are facing. After all, you’re making the product for them.

Importance of Test Automation

Aggressive testing also means aggressive deadlines. Teams face a tremendous amount of pressure to meet these deadlines without compromising on the quality. For this purpose, test automation can be very helpful in easing the tasks of teams and take care of all the tedious and repetitive tasks.

Let’s talk about the risks…

Startups should be aggressive risk-takes with nothing-to-lose mentality. What do you fear of losing when you already have no customers, revenue, or brand? In this scenario, the cost of a mistake is close to none. And that is when you can afford to make mistakes the most. We can’t say the same thing for a well-established company which has a lot on the line. Risk is affected by the development model as well. When customers experience problems, how and how quickly you respond to these problems can be as important as how bad the mistake was in the first place. If you instantly solve every user’s problem, you have much faster remediation than if a two-week QA window is involved in your release process and an app store review process for the smallest code change, after which customers install the patch at their own convenience. If you combine both development and risk, you’ll see that selling an on-premise software to enterprises for a higher price will set you up against the challenge of magnitude-of-pain and lengthy-time-to-update.

Conclusion

After taking your business model, deployment model, and how much risk you’re willing to take, you get a framework sufficient to analyze your processes. If you’ve survived long enough to achieve product-market fit and customer traction, then you’re probably aware of the practices that suit your organization for different situations and goals. This framework can also be helpful in judging which company’s practices you should imitate or whose advice you should take. This framework can be very useful is you’re struggling with business models or technologies.