Exploratory Testing for Product Configuration
When testing product configurations, testers don’t usually rely on exploratory testing. They argue that they are not testing traditional software products but various different configurations for the product. In reality, exploratory testing can be a very useful technique that can reduce testing efforts and improve speed to market. But before we dive into that let me first provide you with some key definitions:
Exploratory testing is methodology where a tester designs and tests at the same time. The focus here is to spend less time on formal documentation like test conditions, and detailed test steps. The process is really about using the tester’s skill and domain knowledge to explore/test the product for issues instead of just ‘checking’.
Product Configuration or product customization, is an activity of customizing or configuring a software product to meet the needs of a particular customer. For example, a pharmaceutical company has asked a clinical trial company to develop a drug study. These drug studies will be different for each pharmaceutical company even though they could be using the same base. In addition, the set of questions that appear to the end-patient will be different depending on multiple factors.
Now that you have some of the key definitions, let’s discuss how exploratory testing can be used in configuration testing.
Stakeholders will tell you that they want effort and speed to market reduced, while still maintaining a high level of quality. No process/approach can provide 100% coverage but testers do have to utilize testing approaches that provide maximum test coverage while providing a high level of confidence. In configuration based testing, the scenarios can be very specific or at times can be very vague. Testing teams will not have the bandwidth to test all scenarios. For example, in our past experience working with a pharmaceutical based ISV that develops customized software for clinical trials, we have noticed many different configurations. Trial users will have different sets of question workflows based on the answers provided. There are many flows that can occur here and not all them could be documented. More importantly, if there are defects due to settings/configurations, it can potentially be life threatening. Applying exploratory testing, you can create sessions and charters to expand the scope, especially if not everything is documented. This provides better coverage.
Exploratory testing can also be used to speed up your release as well. Majority of testers will spend a lot of time on formal documentation, especially if it’s configuration based. They will write tests for every single scenario/configuration. That could take quite a bit of time. Sometimes they spend more time writing then testing. If you create charters and sessions, you are already documenting what you are testing. You save time here as now you’re spending more time testing and less time on documentation. In addition, you can use tools that can automatically capture/record your test steps while you’re testing. This is very important in regulated industries as proof of testing is key. Once such tool that does this is qTest Explorer by QASymphony. It tracks user’s interactions from the testing session and uses that information to automatically create defect documentation. It supports the recording of a wide range of applications and technologies on the browser as well as desktop apps. Using exploratory testing and tools can drastically reduce your effort, hence speed to market is improved.
There are also other areas in configuration based testing that exploratory testing can help. Its proven to find major defects early in testing, which everyone knows is a lot cheaper to fix early in a cycle. It is also good to apply for regression after all defects have been resolved. This ensures maximum coverage.
In conclusion, you can see that exploratory testing can and should be applied to configuration based testing due to the fact that it will provide better coverage and help reduce effort.
Find out more about our software testing services.
