Exploratory testing is hard... if the product is difficult to use
1.
A product or feature with many days worth of nitpicky bugs could be "at risk" because a tester might be distracted by all the easy bugs and not spot a workflow (usecase) that is important.
2.
Features that are hard to use or configure might get missed. Consider a product with 10 major features, and 9 of them take 1 day to configure and get real data back from. The 10th requires 10 days to configure (it requires the sum of all other features to work) plus another day to collect data. In this case the testing is probably going to be interrupted (a customer escalation, a project date boundary, etc.), making it easy to forget to come back to feature #10.
Here are some ideas to mitigate these risks, do you have others?
-Use written testcases for these features. (Might be hard to know that you will hit problem 1 ahead of time)
-Do use case based testing, and track your results by use case instead of by program feature. (I want to try this idea on my current project, but have not yet)
-Put the best most "bloodhound" minded tester on that one problem area, and assume that they are up to the challenge.
1 comment:
You have raised some challenges in scoping and prioritizing testing on a complex product.
Issue #1 could be mitigated in a couple of ways. First, the most useful exploratory testing on complex products seems to require the tester build intermediate or expert knowledge of the feature/product. The tester has to learn from experts -- developers, users, support, product management, and/or other testers. They have to examine documentation and diagrams, and they probably need to create new documentation and diagrams as they build their own model of the system. By building this base of knowledge, the tester is less likely to miss an important workflow.
Second, you can scope an exploratory charter more specifically to address the workflow -- or create scripts. Use cases could be a great method for scoping your exploration to the most important cases.
As for issue #2, forgetfulness seems like a poor excuse for not testing something. Completion of testing should be tracked in some way. Even a to-do list could suffice for this example. If we change "forgetting" to "running out of time", then we come to prioritization. Would the business stakeholders prefer to use the tester's limited time to test case #10 or to resolve the customer escalation case? The decision makers need to be made aware of the tradeoff. Something won't get done. Testing is infinite. Work is infinite in general :)
Post a Comment