At the recent Workshop on Training Software Testers, Morven Gentleman showed us a chart of some test results. I was surprised to see a certain pattern in the results. I began to think of new and better tests to probe the phenomenon.
Morven told us that the tester who produced that chart did not see anything strange about it. This intrigued me. Why did Morven and I see something worth investigation when the tester did not?
Then I stopped myself and tried to discover my own thought process on this. A few minutes later this exploratory testing heuristic came to mind:
I MAKE AN OBSERVATION DURING A TEST…
1. I experience surprise associated with a pattern within the observation.
That triggers REFLECTION about PLAUSIBILITY…
2. The pattern seems implausible relative to my current model of the phenomenon.
That triggers REFLECTION about RISK…
3. I can bring to mind a risk associated with that implausible pattern.
That triggers REFLECTION on MAGNITUDE OF RISK…
4. The risk seems important.
That triggers TEST REDESIGN…
Now, I don’t really know if this is my thought process, but it’s a pattern I might be able to use to explain to new testers how surprise can be a test tool.
Hi James,
may you show us the chart?
[James’ Reply: Morven’s presentation with the particular chart (actually three different charts) is here. You can find all the materials from the WTST 5 meeting, here.]