On Answering Questions About ET

People ask me a lot of questions about ET. I want to be helpful in my answers. A problem I struggle with is that questions about ET often come with a lot of assumptions, and the first thing I have to do is to make the assumptions visible and try to clear away the ones that aren’t helpful. Otherwise, my answers will sound crazy.

Questions of any kind rest on premises. That’s cool, and normally it’s not a big problem. It becomes a problem when questions are asked across a paradigmatic chasm. And there’s a big chasm between the premises of “traditional testing” and those of context-driven test methodology, and those of Rapid Software Testing, which is what I call my test methodology.

Starting in 1987, I tried to learn software testing. Starting in 1989, I started reinventing testing for myself, having become disillusioned with the empty calories of folklore that I found in books by folks like William Perry, or the misanthropic techniquism of Boris Beizer (Boris once told me that it didn’t bother him if people find his advice impractical, since he was merely concerned with documenting “best practices”, a phenomenon that he seemed to think has nothing to do with applicability or utility).

I “invented” testing (with the help of many colleagues) mainly by discovering that the problems of testing have already been solved in the fields of cognitive psychology, epistemology, and general systems thinking. The lessons of these much broader and older fields having been studiously ignored by the majority of authors in our field. This puts me in the odd position of having to defend exploratory thinking in technical work as if it’s some kind of new fangled idea, rather than a prime driver of scientific progress since the advent of science itself.

Anyway, now my island of testing metaphysics is mostly complete. I can plan and do and defend my testing without any reference to ideas published in testing “textbooks” or any oral folklore tradition. Instead I reference ideas from logic, the study of cognition, and the philosophy of science. My system works, but it’s a big job to explain it to testing traditionalists, unless they read broadly. For instance, if I were to say that I test the way Richard Feynman used to test, some people get it right away.

Let me illustrate my difficulty: Julian Harty asks “Do you expect an Exploratory Tester to be well versed in [traditional testing] techniques? Do you check that they are competent in them, etc?”

I’ve had some discussions with Julian. He seems like a friendly fellow. My brother Jonathan, who’s had more discussions with him, says “Julian is one of us.” That’s a serious endorsement. So, I don’t want to alienate Julian. I hope I can turn him into an ally.

Still, his question poses a challenge.

Not “exploratory tester”, just “tester.”
First, there is no such thing as an “exploratory tester”, separate from a “traditional tester”, except as a rhetorical device. I sometimes call myself an exploratory tester in debates, by which I mean someone who studies exploratory testing and tries to do it well. But that doesn’t seem to be how Julian is using the term. The truth is all testers are exploratory testers, in that we all test in exploratory ways. Some of us know how to do it well; fewer of us can explain it or teach it.

Testers are testers. Some testers are especially good at simultaneous learning, test design, and test execution, an intellectual blend called exploratory testing.

Exploratory testing is not a technique, it’s an approach.
A technique is a gimmick. A technique is a little thing. There are a buh-zillion techniques. Exploratory thinking is not a technique, but an approach, just as scripted testing is an approach. Approaches modify techniques. Any technique of testing can be approach in an exploratory way or a scripted way, or some combination of the two.

Traditional testing techniques are often not really techniques of testing, they are symbols in a mythology of testing.
Consider the technique “boundary testing.” One would think that this involves analyzing boundaries, somehow, and testing that there are no bugs in software products that are boundary-related. But actually, the way testing is written about and taught, almost no guidance is given to testers about how to analyze anything, including boundaries. Boundary testing isn’t so much a technique as a label, and by repeating the label to each other, we think we are accomplishing something. Now, I do have an exploratory approach to boundary testing. I use various heuristics as part of the boundary testing process, but for the most part, boundary testing is ordinary testing. The technique is a tiny part of it compared to the generic skills of modeling, observing, and evaluating that underlie all skilled testing.

I don’t teach boundary testing in my classes because it’s too trivial to worry about.

So, with that preamble, I can answer the question:

Julian, I assume by “traditional test techniques” you aren’t referring the tradition of using one’s eyes and brain to test something, but rather to certain high sounding labels like equivalence class partitioning (a fancy way of saying “run a different test instead of the same test over and over again”) or black-box testing (a fancy way of saying “test without knowing everything”) or cause-effect graphing (a way of saying “I repeat things I see in books even if the ideas are totally impractical”). I don’t teach those labels to novice testers, Julian, because they don’t help a new tester actually test anything, and I want novices to learn how to test.

But to be an educated tester who is effective at explaining test methodology, I think you need to know the buzzwords; you need to know the folklore. This is true whether you are a tester who embraces exploratory testing, or one who still pretends that you don’t do ET.

A tester– any tester, not just one who follows my rapid testing vision– needs to develop the cognitive skills to effectively question technology. Gain those and you automatically gain everything important about “traditional test techniques”, in my opinion.

To see such a skill in action, ask yourself this question: how many dimensions of a wine glass can you list? Then watch what your mind does next. To answer this question you need a skill that I have come to call “factoring”, which is a component of modeling skill. It is a skill, not a technique, though there may be many techniques we might apply in the course of exhibiting our skill.