Alex Bunardzic asked me a question in a comment that inspires me to do a new post:
“To me (and I’m sure you’ll find ways to convince me how wrong I am), if someone is intent on improving the quality while observing high ethical standards, then it makes more sense to tackle the issue at its source. Or at least tackle it at as high an upstream level as possible. Testing, according to my understanding, is not about producing quality. It is therefore a downstream activity. Also super important, no question, but in a way comparable to tilting at the windmills. Wouldn’t you rather employ and leverage your formidable intellectual prowess for activities related to producing quality? I am pretty convinced that had you chosen the path of a software developer/engineer/architect, you’d be as successful and as influential as you are as a tester. With possibly a much wider reach.”
I appreciate this question, Alex, and I feel like answering it as a post.
First, my intent as a tester is not to improve quality. That’s a hopeful side effect of my process, but I call that a side effect because it is completely beyond our control. Testers do not create, assure, ensure, or insure quality. We do not in any deep sense prove that a product “works.” The direct intent of testing– what occupies our minds and lies at least somewhat within our power– is to discover the truth about the product. The best testers I know are in love with dispelling illusions for the benefit of our clients. I like to joke that we don’t break your product; we break your dreams about your product.
What I treasure about testing is the openness. I started my career as a developer. My job was to create quality code (for video games). I didn’t enjoy that job. My mind is socially activated, and I didn’t get enough social nourishment. I also wanted to do very good work, but as a professional coder I never felt I had the time to do a good enough job. Finally, I felt that there were lots of people who could code as well or better than I could. My strengths lay in systems thinking, rhetoric, document design, and as I would later find– coaching and teaching. Furthermore, I like variety, and I wasn’t getting it by slaving over the same codebase day after day. So when I was offered a testing role, I immediately seized upon it as a way to apply my technical skills, yet still tinker, explore, and solve puzzles. This is possible because testing is a much more plastic process than development is. To create working code I have to write some code, but to be successful as a tester there is not a single specific thing that I am required to do except encounter the product. It’s all situational. I love that flexibility. I feel like a dolphin playing in a large ocean.
The source of quality is unclear, but yes, if you want quality, look for the source. Just exactly as you suggest, moments after I became a tester (in 1987) my thoughts turned to where quality comes from. But for me that focus was not initially the product, but rather myself. I wanted to know where my quality as a tester came from. From the very beginning of my testing career I was also a testing methodologist. I sought to study and learn how testing should be done. This turned out to be much more difficult than I’d supposed, because the available books on testing were all written by people whom I later concluded did not know how to test. After some years desperately poking and searching I discovered a great truth: testing is in our heads. We must seek quality in testing by studying the patterns of thinking that we engage in when we test well. This led me to epistemology, cognitive science, general systems theory, social science, and eventually the study of tacit knowledge and interactional expertise.
Ultimately looking for the source brought me upstream to people, and to the activities of coaching and training. Quality comes from people, and people are a mess. What can I do to make better products in the world? I can train people and coach them. I can set them free from ignorant patterns. I can help create the next generation of leaders. That’s what I love to do. I do this in the realm of testing for one reason, above all: that’s where the money is, given my background. I would rather train people in pure general systems analysis and methods of rapid self-education, but no one is emailing me about that, nor asking me to teach three-day onsite classes for that. I guess, to some degree, I am economically locked into this.
Listen to Sancho Panza. “Look, your worship,” said Sancho; “what we see there are not giants but windmills, and what seem to be their arms are the sails that turned by the wind make the millstone go.” Listen, you can do all the quality creation you can imagine. Still the question goes unanswered: did you succeed and how do you know? Only testing answers that question. Testing is unnecessary only if the risk due to failure is low enough that you don’t need to wonder if you succeeded. Don Quixote tilted at windmills because he thought they were giants. His perceptions were unreliable, but he didn’t test and he didn’t listen to his tester.
More about the difference between testers and developers: https://www.linkedin.com/pulse/software-developing-testing-two-profoundly-different-which-leon/
Agree?
[James’ Reply: I agree with your overall thrust. I would quibble with some of the details (e.g. I think test coverage is a much bigger topic than mere code coverage… other issues like that). But overall, I would say it’s a well-written article that makes an important point. Good job.]
Excellent post, “The direct intent of testing– what occupies our minds and lies at least somewhat within our power– is to discover the truth about the product”. I would go a step further and say it’s to discover the truth about our user’s experience of the product. We as testers need to move beyond just encountering the product and imagining how it will behave in a variety of scenarios and embrace observability to understand the truth about our user’s actual experiences once in Production.
[James’ Reply: Excellent point.]
It may go beyond the truth of our user’s experience of the product. Some products may produce unwanted side effects during the course of operation. Those unwanted side effects may never get experienced by the user, but they nevertheless remain detrimental/unwanted. Discovering that truth is also the job of a tester.
[James’ Reply: Yes. Thank you for making that point. And also thank you for being disputatious, in general. It helps us all raise our minds up.]
Did you mean to say “plastic process”, or “elastic process”? If so, what is a plastic process?
[James’ Reply: Plastic is a nominalization of “plasticity” that I then used as an adjunct noun because it sounds better than saying “process that has plasticity.” Plasticity means basically “malleable” which is more to the point than “elastic.” (Elastic would imply something that is deformable but that returns to a certain shape.) I used plastic also because it is a glancing reference to “neuroplasticity,” which is an important factor in the study of skilled people at work.]
Good answer, James. Thank you for taking the time to explain what’s driving your passion. I enjoyed reading your post (nice touch with Sancho Panza there).
If I may be allowed to play the devil’s advocate for a moment, I’d be tempted to summarize your motives as you’ve decided to be a big fish in a small pond, rather than a regular fish in a big pond.
[James’ Reply: Economically, testing is a small pond. But not in terms of how difficult it is to do well, nor how important it is to organizations that want to build complicated things that work. Still, it is true there is not a lot of money in the testing business; and it doesn’t get a lot of attention, relative to development.]
I totally understand your point about money playing the decisive factor. We all have bills to pay, plus some of us have other needs too. I’m the same — money is important to me and I always strive to find ways to make money while pursuing my passions.
But I’d like to add here that we, software development community, desperately need a person of your caliber to join the ranks. It’s a joy to read sentences like “I feel like a dolphin playing in a large ocean”, and that just confirms my initial recognition of you being an extremely playful mind. What I find of great value in your line of reasoning is that you are very good at thinking inside the box. We need more experts with such propensity. I’ll try to explain why.
Our world is currently plagued by too many people who are thinking outside of the box. This way of thinking is very fashionable now, and is driven by the relentless dictum to disrupt anything and everything. Which, to be frank, sickens me.
But the tragic thing is that very few things in our lives need to be disrupted (however, if we don’t disrupt pretty much everything that comes our way, how are we to make huge profits?)
I feel as if you’re averse to mindless disruption. And if I’m correct, then it would be such a tonic to have a mind of your stature and caliber contribute to the problem spot — software development. And I also feel that if you’d to do that, you’d become a big fish in a big pond.
[James’ Reply: I’m not opposed to people hiring me for that. Perhaps someone will. Thanks for the encouragement.]