Once in a while, when I claim that there are fools out there who want to destroy the culture and craft of testing, some people don’t believe me. So, here is a good example of what I’m talking about: Tests are Bad for Developers.
The post is by Stephan Schmidt. He says he has 40+ years of development experience. I also have 40+ years (it will be 41 on January 1st, 2024, the anniversary of my first day at Dale Disharoon, Inc. My first project was writing Hey Diddle Diddle for Apple II and C64.) A difference between us may be that I’ve spent most of my career in the testing field itself, whereas he says he “writes tests” as part of his developer job. In between writing tests I’m guessing he wrote production code, which is not testing. So, at most he’s dabbled in testing.
In his post he says that “QA” people should not do testing. Only developers should do testing. Let me expand that: he thinks the world would be better if developers, who at best test a fraction of their time and rarely receive any training for it, are the only people who test. He says this is part of developer accountability. I say this a shockingly stupid idea.
Note: When I use the word stupid I’m following a protocol I developed from reading The Triumph of Emptiness. In this book, Mats Alvesson defines “functional stupidity” not as the inability to think, but rather as “a socially supported lack of reflexivity, substantive reasoning, and justification. It entails a refusal to use intellectual resources outside a narrow and ‘safe’ terrain of adaptation to and exploitation of a given social situation. It has nothing to do with low intelligence—people can be intelligent (be good at IQ tests) and be functionally stupid.” (Triumph of Emptiness, p. 216)
The testing field within the computing field has been around for at least 60 years. There are many books and countless scholarly papers about software testing. There are testing conferences, testing companies, and people who are recognized as experts in testing. There are people– me for instance– who have dedicated their careers to testing. No one with 40 years of experience has ANY EXCUSE for not knowing this. But Stephan ignores it. He dismisses us all with a wave of this pen. This is functional stupidity in action.
Schmidt repeatedly refers to the activity of “writing tests.” This tells me he thinks testing is a mechanical process of output checking and nothing more than that. What about test strategy, test design, and testing that embraces the entire user experience? I’m going to guess that Mr. Schmidt’s eyes glaze over when serious testers want to talk about the governing heuristics, structures and skills of testing. How can I possibly know that? Because people who are excited to study and think about testing understand that the test process is not encodeable; they understand that output checking is not the goal or soul of testing. People who enjoy testing understand that most important bugs are NOT found by ANY highly scripted process. Want proof? Just keep track of how bugs are found and you’ll see for yourself. The proof is happening all around you. Just look! I’ve been looking for decades.
Here’s an example from YESTERDAY. I am writing an extension for Chrome (yes, I’m a developer, sometimes). I read lots of documentation and followed online tutorials. I got my extension to work reliably on my machine. But when my brother tried it, it was very flakey. After two days of experiments and lots of different theories, we realized what was going on and I believe I have fixed it. Developing an automated or otherwise scripted check to reveal this problem would have been a great waste of time. Not only would I not be sure (before I knew that there was a problem) what such a check should be, but any check that could have found the problem would have needed expensive GUI-level automation. Due to the nature of this specific bug, the problem would have occurred probabilistically, depending on the architecture of a particular website being browsed at the moment the extension was invoked.
You shouldn’t “write a test” for that, unless you want to waste your company’s time and money. You should perform experiential, highly exploratory testing. If you are a developer, you don’t have a lot of time for that. There is, however, a different kind of person who has both the time and the desire: a tester.
Recommendation: Never use the phrase “write a test.” Procedures can be written, but a test is not a procedure. Instead, a test is a performance. You can say “I created a test” or “I designed a test” or “I wrote a test procedure” or even “I wrote some test code.” None of those phrases imply that testing is an algorithmic activity.
My father was a fighter pilot. He said the most common mistake new fighter pilots make is that they think in only two dimensions. I remember that when I hear people confuse checking with testing, and when people think there is nothing more to testing than the bit of coding that a developer feels like doing (while dissing the entire field of skilled testing). These people are living in flatland.
Earlier this year I had to sit down with a PM and explain that the statement “fully unit tested” from a vendor meant next to nothing from a testing perspective. We ended up testing quality in the hard way unfortunately.
I can relate to this experience. When dealing with vendors, all testing is often conducted within a very limited timeframe. The term “shallow testing” comes to mind in such situations.
That’s a view point from unit , Considering a product X , built by company X1 . Considering fully tested from the above write up , when the X gets integrated to a larger ecosystem of Enterprises . merely a small flaw disrupts the whole ecosystem . So testing , quality assurance , engineering , test automation , automation all have its own respective respect and reason why they matter for larger ball games… No products built today are for one goal , they have way too larger connected ecosystem and quality/testing from the bottom makes an important aspect for success. I see that day in day out ..
The challenge is that this thinking is all around us. It always has been, though recently I’ve experienced it more as projects I’ve been engaged in have gone back to devops, I say gone back to, because my recollection of 20 odd years in testing was devs building, testing and supporting their own code was how we used to work back in the 20th century, Then we collectively learnt that doesn’t result in quality solutions as devs might check stuff but they’re not testers and typically this failed at integration, 20 odd years on and it feels like de ja vu
Nah, It looks like this post is a reaction to a purposeful misinterpretation of Schmidt’s post.
[James’ Reply: I’m reacting directly to Schmidt’s post. I’m reacting to the words he used. These words have important meanings. I suppose it’s possible that Schmidt means something different than what I think he means. If so, I’d appreciate if he were to correct his post.]
It’s a matter of using the same words to designate different things.
He’s stating that developers should write a lot of checks (he seems to be referring mainly to unit-checks, but I’m guessing it’s not limited to it only), and at least if we accept the claims made in “Accelerate”, there’s a strong correlation between developers owning the automated checks and company success, so there might be something in this claim.
[James’ Reply: He is certainly claiming that developers should own automated checks. And also everything that is called a test. I challenge you to point to any sentence where he indicates that anyone else other than a developer should do anything called “testing.” At most he allows that there could be “QA” people (assuming you have them, which he indicates is optional) who peer over the shoulder of developers who are writing tests.]
The way I prefer to read this advice is that each developer should own the shallow testing of their work – There’s a lot of value by covering that, and not a lot of need for highly skilled testers to perform those. Once the shallow tests are covered, a skilled tester can do much better work without stupid roadblocks or distractions. At this point, a company can decide whether they need a tester to go deeper or not (Judging by the amount of companies whose dedicated testers only do shallow tests today, I won’t be surprised if some companies will say “ok, that’s good enough for us”).
The weak point in the article is “there’s nothing in it for the devs”, but it seems to me this is not the topic here.
[James’ Reply: Your interpretation is charitable to the point of being Schmidt’s personal Father Christmas. Come on, man. The author said not one word about the process of testing as done by testers, but your sure that he thinks there should be testers who do sophisticated testing. There is no way any CTO is going to read this article that way. This article give 100% aid and comfort to the “who needs testers?” crowd. And that’s what I’m complaining about.]
Schmidt writes: “If you have QA engineers, instead of testing, they should review developer written tests. They can make sure tests are testing the right thing. They can help developers write better tests. They can help developers write tests in a way so that they are easy to change to changing requirements (…)” and he is right in that case. We, as testers, can do that mostly better. Work together to get the right ones.
[James’ Reply: He said that, and what he said disrespects testers and the testing craft as understood by testers. I don’t know why you are ignoring the phrase “instead of testing,” but please don’t give him a pass about that.]
I find myself in furious agreement with you, as usual, James. Any intelligent person who’s spent time testing knows that it isn’t an activity that can be scripted or programmed ahead of time. But the issue is that you typically get people ‘managing’ testing who think that the testing process is about first writing test cases, then executing them all, until at the end your product is ‘fully tested’. To a testing professional, this is insane, but it’s often the world we live in.
Great story/example. I think you would have liked my 2nd presentation on TestNet.
“Actually the minefield is more like a 3D place as well” (I think you understand what I say).
You make a lot of assumptions about me 🙂
[James’ Reply: Oh? Feel free to correct them, if they are wrong. Here’s my method, which the same as your method: I read, I interpret, I consider, I respond.
You publicly wrote words, so you must expect people to read and interpret them. Of course I may have misinterpreted, but it’s not wrong to draw plausible inferences. It’s normal practice.]