So, for exercise, I’m picking up trash. Here is a picture of me all kitted up:
So far, I’ve done 37 miles of trash collecting. And I can’t help but see some interesting parallels with software testing…
Just Like Testing #1: I can use tools, but I cannot automate my work.
I have to make a lot of judgments about what to pick up and what to leave. It would be difficult to even to write a detailed and complete specification for what constitutes trash and what does not, let alone design a machine to pick it up. Yes, there are semi-automated street sweeping machines, and they do great things– but they are also expensive, loud, and disruptive. They also work only on flat paved surfaces, as far as I know, whereas I am cleaning along country roads and fishing garbage out of ditches.
Just Like Testing #2: I crave trouble. If the product is too clean I feel depressed.
I smile when I see a nice juicy old beer bottle. That is paydirt, baby. Aluminum cans and brightly colored drinking cups are almost as sweet. Apart from anything else, they weigh down my trash bag so that it doesn’t flap in the wind, but mainly it is from these undeniably pieces of unsightly rubbish that give me a charge.
On the other hand, when I don’t see trash, I feel like I haven’t done anything. I know that I have: my eyes have searched for trash and that’s a service. But finding trash gives me something to show for my work. I can drop the bag in front of my wife and say “seeee? I’m useful!”
Just Like Testing #3: Trouble that is most likely to upset normal people makes me most happy.
Brightly colored candy wrappers are terrible to see on a country road surrounded by nature, but that same bright color makes it easy for me to spot. So, I hope candy and soda companies don’t start marketing their wares in camouflaged containers. Similarly in testing, when we see a dramatic crash or data loss in testing, we testers give each other high-fives and yessses and “you are a steely eyed missile man”-type comments. It takes extraordinary restraint not to do that right in front of the developer whose product has just failed.
Just Like Testing #4: Gratuitous trouble makes me tired and depressed.
I have sometimes come across caches of garbage, as if someone just hurled a kitchen trash bag off the back of a truck. This is not fun. I don’t mind the ordinary careless litter, to some degree, but industrial scale contempt for the environment make me feel disgust instead of fun.
Most of the trash I find is rather innocent. It falls into these categories:
- Food wrappers: things a kid might throw out a car window.
- Brick-a-brack: things that might fall out of the back of a contractor’s pickup truck.
- Featherweight trash: things that accidentally blow out the window of a car
- Cycling debris: things that a cyclist might drop accidentally; occasional items of clothing
- Auto debris: pieces of cars
- Transported trash: things blown onto the road from adjoining property
But when an item or items of garbage seem diabolical or contemptuous, or systematically careless, I do get a little angry. This is similar to the feeling a tester gets when the developer won’t even do the most basic of testing before throwing it over the wall for a test cycle.
Just Like Testing #5: I became hyper-sensitive to regressions.
Today I drove into town and saw at least four pieces of trash along the way that had not been there yesterday. I am annoyed. This was a perfectly good road when I last cleaned it and now it’s all messed up again. Now, I know, objectively, it’s not “all messed up.” It is still far cleaner than it was when I started working it. But all I can think about is that new trash! Who did it? BURN THEM!
Testers also tend to get oversensitive and find it hard to accept that quality can be good enough when we know that there are unfixed bugs in the product. I guess anything you invest yourself in becomes sharper and larger and more important in that way.
Just Like Testing #6: I overlook some trash no matter how hard I try to look for it.
My wife helped me clean the local beach. We went single file, so she caught some of the things that I missed. There were a lot of them. Some trash I didn’t see was pretty big. My inner experience is “How did I miss that???!?!?!” But I know how I missed it: inattentional blindness.
Inattentional blindness is when you don’t see something that is in your field of view because your attention is on something else. This can have the effect of feeling as if an object literally appeared out of thin air when it was right in front of you all the time. I once covered an area, then turned and looked behind me, and saw a medium sized plastic bag just a few feet behind me. I had walked right over it without seeing it. It’s frustrating, but it’s a fact of life I must accept.
This is why pair testing, group testing, or making multiple passes through the same product helps so much. I always want redundancy. Along the main roads, I want to make at least two passes on each side before I move on to another road.
When I am “regression walking,” I might expect to find only newly dropped trash since my last walk. Instead, just like in testing life, I often find old trash that has been there all along but never before noticed.
[Added August 7th, 2017]
Just Like Testing #7: My quality standards are not fixed or absolute; they vary and they are relative.
I notice that when I am cleaning a very cluttered area, I tend to ignore very small pieces of trash, or trash that is hard to access. But when I covering a clean area, I raise my standards and pick up even tiny pieces (smaller than a bottle cap), as if I am “hungry” for trash.
Similarly, I might pick up a small piece because it is next to a large piece, since I am already “in the neighborhood.” Also, if a large piece has been shattered into small pieces, like a broken beer bottle, I will pick up even tiny pieces of the bottle in order to get the “whole bottle.”
All this is evidence that I do not judge trash on an absolute scale, but rather judge it differently according to a variety of factors, including what’s nearby, what I’ve recently seen, my fatigue, my self-judgment, etc. It’s the same with bugs. I want to find something, but I also have limited energy. And this is why it is good for me to take multiple passes through an area. It helps me to square my selection heuristics with my general and absolute sense of my mission and proper quality standard.
Thanks for a very lovely post, not Trashy at all 🙂
I liked this post for various reasons. Foremost, because I think that metaphors are useful. That is, I think that considering something in another context can be valuable.
Here are a few points that you considered and made explicit that I’d like to emphasize and build on, as well.
|Just Like Testing #1: I can use tools, but I cannot automate my work.|
As you suggested, in addition to semi-automated street sweeping machines being expensive, loud, and disruptive, I think it is very important to highlight that semi-automated street sweeping machines sometimes a) neglect to pick up some trash, and b) pick up some non-trash. If a desired goal is to “pick up trash”, then semi-automated street sweeping machines will fail to achieve that goal if they a) miss some trash, and b) pick up non-trash. Similarly, if a desired goal is to “find bugs”, then semi-automated bug finding machines will fail to achieve that goal for similar reasons: if they a) miss some bugs, and b) find some non-bugs (things that aren’t a problem for someone at some time). Semi-automated bug finding machines cannot make “judgments (the formation of an opinion or conclusion concerning something) about what to pick up and what to leave.” as humans can.
|Just Like Testing #2: I crave trouble. If the product is too clean I feel depressed.|
I think that the way I feel while performing some task, and the way I feel about the results of that task are important. And I think it is interesting that, although I might logically understand that my performance and the results of some task are valuable, my emotions and perceptions can sometimes make me feel the opposite.
I try to encourage and remind myself that my testing performance itself might be useful, even if the results might not seem to reflect that. A particular street might already be clean. However, until I perform the task of searching the street for trash (testing the street for bugs), I cannot be sure. And so, I think that even if no trash/bugs are found, the mere act of searching and reporting “how I searched” and “what I found” can be valuable.
|Just Like Testing #3: Trouble that is most likely to upset normal people makes me most happy.|
I think this is important for several reasons, including our behavior and communication. The way I behave and report my findings to others can greatly affect the way my findings are received. And so, as you suggested, I think it makes sense to sometimes practice reasonable restraint, AND to also to employ empathy and carefully choose my words when providing information to others. Especially when that information might likely be perceived as critical/negative.
| Just Like Testing #5: I became hyper-sensitive to regressions. |
| Just Like Testing #6: I overlook some trash no matter how hard I try to look for it. |
I see a possible connection between these 2 points. Perhaps, the bag you missed was due to inattentional blindness. Another possibility is that the bag simply wasn’t there when you searched that area. Perhaps, after you passed by that area, a gust of wind blew the bag behind you. That part of the beach “regressed” to a less-clean state. Regardless of the reason, I think both reasons provide a reasonable argument for some type of testing redundancy.
I think that “picking up trash” as a metaphor for “testing” is very rich, and could provide many more relevant comparisons to consider.
Really nice the connection between inattentional blindness and regression.
I am so glad that you begun to wrote again on this bloged!
Very creative analogy.
Very nice and I found myself laughing a long the way since I have occasionally donned a green vest that I saved from my town dump visit and I too have picked up trash along side of the road and felt great joy and angst and some in same items you saw on the road. You steely-eyes missile man you.
[James’ Reply: I’m not the only one? I also think the fact that I’m a father is part of my motivation. It’s just the sort of think I think a father should do: make the world better for the kids to play in.]
Dear James,
you are a great inspiration for me. I was put into our newly founded software testing department and wasn’t really happy about it (being a software developer). By now I’m seeing it as an opportunity to take over more responsibility as an employee (becoming the indispensable linchpin as you narrate it in some of your speeches).
I’ve come across your presentations on youtube when I was researching about how to test efficiently and learnt a lot from you. I wish you would return to your youtube channel by the way.
Thank you again for being such a great inspiration.
[James’ Reply: Thanks. What would you want to see a video about?]
I’d enjoy watching your blog as a vlog (because you are a good speaker and I also enjoyed your Easy Button, Describe A Stone and Swashbooking videos). But I guess you prefer blogging (maybe for time consuming issues).
As a concrete idea a video about your trash walking would be interesting. You could equip yourself with a go pro and take us along your walk (or ride).
I disagree that work cannot be fully automated: in your work, you are the machine, even if biological. So you are the tool, and to the rest of us, this work is automated. Stop thinking of tools strictly as digital or mechanical machines.
[James’ Reply: This is not what automation means. Nobody hires a person to test and calls that “automation.” What I am referring to is a non-human contrivance of some sort. I don’t know how to write code or design a machine that fulfills the responsibilities of a tester and neither do you. That’s more my point.]
Further: automation does NOT imply perfection. you seem to only allow an “all or nothing” in your mind for automation. That’s so lame.
[James’ Reply: That’s not my argument at all. I use tools happily in my work. What I don’t do is pretend that my tools are doing testing. I am the only one doing testing. The tools assist me. This is exactly the same with developers. Developers don’t say that they are “automated” developers when they use tools. Ask yourself why.]
Lastly, your view of “throwing it over the wall” to test is so old. Have you not been part of any agile team?
[James’ Reply: Yes, there are lots of Agile groups out there, and most of those are still “throwing it over the wall” kid. Maybe you should visit more companies and then you’d see what’s happening.]
The 1960s called: they want their tester back.
[James’ Reply: I’ve only been in the testing business since 1987.]
You state: Nobody hires a person to test and calls that “automation.”
Look up Amazon’s Mechanical Turk.
[James’ Reply: That is the cleverest but still wrong reply I think you could have made…
1. Nobody who uses Mechanical Turk would claim that they have automated their process, but only that they have automated the assigning of tasks and collation of results.
2. The name itself comes from a famous fraud where something looked automated but wasn’t.
3. Using that method is an admission that you need people.
4. The system generally relies on exploiting the availability of poor people. It’s not a system that relates to skilled technical workers, by and large.]
[Comment Redacted]
[James’ Reply: I’m sorry but your comment does not meet our minimum quality standard. Please see the commenting policy details and also get your head out of your ass.]
Related to trash walking, are you familiar with David Sedaris?
Rubbish job: Why don’t we just pick up litter? – http://www.bbc.co.uk/news/uk-england-36090973
This is now officially “a thing” – good on you!
James,
Related to your “Gratuitous trouble makes me tired and depressed” but much worse occurs when I pickup trash on the wooded street around my house in Connecticut. My biggest challenge is the waves of hatred that come over me as I wade through the empty scotch bottles and diapers that people throw out of their car windows for some reason.
It’s so bad I want to start yelling at random cars driving past.
I think even the worst developers don’t deliberately create bugs.
This leads me to the conclusion that the world of software development is a fine place to be with respect to say … a street in a leafy suburb.
Regards,
I am quite glad that heroes like you exist in our world as not everyone today keeps cleanliness in mind. That is why I am aiming to purchase some litter pickup tools for myself and my wife as we’d like to do our part too even if just by the area of our property or the nearby park. We’ll be doing our best to use the tools as properly as we can too. Thanks for the entertaining and informative article about cleaning up the trash!
What happened to the pictures of the blog? I can’t see any of those picture (anythere in other blog posts too).
[James’ Reply: Ugh I have to fix that…]