Sometimes I hear people react to context-driven methodology with a shrug. “Yeah, everything depends on context. So what?”
Here’s the so what: If all practice depends upon context, then the competent practitioner must know how to invent, evaluate, criticize, and modify practices. In other words, focus must shift from merely copying and memorizing practices to developing skill in the craft.This is a profoundly different focus.
Wouldn’t it be absurd for someone to claim that the best way to commute to work, for everyone in the software industry, is to take a horse and buggy? Wouldn’t it still be absurd even if they added the words “but you should take context into account and do what’s right for you” at the end of such a strange claim? When consultants talk to people from different practical contexts, and they treat context variables as a trivial concern, they do their audience little service. That’s the trouble with “best practices.”
To say that context matters but that “almost always X is the right thing to do” is to speak like a parent dictating to a child. I fear that many testing bloggers and consultants are happy to do just that. But if what we want is a meeting of minds and sharing of wisdom, we need to practice talking about context-dynamics and the dynamics of practices that serve those contexts.
The most recent time I practiced this, myself, was in writing the previous paragraph. I almost wrote that you should not, as a methodologist, speak to your clients as a parent dictating to children. But then I noticed that very statement is just such a dictum. I don’t want to be like that. So I had to add “But if what we want is…” to at least acknowledge that even here there are different ways to be, depending on context.
A Suggestion
A great way to automatically avoid the perils of best practice talk is simply to talk about your own experiences and preferences, rather than make general prescriptions. This is why, in peer conferences such as LAWST, we focus on experience reports (actually we called them “war stories” until our terminology was attacked by bloodthirsty pacifists) rather than advertisments for good practice.
James, it seems to me that you are talking about design skills. As an actuary, I build complex financial spreadsheets, and while I know lots of techniques, what matters most is which I choose to use, and how, and that is design. Similarly, testers need to master many different techniques, but what matters is how they apply that knowledge to each unique job, which is all about craftsmanship and design choices.
In my industry, like yours, many people pay lip service to design, but in practice they tend to stick to only a few methods with which they are familiar, and use them for everything. Unfortunately, I think it is human nature to keep things simple for ourselves…
[James’ Reply: Exactly. Design.]
I’m sure you know this, James, but I’ll say it anyway: Dicta have tremendous appeal. If they didn’t, “mindfulness” for humans would be like the water fish swim in, instead of something that is consigned to the realm of the exotic, or something paid brief introductory lip service…see de Bono’s claim on “the primary purpose of thinking”.
The persistent images of testing as (1) rote monkey work, and (2) something the smart employee will grow out of — these feed a dictator/dictatee tendency.
I just read “The One Best Way”, a book about Fred Taylor, the father of “scientific management” (and the techniques of manufacturing that his school of thought drove into mass consciousness in World War I and thereafter), and I think there’s a correlation.
Laborers are there to be told. Only top-tier managers are paid to think. But of course, now even top-tier managers at small companies feel the pressure to (appear to) conform, to “Best Practices”; if they are lucky, they will get to preen over how smart they were for the coincidence. If unlucky, well, perhaps the smart ones will say to themselves “what was I missing? What did I not pay attention to?” The rest will just mutter “but I followed Best Practices”, and muddle through.
It’s odd — or maybe it isn’t — that this sort of sentiment seems once again tied to the sentiment that software is trivially a kind of manufacturing, and so, Taylorism, Deming and the Gilbreths are relevant. This is confusing the making of buggies with the driving of them.
Cab drivers do have heuristics, I’m sure, and they probably share them as “cabbie best practices”, especially if they take a new practitioner under wing. But the good cabbies probably don’t sleepwalk through the job much. Because even driving a cab, there are multiple stakeholders and differing definitions of quality. And in that context, those differences can be crucial.
My intended point here is that test is (or can be) integrally part of the quality software creation process, writ large enough; perhaps my analogy about cab driving needs to be elaborated to reflect that. It’s a flawed and partial impression as it stands. Apologies.
Also, I’ll note you can still test software as a customer, after the fact, and such “tourists” still have an impact on the cab-driving industry.
Thanks for the post – I was shrugging exactly that.
I’m a little uneasy about the “a competent practitioner must ” statement, it’s a bit too binary for me. My perception is that we grow such skills over time, I wouldn’t characterise people who are still growing as lacking all competence. Do we ever stop learning because we are now competent?
[James’ Reply: I respect your concern. My desire to write pithily sometimes conflicts with the need to insert the relevant qualifiers. Competence is not binary. There are many skills worth developing, and I continue to develop my own basic testing skills even after all these years. It’s an ongoing process.]
I very much agree about the value of “war stories” – I suspect that the following point may be implicit in this idea, but to reinforce it: I want to hear about the decision making process, the factors that lead to the decisions, the alternatives that were discarded and the reflections on the effectiveness of the choices made.
[James’ Reply: That’s exactly what happens in the peer conferences. Each speaker is questioned extensively (he can’t sit down until the group is finished questioning, no matter, theoretically, how long it takes) about why he did what he did and how he might have done things differently. After attending a few peer conferences, I found that all remaining urges I had to define best practices were swept clean.]
As an aside, here’s another important aspect of true context sensitivity: the “not inflicting help” sort. “I’m helping you, I’m helping you…”
And believe me, do I have war stories about times I tried to inflict help! 🙂
I have a question for you James. What, if anything, in this craft (testing), is not context sensitive?
I have a followup question/comment depends on your answer.
Yan
[James’ Reply: My answer is the people. Anyone who does anything will be, to some degree, insensitive to some aspects of the context in which they act. I’m speaking up for the merits of becoming sensitive to important aspects of context in testing.]
Perhaps my question was unclear. I was trying to ask what in the craft/practice itself, besides people, that stays largely unchanged regardless of the context? Or do you think just about all aspects of testing vary with the context you practice it in?
[James’ Reply: Your question is still unclear, since I don’t know what problem you are trying to address with it. Here are a range of answers… all true:
1. Any aspect of testing behavior may justifiably change due to contextual factors. However, someone may change testing, in response to some context, in such a way that I would no longer recognize it as testing. So, while I would say that anything may change, I’m not saying that I would do any old thing and call it testing, nor that any old thing called testing by someone else would look like testing to me. For instance, if someone told me that in his context, testers are people who serve food to developers, I might say “I call them caterers.”
2. Something I don’t consider testing may be transformed into “testing” in certain situations. For instance, I once watched a Kung Fu movie at 3AM, at work, with the development team, as part of my duties as the test manager on that project. And YES, I have served food to programmers as part of my tester “charm-and-awe” offensive.
3. I may fail to appreciate a contextual detail that should change my practices. For instance, I once blocked a build because of an unauthorized change in the code. When a programmer explained that the unauthorized code was not production code, I realized our build policy should be amended.
4. Different people, with different skills and values, may also differ on how context should relate to practice. Some may change that context (replacing a member of the team, for instance) while others may change the practice (using the test tool preferred by that member of the team instead of the one everyone else wants to use). I have my own testing methodology, which I call “Rapid Testing”, which describes the patterns by which I respond to situations. Rapid testing is itself context-driven, but other people have other methodologies that may be effectively context-specific.]
Those are great examples for things in testing that might change based on context. And I can see that any testing behavior could change. But what I am interested in is what, if anything, stays largely constant, in your craft, regardless of the context, that is unique to the craft?
Methodology may change, behavior may change, but is there anything you consider to be essential that you carry from one context to another?
What makes testing testing?
[James’ Reply: That’s a matter of definition. My working definition of testing is “questioning a product for the purposes of evaluating it.” A little more broadly I say “testers light the way.” Testers do whatever they need to do to dispel the illusions surrounding a product or situation by performing an empirical study of it.
Also, as I move from context to context, one part of the context doesn’t change a whole lot: me. I carry with me my talents, skills, experiences, and toolbox of preferred techniques.]